Compare buffers in diff example

Include ahead_behind in the test suite

config: separate file and snapshot backends

object: deprecate git_object__size for removal

In #5118 we remove the double-underscore to make it a normally-named public
function. However, this is not an interesting function outside of the library
and it takes up a name for something that could be more useful.

Remove the single-underscore version as we have not done any releases with it.

The internal backend structures are kind-of legacy and do not really
speak for themselves. Rename them accordingly to make them easier to
understand.

To further distinguish the file writeable and readonly backends,
separate the readonly backend into its own "config_snapshot.c"
implementation. The snapshot backend can be generically used to snapshot
any type of backend.

In `backend_readonly_free`, the passed in config backend is being cast
to a `diskfile_backend` instead of to a `diskfile_readonly_backend`.
While this works out just fine because we only access its header values,
which were shared between both backends, it is undefined behaviour.

Use the correct type to fix this.

The `diskfile_header` structure is shared between both
`diskfile_backend` and `diskfile_readonly_backend`. The separation and
resulting casting is confusing at times and a source for programming
errors.

Remove the shared structure and inline them directly.

While most functions of the readonly configuration backend are
implemented separately from the writeable configuration backend, the two
functions `config_iterator_new` and `config_get` are shared between
both. This sharing makes it necessary to have some shared data
structures, which is the `diskfile_header` structure. Unfortunately, this
makes the backends harder to grasp than necessary due to all the casting
between structs and also quite error prone.

Reimplement those functions for the readonly backends. As readonly
backends cannot be refreshed anyway, we can remove the calls to
`config_refresh` in there.

The `config_readonly_open` function currently receives as input a
diskfile backend and will copy its entries to a new snapshot. This is
rather intimate, as we need to assume that the source config backend is
in fact a diskfile entry. We can do better than this though by using
generic methods to copy contents of the provided backend, e.g. by using
a config iterator. This also allows us to decouple the read-only backend
from the read-write backend.

When duplicating a configuration entry, we allocate a new entry but do
not verify that we get a valid pointer back. As we're dereferencing the
pointer afterwards, we might thus run into a segfault in out-of-memory
situations.

Extract a new function `git_config_entries_dup_entry` that handles the
complete entry duplication. Fix the error by using
`GIT_ERROR_CHECK_ALLOC`.

config_file: refresh when creating an iterator

There was a bug when calling `git_remote_list` that caused us to not
re-read modified configurations when using `git_config_iterator`. This
bug also impacted `git_remote_list`, which thus failed to provide an
up-to-date list of remotes. Add a test suite remote::list with a single
test that verifies we do the right thing.

When creating a new iterator for a config file backend, then we should
always make sure that we're up to date by calling `config_refresh`.
Otherwise, we might not notice when another process has modified the
configuration file and thus will represent outdated values.

Add two tests to config::stress that verify that we get up-to-date
values when reading configuration entries via `git_config_iterator`.

If calling `config_refresh` on a read-only configuration file backend,
then we will segfault when comparing the timestamp of the file due to
`path` being uninitialized. As a read-only snapshot should not be
refreshed anyway and stay consistent, we can simply return early when
calling `config_refresh` on a read-only snapshot.

azure: drop powershell

fuzzer: use futils instead of fileops

w32: fix unlinking of directory symlinks

On most platforms it's fine to create symlinks to nonexisting files. Not
so on Windows, where the type of a symlink (file or directory) needs to
be set at creation time. So depending on whether the target file exists
or not, we may end up with different symlink types. This creates a
problem when performing checkouts, where we simply iterate over all blobs
that need to be updated without treating symlinks any special. If the
target file of the symlink is going to be checked out after the symlink
itself, then the symlink will be created as directory symlink and not as
file symlink.

Fix the issue by iterating over blobs twice: once to perform postponed
deletions and updates to non-symlink blobs, and once to perform updates
to symlink blobs.

When creating a symlink in Windows, one needs to tell Windows whether
the symlink should be a file or directory symlink. To determine which
flag to pass, we call `GetFileAttributesW` on the target file to see
whether it is a directory and then pass the flag accordingly. The
problem though is if create a symlink with a relative target path, then
we will check that relative path while not necessarily being inside of
the working directory where the symlink is to be created. Thus, getting
its attributes will either fail or return attributes of the wrong
target.

Fix this by resolving the target path relative to the directory in which
the symlink is to be created.

Add two more tests to verify that we're not deleting symlink targets,
but the symlinks themselves. Furthermore, convert several `cl_skip`s on
Win32 to conditional skips depending on whether the clar sandbox
supports symlinks or not. Windows is grown up now and may allow
unprivileged symlinks if the machine has been configured accordingly.

Several function calls to `p_stat` and `p_close` have no verification if
they actually succeeded. As these functions _may_ fail and as we also
want to make sure that we're not doing anything dumb, let's check them,
too.

When deleting a symlink on Windows, then the way to delete it depends on
whether it is a directory symlink or a file symlink. In the first case,
we need to use `DeleteFile`, in the second `RemoveDirectory`. Right now,
`p_unlink` will only ever try to use `DeleteFile`, though, and thus fail
to remove directory symlinks. This mismatches how unlink(3P) is expected
to behave, though, as it shall remove any symlink disregarding whether
it is a file or directory symlink.

In order to correctly unlink a symlink, we thus need to check what kind
of file this is. If we were to first query file attributes of every file
upon calling `p_unlink`, then this would penalize the common case
though. Instead, we can try to first delete the file with `DeleteFile`
and only if the error returned is `ERROR_ACCESS_DENIED` will we query
file attributes and determine whether it is a directory symlink to use
`RemoveDirectory` instead.

When initializing a repository, we need to check whether its working
directory supports symlinks to correctly set the initial value of the
"core.symlinks" config variable. The code to check the filesystem is
reusable in other parts of our codebase, like for example in our tests
to determine whether certain tests can be expected to succeed or not.

Extract the code into a new function `git_path_supports_symlinks` to
avoid duplicate implementations. Remove a duplicate implementation in
the repo test helper code.

Our file utils functions all have a "futils" prefix, e.g.
`git_futils_touch`. One would thus naturally guess that their
definitions and implementation would live in files "futils.h" and
"futils.c", respectively, but in fact they live in "fileops.h".

Rename the files to match expectations.

On Win32 builds, the PID file created by git-daemon contained in invalid
PID that we were not able to kill afterwards. Somehow, it seems like the
contained PID was wrapped in braces. Consequentially, kill(1) failed and
thus caused the build to error.

Fix this by directly grabbing the PID of the spawned git-daemon process.

On Win32 build hosts, we do not have an SSH daemon readily available and
thus cannot perform the SSH tests. Let's skip the tests to not let Azure
Pipelines fail.

Right now, we maintain semantically equivalent build scripts in
both Bash and Powershell to support both Windows and non-Windows
hosts. Azure Pipelines supports Bash on Windows, too, via Git for
Windows, and as such it's not really required to maintain the
Powershell scripts at all.

Remove them to reduce our own maintenance burden.

Until now, we always had the CC variable defined in the build.sh
pipeline. But as we're about to migrate the Windows jobs to Bash, as
well, those will not have CC defined and thus we cannot use "$CC" to
determine the compiler version.

Fix this by only executing "$CC" if the variable was set.

We currently specify the CMake generator as part of the CMAKE_OPTIONS
variable. This is fine in the current setup, but during the conversion
to drop PowerShell scripts this will prove problematic for all
generators that have spaces in their names due to quoting issues.

Convert to use an explicit CMAKE_GENERATOR variable that makes it easier
to get quoting right.

We're about to phase out our Powershell scripts as Azure
Pipelines does in fact support Bash scripts on all platforms. As
a preparatory step, let's replace our MinGW setup script with a
Bash script.

Azure Pipelines supports bash tasks on Windows hosts due to it always
having Git for Windows included. To support this, the Git for Window
directory is added to the PATH environment to make the bash shell
available for execution. Unfortunately, this breaks CMake with the MinGW
generator, as it has sanity checks to verify that no bash executable is
in the PATH. So we can either remove Git for Windows from the path, but
then we're unable to execute bash jobs. Or we can add it to the path,
but then we're unable to execute CMake with the MinGW generator.

Let's re-model how we set the PATH environment. Instead of setting up
PATH for the complete build job, we now set a variable "BUILD_PATH" for
the job. This variable is only being used when executing CMake so that
it encounters a sanitizied PATH environment without GfW's bash shell.

Since we have migrated to Azure Pipelines, we have deprecated and
subsequentally removed all infrastructure for AppVeyor and
Travis. Thus it doesn't make a lot of sense to have the split
between "ci/" and "azure-pipelines/" directories anymoer, as
"azure-pipelines/" is essentially our only CI.

Move all CI scripts into the "azure-pipelines/" directory to have
everything centrally located and to remove clutter in the
top-level directory.

Right now, we have an awful hack in our test CI setup that extracts the
test command from CTest's output and then prepends the leak checker.
This is dependent on non-machine-parseable output from CMake and also
breaks on various ocassions, like for example when we have spaces in the
current path or when the path contains backslashes. Both conditions may
easily be triggered on Win32 systems, and in fact they do break our
Azure Pipelines builds.

Remove the awful hack in favour of a new CMake build option
"USE_LEAK_CHECKER". If specifying e.g. "-DUSE_LEAK_CHECKER=valgrind",
then we will set up all tests to be run under valgrind. Like this, we
can again simply execute ctest without needing to rely on evil sourcery.

As different test suites for our CI are mostly defined via CMake, it's
hard to run those tests with a summary file path as that'd require us to
add another parameter to all unit tests. As we do not want to
unconditionally run unit tests with a summary file, we would have to add
another CMake build parameter for test execution, which is ugly.

Instead, implement a way to provide a summary file path via the
environment.

Instead of having to find the fuzzer executables in our Azure test
scripts, provide test targets for each of our fuzzers that will
run them with the correct paths.