When checking out a file, we determine whether the baseline (what we
expect to be in the working directory) actually matches the contents
of the working directory.  This is safe behavior to prevent us from
overwriting changes in the working directory.

We look at the index to optimize this test: if we know that the index
matches the working directory, then we can simply look at the index
data compared to the baseline.

We have historically compared the baseline to the index entry by oid.
However, we must also compare the mode of the two items to ensure that
they are identical.  Otherwise, we will refuse to update the working
directory for a mode change.

A rewritten file can either be classified as a modification of its
contents or of a delete of the complete file followed by an addition of
the new content. This distinction becomes important when we want to
detect renames for rewrites. Given a scenario where a file "a" has been
deleted and another file "b" has been renamed to "a", this should be
detected as a deletion of "a" followed by a rename of "a" -> "b". Thus,
splitting of the original rewrite into a delete/add pair is important
here.

This splitting is represented by a flag we can set at the current delta.
While the flag is already being set in case we want to break rewrites,
we do not do so in case where the `GIT_DIFF_FIND_RENAMES_FROM_REWRITES`
flag is set. This can trigger an assert when we try to match the source
and target deltas.

Fix the issue by setting the `GIT_DIFF_FLAG__TO_SPLIT` flag at the delta
when it is a rename target and `GIT_DIFF_FIND_RENAMES_FROM_REWRITES` is
set.

Add two more scenarios to the "renames" repository. The first scenario
has a major rewrite of a file and a delete of another file, the second
scenario has a deletion of a file and rename of another file to the
deleted file. Both scenarios will be used in the following commit.

While we frequently reuse commit OIDs throughout the file, we do not
have any constants to refer to these commits. Make this a bit easier to
read by giving the commit OIDs somewhat descriptive names of what kind
of commit they refer to.

Our virtual commit must be the last argument to merge-base: since our
algorithm pushes _both_ parents of the virtual commit, it needs to be
the last argument, since merge-base:

> Given three commits A, B and C, git merge-base A B C will compute the
> merge base between A and a hypothetical commit M

We want to calculate the merge base between the actual commit ("two")
and the virtual commit ("one") - since one actually pushes its parents
to the merge-base calculation, we need to calculate the merge base of
"two" and the parents of one.

Virtual base building: ensure that the virtual base is created and
revwalked in the same way as git.

When the commits being merged have multiple merge bases, reverse the
order when creating the virtual merge base.  This is for compatibility
with git's merge-recursive algorithm, and ensures that we build
identical trees.

Git does this to try to use older merge bases first.  Per 8918b0c:

> It seems to be the only sane way to do it: when a two-head merge is
> done, and the merge-base and one of the two branches agree, the
> merge assumes that the other branch has something new.
>
> If we start creating virtual commits from newer merge-bases, and go
> back to older merge-bases, and then merge with newer commits again,
> chances are that a patch is lost, _because_ the merge-base and the
> head agree on it. Unlikely, yes, but it happened to me.

Provide a simple function to reverse an oidarray.

Fixes #4492, #4496.

If an element has been cached, but then the call to
packfile_unpack_compressed() fails, the very next thing that happens is
that its data is freed and then the element is not removed from the
cache, which frees the data again.

This change sets obj->data to NULL to avoid the double-free. It also
stops trying to resolve deltas after two continuous failed rounds of
resolution, and adds a test for this.

When initializing a `git_diff_file_content` from a source whose data is
derived from a blob, we simply assign the blob's pointer to the
resulting struct without incrementing its refcount. Thus, the structure
can only be used as long as the blob is kept alive by the caller.

Fix the issue by using `git_blob_dup` instead of a direct assignment.
This function will increment the refcount of the blob without allocating
new memory, so it does exactly what we want. As
`git_diff_file_content__unload` already frees the blob when
`GIT_DIFF_FLAG__FREE_BLOB` is set, we don't need to add new code
handling the free but only have to set that flag correctly.

Fixes #4440

The function `ERR_error_string` can be invoked without providing a
buffer, in which case OpenSSL will simply return a string printed into a
static buffer. Obviously and as documented in ERR_error_string(3), this
is not thread-safe at all. As libgit2 is a library, though, it is easily
possible that other threads may be using OpenSSL at the same time, which
might lead to clobbered error strings.

Fix the issue by instead using a stack-allocated buffer. According to
the documentation, the caller has to provide a buffer of at least 256
bytes of size. While we do so, make sure that the buffer will never get
overflown by switching to `ERR_error_string_n` to specify the buffer's
size.

The OpenSSL functions `SHA1_Init`, `SHA1_Update` and `SHA1_Final` all
return 1 for success and 0 otherwise, but we never check their return
values. Do so.

The type of |base_offset| in get_delta_base() is `git_off_t`, which is a
signed `long`. That means that we need to make sure that the 8 most
significant bits are zero (instead of 7) to avoid an overflow when it is
shifted by 7 bits.

Found using libFuzzer.

This change fixes an invalid memory access when the trailer is missing /
corrupt.

Found using libFuzzer.

This change ensures that the git_packfile_stream object in
git_indexer_append() does not leak when the stream has errors.

Found using libFuzzer.

While the OpenSSL library provides all means to work safely in a
multi-threaded application, we fail to do so correctly. Quoting from
crypto_lock(3):

    OpenSSL can safely be used in multi-threaded applications provided
    that at least two callback functions are set, locking_function and
    threadid_func.

We do in fact provide the means to set up the locking function via
`git_openssl_set_locking()`, where we initialize a set of locks by using
the POSIX threads API and set the correct callback function to lock and
unlock them.

But what we do not do is setting the `threadid_func` callback. This
function is being used to correctly locate thread-local data of the
OpenSSL library and should thus return per-thread identifiers. Digging
deeper into OpenSSL's documentation, the library does provide a fallback
in case that locking function is not provided by the user. On Windows
and BeOS we should be safe, as it simply "uses the system's default
thread identifying API". On other platforms though OpenSSL will fall
back to using the address of `errno`, assuming it is thread-local.

While this assumption holds true for glibc-based systems, POSIX in fact
does not specify whether it is thread-local or not. Quoting from
errno(3p):

    It is unspecified whether errno is a macro or an identifier declared
    with external linkage.

And in fact, with musl there is at least one libc implementation which
simply declares `errno` as a simple `int` without being thread-local. On
those systems, the fallback threadid function of OpenSSL will not be
thread-safe.

Fix this by setting up our own callback for this setting. As users of
libgit2 may want to set it themselves, we obviously cannot always set
that function on initialization. But as we already set up primitives for
threading in `git_openssl_set_locking()`, this function becomes the
obvious choice where to implement the additional setup.

The macro `DIFF_FLAG_SET` can be used to set or unset a flag by
modifying the diff's bitmask. While the case of setting the flag is
handled correctly, the case of unsetting the flag was not. Instead of
inverting the flags, we are inverting the value which is used to decide
whether we want to set or unset the bits.

The value being used here is a simple `bool` which is `false`. As that
is being uplifted to `int` when getting the bitwise-complement, we will
end up retaining all bits inside of the bitmask. As that's only ever
used to set `GIT_DIFF_IGNORE_CASE`, we were actually always ignoring
case for generated diffs.

Fix that by instead getting the bitwise-complement of `FLAG`, not `VAL`.

In commit 9be638ec (git_diff_generated: abstract generated diffs,
2016-04-19), the code for generated diffs was moved out of the generic
"diff.c" and instead into its own module. During that conversion, it was
forgotten to remove the macros `DIFF_FLAG_IS_SET`, `DIFF_FLAG_ISNT_SET`
and `DIFF_FLAG_SET`, which are now only used in "diff_generated.c".

Remove those macros now.

I'm not sure if worktree.h was intentionally left out of git2.h. Looks like an oversight since it is in fact documented.

Test that we can successfully force checkout a target when the file
contents are identical, but the mode has changed.

On Windows, we do not support file mode changes, so do not test
for type changes between the disk and tree being checked out.

We could have false positives since the on-disk file can only have
an (effective) mode of 0100644 since NTFS does not support executable
files.  If the tree being checked out did have an executable file,
we would erroneously decide that the file on disk had been changed.

When performing a forced checkout, treat files as modified when the
workdir or the index is identical except for the mode.  This ensures
that force checkout will update the mode to the target.  (Apply this
check for regular files only, if one of the items was a file and the
other was another type of item then this would be a typechange and
handled independently.)

When we setup the revision walk for negotiating references with a
remote, we iterate over all references, ignoring tags and symbolic
references. While skipping over symbolic references, we forget to free
the looked up reference, resulting in a memory leak when the next
iteration simply overwrites the variable.

Fix that issue by freeing the reference at the beginning of each
iteration and collapsing return paths for error and success.

If a reference stored in a packed-refs file does not directly point to a
commit, tree or blob, the packed-refs file will also will include a
fully-peeled OID pointing to the first underlying object of that type.
If we try to peel a reference to an object, we will use that peeled OID
to speed up resolving the object.

As a reference for an annotated tag does not directly point to a commit,
tree or blob but instead to the tag object, the packed-refs file will
have an accomodating fully-peeled OID pointing to the object referenced
by that tag. When we use the fully-peeled OID pointing to the referenced
object when peeling, we obviously cannot peel that to the tag anymore.

Fix this issue by not using the fully-peeled OID whenever we want to
peel to a tag. Note that this does not include the case where we want to
resolve to _any_ object type. Existing code may make use from the fact
that we resolve those to commit objects instead of tag objects, even
though that behaviour is inconsistent between packed and loose
references. Furthermore, some tests of ours make the assumption that we
in fact resolve those references to a commit.

When we duplicate a user-provided options struct, we're stuck with freeing the
url in it. In case we add stuff to the proxy struct, let's add a function in
which to put the logic.

If fetch_spec was a non-pattern, and it is not the first iteration of push_status vector, then git_refspec_transform would result in the new value appended via git_buf_puts to the previous iteration value.

Forcibly clearing the buffer on each iteration to prevent this behavior.

If we enter the sha1_position() function with "lo == hi",
we have no elements. But the do-while loop means that we'll
enter the loop body once anyway, picking "mi" at that same
value and comparing nonsense to our desired key. This is
unlikely to match in practice, but we still shouldn't be
looking at the memory in the first place.

This bug is inherited from git.git; it was fixed there in
e01580cfe01526ec2c4eb4899f776a82ade7e0e1.

If the signature is invalid but we detect that after allocating the strings, we
free them. We however leave that pointer dangling in the structure the caller
gave us, which can lead to double-free.

Set these pointers to `NULL` after freeing their memory to avoid this.

When computing negative ignores, we throw away any rule which does not
undo a previous rule to optimize. But on case insensitive file systems,
we need to keep in mind that a negative ignore can also undo a previous
rule with different case, which we did not yet honor while determining
whether a rule undoes a previous one. So in the following example, we
fail to unignore the "/Case" directory:

    /case
    !/Case

Make both paths checking whether a plain- or wildcard-based rule undo a
previous rule aware of case-insensitivity. This fixes the described
issue.