open:fix memory leak when passing NULL to git_repository_open_ext

apply: Fix a patch corruption related to EOFNL handling

ignore: correct handling of nested rules overriding wild card unignore

Memory allocation fixes for diff generator

Use an HTTP scheme that supports the given credentials

Introduce an unit test to validate that git_apply__patch() properly
handles EOFNL changes in case of patches with several hunks.

problem:
filesystem_iterator loads .gitignore files in top-down order.
subsequently, ignore module evaluates them in the order they are loaded.
this creates a problem if we have unignored a rule (using a wild card)
in a sub dir and ignored it again in a level further below (see the test
included in this patch).

solution:
process ignores in reverse order.

closes #4963

apply: git_apply_to_tree fails to apply patches that add new files

Optionally read `.gitattributes` from HEAD

config: implement "onbranch" conditional

When allocating new tree iterator frames, we zero out the allocated
memory twice. Remove one of the `memset` calls.

When allocating tree iterator entries, we use GIT_ERROR_ALLOC_CHECK` to
check whether the allocation has failed. The macro will cause the
function to immediately return, though, leaving behind a partially
initialized iterator frame.

Fix the issue by manually checking for memory allocation errors and
using `goto done` in case of an error, popping the iterator frame.

When preparing options for the two iterators that are about to be
diffed, we allocate a common prefix for both iterators depending on
the options passed by the user. We do not check whether the allocation
was successful, though. In fact, this isn't much of a problem, as using
a `NULL` prefix is perfectly fine. But in the end, we probably want to
detect that the system doesn't have any memory left, as we're unlikely
to be able to continue afterwards anyway.

While the issue is being fixed in the newly created function
`diff_prepare_iterator_opts`, it has been previously existing in the
previous macro `DIFF_FROM_ITERATORS` already.

While the `DIFF_FROM_ITERATORS` does make it shorter to implement the
various `git_diff_foo_to_bar` functions, it is a complex and unreadable
beast that implicitly assumes certain local variable names. This is not
something desirable to have at all and obstructs understanding and more
importantly debugging the code by quite a bit.

The `DIFF_FROM_ITERATORS` macro basically removed the burden of having
to derive the options for both iterators from a pair of iterator flags
and the diff options. This patch introduces a new function that does the
that exact and refactors all callers to manage the iterators by
themselves.

As we potentially need to allocate a shared prefix for the
iterator, we need to tell the caller to allocate that prefix as soon as
the options aren't required anymore. Thus, the function has a `char
**prefix` out pointer that will get set to the allocated string and
subsequently be free'd by the caller.

While this patch increases the line count, I personally deem this to an
acceptable tradeoff for increased readbiblity.

Fix include casing for case-sensitive filesystems.

When a server responds with multiple scheme support - for example,
Negotiate and NTLM are commonly used together - we need to ensure that
we choose a scheme that supports the credentials.

util: use 64 bit timer on Windows

Memory allocation audit

Our hand-rolled fallback sorting function `git__insertsort_r` does an
in-place sort of the given array. As elements may not necessarily be
pointers, it needs a way of swapping two values of arbitrary size, which
is currently implemented by allocating a temporary buffer of the
element's size. This is problematic, though, as the emulated `qsort`
interface doesn't provide any return values and thus cannot signal an
error if allocation of that temporary buffer has failed.

Convert the function to swap via a temporary buffer allocated on the
stack. Like this, it can `memcpy` contents of both elements in small
batches without requiring a heap allocation. The buffer size has been
chosen such that in most cases, a single iteration of copying will
suffice. Most importantly, it can fully contain `git_oid` structures and
pointers.

Add a bunch of tests for the `git__qsort_r` interface to verify nothing
breaks. Furthermore, this removes the declaration of `git__insertsort_r`
and makes it static as it is not used anywhere else.

The xdiff code contains multiple call sites where the results of
`xdl_malloc` are not being checked for memory allocation errors.

Add checks to fix possible segfaults due to `NULL` pointer accesses.

When allocating a chunk that is used to write to HTTP streams, we do not
check for memory allocation errors. This may lead us to write to a
`NULL` pointer and thus cause a segfault.

Fix this by adding a call to `GIT_ERROR_CHECK_ALLOC`.

The "trailer.c" code has been copied mostly verbatim from git.git with
minor adjustments, only. As git.git's `xmalloc` function, which aborts
on memory allocation errors, has been swapped out for `git_malloc`,
which doesn't abort, we may inadvertently access `NULL` pointers.

Add checks to fix this.

In "posix.c" there are multiple callsites which execute `malloc` instead
of `git__malloc`. Thus, users of library are not able to track these
allocations with a custom allocator.

Convert these call sites to use `git__malloc` instead.

When adding OIDs to the indexer's map of yet-to-be-seen OIDs to verify
that packfiles are complete, we do so by first allocating a new OID and
then calling `git_oidmap_set` on it. There was no check for memory
allocation errors in place, though, leading to possible segfaults due to
trying to copy data to a `NULL` pointer.

Verify the result of `git__malloc` with `GIT_ERROR_CHECK_ALLOC` to fix
the issue.

The function `git_commit_list_insert` dynamically allocates memory and
may thus fail to insert a given commit, but we didn't check for that in
several places in "merge.c".

Convert surrounding functions to return error codes and check whether
`git_commit_list_insert` was successful, returning an error if not.

The code in "blame_git.c" was mostly imported from git.git with only
minor changes. One of these changes was to use our own allocators
instead of git's `xmalloc`, but there's a subtle difference: `xmalloc`
would abort the program if unable to allocate any memory, bit
`git__malloc` doesn't. As we didn't check for memory allocation errors
in some places, we might inadvertently dereference a `NULL` pointer in
out-of-memory situations.

Convert multiple functions to return proper error codes and add calls to
`GIT_ERROR_CHECK_ALLOC` to fix this.

The dummy negotiation scheme is used for known authentication strategies
that do not wish to act.  For example, when a server requests the
"Negotiate" scheme but libgit2 is not built with Negotiate support, and
will use the "dummy" strategy which will simply not act.

Instead of setting `out` to NULL and returning a successful code, return
`GIT_PASSTHROUGH` to indicate that it did not act and catch that error
code.

Introduce an unit test to validate if git_apply_to_tree() fails when an
applied patch adds new files.

git_apply_to_tree() cannot be used apply patches with new files. An attempt
to apply such a patch fails because git_apply_to_tree() tries to remove a
non-existing file from an old index.

The solution is to modify git_apply_to_tree() to git_index_remove() when the
patch states that the modified files is removed.

Use of apply's API can lead to an improper patch application and a corruption
of the modified file.

The issue is caused by mishandling of the end of file changes if there are
several hunks to apply. The new line character is added to a line from a wrong
hunk.

The solution is to modify apply_hunk() to add the newline character at the end
of a line from a right hunk.

clone: don't decode URL percent encodings

Security updates from 0.28.3

The function `commit_quick_parse` provides a way to quickly parse
parts of a commit without storing or verifying most of its
metadata. The first thing it does is calculating the number of
parents by skipping "parent " lines until it finds the first
non-parent line. Afterwards, this parent count is passed to
`alloc_parents`, which will allocate an array to store all the
parent.

To calculate the amount of storage required for the parents
array, `alloc_parents` simply multiplicates the number of parents
with the respective elements's size. This already screams "buffer
overflow", and in fact this problem is getting worse by the
result being cast to an `uint32_t`.

In fact, triggering this is possible: git-hash-object(1) will
happily write a commit with multiple millions of parents for you.
I've stopped at 67,108,864 parents as git-hash-object(1)
unfortunately soaks up the complete object without streaming
anything to disk and thus will cause an OOM situation at a later
point. The point here is: this commit was about 4.1GB of size but
compressed down to 24MB and thus easy to distribute.

The above doesn't yet trigger the buffer overflow, thus. As the
array's elements are all pointers which are 8 bytes on 64 bit, we
need a total of 536,870,912 parents to trigger the overflow to
`0`. The effect is that we're now underallocating the array
and do an out-of-bound writes. As the buffer is kindly provided
by the adversary, this may easily result in code execution.

Extrapolating from the test file with 67m commits to the one with
536m commits results in a factor of 8. Thus the uncompressed
contents would be about 32GB in size and the compressed ones
192MB. While still easily distributable via the network, only
servers will have that amount of RAM and not cause an
out-of-memory condition previous to triggering the overflow. This
at least makes this attack not an easy vector for client-side use
of libgit2.

When the VirtualStore feature is in effect, it is safe to let random
users write into C:\ProgramData because other users won't see those
files. This seemed to be the case when we introduced support for
C:\ProgramData\Git\config.

However, when that feature is not in effect (which seems to be the case
in newer Windows 10 versions), we'd rather not use those files unless
they come from a trusted source, such as an administrator.

This change imitates the strategy chosen by PowerShell's native OpenSSH
port to Windows regarding host key files: if a system file is owned
neither by an administrator, a system account, or the current user, it
is ignored.