While we were still supporting Trusty, using Ninja as a build tool would
have required us to first setup pip and then use it to install Ninja.
As a result, the speedups from using Ninja were drowned out by the
time required to install Ninja. But as we have deprecated Trusty now,
both Xenial and Bionic have recent versions of Ninja in their
repositories and thus we can now use Ninja.

The Valgrind version shipped with Xenial has some bugs that keep our
tests from working due to bad interactions with openssl [1]. Fix this by
using the "hola-launchpad/valgrind" PPA that provides a newer version
for which the bug has been fixed.

[1]: https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1574437

Pass the flag "--no-install-recommends" to apt-get in order to trim down
the number of packages installed, both reducing build time and image
size. As this also causes some required packages to not be installed
anymore, add these explicitly to the set of packages installed.

Reformat both Xenial and Bionic's Dockerfiles to use best practices.
Most importantly, we now run `apt-get update` and `apt-get install` in
one step followed up by removing the package lists to speed up
installation and keep down the image size.

While Xenial provides libssh2 in its repositories, it only has version
1.5.0 available. This version will unfortunately not be able to connect
to GitHub due to their removal of weak cryptographic standards [1]. To
still enable our CI to execute tests against GitHub, we thus have to
update the provided libssh2 version to a newer one.

Manually install libssh2 1.8.2 on Xenial. There's no need to do the same
for Bionic, as it already provides libssh2 1.8.0.

[1]: https://github.blog/2018-02-01-crypto-removal-notice/

We're about to phase out support for Trusty, but neither Bionic nor
Xenial images provide the mbedTLS library that's available in Trusty.
Build them for both to pull them in line with Trusty.

The Coverity build is still referencing an old "trusty-openssl"
container that is not provided by either our own now-inlined images nor
by the libgit2/libgit2-docker repository.

Convert it to build and use Xenial images instead.

Support for the LTS release Ubuntu 14.04 Trusty has been dropped in
April 2019, but Azure is still using Trusty as its primary platform to
build and test against. Let's deprecate it in favor of Xenial.

The Docker images used for our continuous integration builds currently
live in the libgit2/libgit2-docker repository. To make any changes in
them, one has to make a PR there, get it reviewed, re-build the images
and publish them to Docker Hub. This process is slow and tedious, making
it harder than necessary to perform any updates to our Docker-based
build pipeline.

To fix this, we include all Dockerfiles used by Azure from the mentioned
repository and inline them into our own repo. Instead of having to
manually push them to the CI, it will now build the required containers
on each pull request, allowing much greater flexibility.

cmake: remove extraneous logging

open:fix memory leak when passing NULL to git_repository_open_ext

apply: Fix a patch corruption related to EOFNL handling

ignore: correct handling of nested rules overriding wild card unignore

Memory allocation fixes for diff generator

Use an HTTP scheme that supports the given credentials

Introduce an unit test to validate that git_apply__patch() properly
handles EOFNL changes in case of patches with several hunks.

problem:
filesystem_iterator loads .gitignore files in top-down order.
subsequently, ignore module evaluates them in the order they are loaded.
this creates a problem if we have unignored a rule (using a wild card)
in a sub dir and ignored it again in a level further below (see the test
included in this patch).

solution:
process ignores in reverse order.

closes #4963

apply: git_apply_to_tree fails to apply patches that add new files

Optionally read `.gitattributes` from HEAD

config: implement "onbranch" conditional

When allocating new tree iterator frames, we zero out the allocated
memory twice. Remove one of the `memset` calls.

When allocating tree iterator entries, we use GIT_ERROR_ALLOC_CHECK` to
check whether the allocation has failed. The macro will cause the
function to immediately return, though, leaving behind a partially
initialized iterator frame.

Fix the issue by manually checking for memory allocation errors and
using `goto done` in case of an error, popping the iterator frame.

When preparing options for the two iterators that are about to be
diffed, we allocate a common prefix for both iterators depending on
the options passed by the user. We do not check whether the allocation
was successful, though. In fact, this isn't much of a problem, as using
a `NULL` prefix is perfectly fine. But in the end, we probably want to
detect that the system doesn't have any memory left, as we're unlikely
to be able to continue afterwards anyway.

While the issue is being fixed in the newly created function
`diff_prepare_iterator_opts`, it has been previously existing in the
previous macro `DIFF_FROM_ITERATORS` already.

While the `DIFF_FROM_ITERATORS` does make it shorter to implement the
various `git_diff_foo_to_bar` functions, it is a complex and unreadable
beast that implicitly assumes certain local variable names. This is not
something desirable to have at all and obstructs understanding and more
importantly debugging the code by quite a bit.

The `DIFF_FROM_ITERATORS` macro basically removed the burden of having
to derive the options for both iterators from a pair of iterator flags
and the diff options. This patch introduces a new function that does the
that exact and refactors all callers to manage the iterators by
themselves.

As we potentially need to allocate a shared prefix for the
iterator, we need to tell the caller to allocate that prefix as soon as
the options aren't required anymore. Thus, the function has a `char
**prefix` out pointer that will get set to the allocated string and
subsequently be free'd by the caller.

While this patch increases the line count, I personally deem this to an
acceptable tradeoff for increased readbiblity.

Fix include casing for case-sensitive filesystems.

When a server responds with multiple scheme support - for example,
Negotiate and NTLM are commonly used together - we need to ensure that
we choose a scheme that supports the credentials.

util: use 64 bit timer on Windows

Memory allocation audit

Our hand-rolled fallback sorting function `git__insertsort_r` does an
in-place sort of the given array. As elements may not necessarily be
pointers, it needs a way of swapping two values of arbitrary size, which
is currently implemented by allocating a temporary buffer of the
element's size. This is problematic, though, as the emulated `qsort`
interface doesn't provide any return values and thus cannot signal an
error if allocation of that temporary buffer has failed.

Convert the function to swap via a temporary buffer allocated on the
stack. Like this, it can `memcpy` contents of both elements in small
batches without requiring a heap allocation. The buffer size has been
chosen such that in most cases, a single iteration of copying will
suffice. Most importantly, it can fully contain `git_oid` structures and
pointers.

Add a bunch of tests for the `git__qsort_r` interface to verify nothing
breaks. Furthermore, this removes the declaration of `git__insertsort_r`
and makes it static as it is not used anywhere else.

The xdiff code contains multiple call sites where the results of
`xdl_malloc` are not being checked for memory allocation errors.

Add checks to fix possible segfaults due to `NULL` pointer accesses.

When allocating a chunk that is used to write to HTTP streams, we do not
check for memory allocation errors. This may lead us to write to a
`NULL` pointer and thus cause a segfault.

Fix this by adding a call to `GIT_ERROR_CHECK_ALLOC`.

The "trailer.c" code has been copied mostly verbatim from git.git with
minor adjustments, only. As git.git's `xmalloc` function, which aborts
on memory allocation errors, has been swapped out for `git_malloc`,
which doesn't abort, we may inadvertently access `NULL` pointers.

Add checks to fix this.

In "posix.c" there are multiple callsites which execute `malloc` instead
of `git__malloc`. Thus, users of library are not able to track these
allocations with a custom allocator.

Convert these call sites to use `git__malloc` instead.

When adding OIDs to the indexer's map of yet-to-be-seen OIDs to verify
that packfiles are complete, we do so by first allocating a new OID and
then calling `git_oidmap_set` on it. There was no check for memory
allocation errors in place, though, leading to possible segfaults due to
trying to copy data to a `NULL` pointer.

Verify the result of `git__malloc` with `GIT_ERROR_CHECK_ALLOC` to fix
the issue.

The function `git_commit_list_insert` dynamically allocates memory and
may thus fail to insert a given commit, but we didn't check for that in
several places in "merge.c".

Convert surrounding functions to return error codes and check whether
`git_commit_list_insert` was successful, returning an error if not.