The flag GIT_FILEBUF_FORCE currently does two things:
     1. It will cause the filebuf to create non-existing leading
        directories for the file that is about to be written.
     2. It will forcibly remove any pre-existing locks.
While most call sites actually do want (1), they do not want to
remove pre-existing locks, as that renders the locking mechanisms
effectively useless.
Introduce a new flag `GIT_FILEBUF_CREATE_LEADING_DIRS` to
separate both behaviours cleanly from each other and convert
callers to use it instead of `GIT_FILEBUF_FORCE` to have them
honor locked files correctly.

As this conversion removes all current users of `GIT_FILEBUF_FORCE`,
this commit removes the flag altogether.

DRY commit parsing

The commit list's in- and out-degrees are currently stored as `unsigned
short`. When assigning it the value of `git_array_size`, which returns
an `size_t`, this generates a warning on some Win32 platforms due to
loosing precision.

We could just cast the returned value of `git_array_size`, which would
work fine for 99.99% of all cases as commits typically have less than
2^16 parents. For crafted commits though we might end up with a wrong
value, and thus we should definitely check whether the array size
actually fits into the field.

To ease the check, let's convert the fields to store the degrees as
`uint16_t`. We shouldn't rely on such unspecific types anyway, as it may
lead to different behaviour across platforms. Furthermore, this commit
introduces a new `git__is_uint16` function to check whether it actually
fits -- if not, we return an error.

This allows us to pick which data from a commit we're interested in.
This will be used by the revwalk code, which is only interested in
parents' and committer data.

azure: avoid building and testing in Docker as root

regexp: implement a new regular expression API

git_refdb API fixes

Don't use enum for flags

Using an `enum` causes trouble when used with C++ as bitwise operations are not possible w/o casting (e.g., `opts.flags &= ~GIT_BLOB_FILTER_CHECK_FOR_BINARY;` is invalid as there is no `&=` operator for `enum`).

Signed-off-by: Sven Strickroth <email@cs-ware.de>

Right now, all tests in libgit2's CI are being executed as root
user. As libgit2 will usually not run as a root user in "normal"
usecases and furthermore as there are tests that rely on the
ability to _not_ be able to create certain paths, let's instead
create an unprivileged user "libgit2" and use that across all
docker images.

valgrind: suppress memory leaks in libssh2_session_handshake

On Ubuntu, the combination of libgcrypt and libssh2 is quite old and
known to contain memory leaks. We thus have several functions listed in
our suppressions file that are known to leak. Due to a recent update of
libssh2 or libgcrypt, there now are new memory leaks caused by
libssh2_session_handshake and libssh2_init that cause the CI to fail.

Add a new suppression to fix the issue.

buffer: fix writes into out-of-memory buffers

The old POSIX regex wrappers have been superseded by our own regexp API
that provides a higher-level abstraction. Remove the POSIX wrappers in
favor of the new one.

The old POSIX regex API has been superseded by our new regexp API.
Convert all users to make use of the new one.

We currently support a set of different regular expression backends with
PCRE, PCRE2, regcomp(3P) and regcomp_l(3). The current implementation of
this is done via a simple POSIX wrapper that either directly uses
supplied functions or that is a very small wrapper.

To support PCRE and PCRE2, we use their provided <pcreposix.h> and
<pcre2posix.h> wrappers. These wrappers are implemented in such a way
that the accompanying libraries pcre-posix and pcre2-posix provide the
same symbols as the libc ones, namely regcomp(3P) et al. This works out
on some systems just fine, most importantly on glibc-based ones, where
the regular expression functions are implemented as weak aliases and
thus get overridden by linking in the pcre{,2}-posix library. On other
systems we depend on the linking order of libc and pcre library, and as
libc always comes first we will end up with the functions of the libc
implementation. As a result, we may use the structures `regex_t` and
`regmatch_t` declared by <pcre{,2}posix.h>, but use functions defined by
the libc, leading to segfaults.

The issue is not easily solvable. Somed distributions like Debian have
resolved this by patching PCRE and PCRE2 to carry custom prefixes to all
the POSIX function wrappers. But this is not supported by upstream and
thus inherently unportable between distributions. We could instead try
to modify linking order, but this starts becoming fragile and will not
work e.g. when libgit2 is loaded via dlopen(3P) or similar ways. In the
end, this means that we simply cannot use the POSIX wrappers provided by
the PCRE libraries at all.

Thus, this commit introduces a new regular expression API. The new API
is on a tad higher level than the previous POSIX abstraction layer, as
it tries to abstract away any non-portable flags like e.g. REG_EXTENDED,
which has no equivalents in all of our supported backends. As there are
no users of POSIX regular expressions that do _not_ reguest REG_EXTENDED
this is fine to be abstracted away, though. Due to the API being
higher-level than before, it should generally be a tad easier to use
than the previous one.

Note: ideally, the new API would've been called `git_regex_foobar` with
a file "regex.h" and "regex.c". Unfortunately, this is currently
impossible to implement due to naming clashes between the then-existing
"regex.h" and <regex.h> provided by the libc. As we add the source
directory of libgit2 to the header search path, an include of <regex.h>
would always find our own "regex.h". Thus, we have to take the bitter
pill of adding one more character to all the functions to disambiguate
the includes.

To improve guarantees around cross-backend compatibility, this commit
also brings along an improved regular expression test suite
core::regexp.

Before printing into a `git_buf` structure, we always call `ENSURE_SIZE`
first. This macro will reallocate the buffer as-needed depending on
whether the current amount of allocated bytes is sufficient or not. If
`asize` is big enough, then it will just do nothing, otherwise it will
call out to `git_buf_try_grow`. But in fact, it is insufficient to only
check `asize`.

When we fail to allocate any more bytes e.g. via `git_buf_try_grow`,
then we set the buffer's pointer to `git_buf__oom`. Note that we touch
neither `asize` nor `size`. So if we just check `asize > targetsize`,
then we will happily let the caller of `ENSURE_SIZE` proceed with an
out-of-memory buffer. As a result, we will print all bytes into the
out-of-memory buffer instead, resulting in an out-of-bounds write.

Fix the issue by having `ENSURE_SIZE` verify that the buffer is not
marked as OOM. Add a test to verify that we're not writing into the OOM
buffer.

When growing buffers, we repeatedly multiply the currently allocated
number of bytes by 1.5 until it exceeds the requested number of bytes.
This has two major problems:

    1. If the current number of bytes is tiny and one wishes to resize
       to a comparatively huge number of bytes, then we may need to loop
       thousands of times.

    2. If resizing to a value close to `SIZE_MAX` (which would fail
       anyway), then we probably hit an infinite loop as multiplying the
       current amount of bytes will repeatedly result in integer
       overflows.

When reallocating buffers, one typically chooses values close to 1.5 to
enable re-use of resulting memory holes in later reallocations. But
because of this, it really only makes sense to use a factor of 1.5
_once_, but not looping until we finally are able to fit it. Thus, we
can completely avoid the loop and just opt for the much simpler
algorithm of multiplying with 1.5 once and, if the result doesn't fit,
just use the target size. This avoids both problems of looping
extensively and hitting overflows.

This commit also adds a test that would've previously resulted in an
infinite loop.

If growing a buffer fails, we set its pointer to the static
`git_buf__oom` structure. While we correctly free the old pointer if
`git__malloc` returned an error, we do not free it if there was an
integer overflow while calculating the new allocation size. Fix this
issue by freeing the pointer to plug the memory leak.

cred: add missing private header in GSSAPI block

Should have been part of 8bf0f7eb

CMake pkg-config modulification

Update chat resources in README.md

Circular header splitting

azure: build Docker images as part of the pipeline

smart: use push_glob instead of manual filtering

The MESSAGE() function expects as first argument the message type, e.g.
STATUS or FATAL_ERROR. In some places, we were misusing this to either
not provide any type, which would then erroneously print the message to
standard error, or to use FATAL instead of FATAL_ERROR.

Fix all of these instances. Also, remove some MESSAGE invocations that
are obvious leftovers from debugging the build system.

When using mbedTLS as backend, then the user may specify the location of
where system certificates are installed. If no such location is provided
by the user, CMake will try to autodetect the location by using the
openssl executable, if installed. If no location could be detected, then
the mbedTLS is essentially worthless as it is completely unable to
verify any certificates.

To avoid use of such misconfigured mbedTLS configurations, let's error
out if we were unable to find out the location.

While we were still supporting Trusty, using Ninja as a build tool would
have required us to first setup pip and then use it to install Ninja.
As a result, the speedups from using Ninja were drowned out by the
time required to install Ninja. But as we have deprecated Trusty now,
both Xenial and Bionic have recent versions of Ninja in their
repositories and thus we can now use Ninja.

The Valgrind version shipped with Xenial has some bugs that keep our
tests from working due to bad interactions with openssl [1]. Fix this by
using the "hola-launchpad/valgrind" PPA that provides a newer version
for which the bug has been fixed.

[1]: https://bugs.launchpad.net/ubuntu/+source/valgrind/+bug/1574437

Pass the flag "--no-install-recommends" to apt-get in order to trim down
the number of packages installed, both reducing build time and image
size. As this also causes some required packages to not be installed
anymore, add these explicitly to the set of packages installed.

Reformat both Xenial and Bionic's Dockerfiles to use best practices.
Most importantly, we now run `apt-get update` and `apt-get install` in
one step followed up by removing the package lists to speed up
installation and keep down the image size.

While Xenial provides libssh2 in its repositories, it only has version
1.5.0 available. This version will unfortunately not be able to connect
to GitHub due to their removal of weak cryptographic standards [1]. To
still enable our CI to execute tests against GitHub, we thus have to
update the provided libssh2 version to a newer one.

Manually install libssh2 1.8.2 on Xenial. There's no need to do the same
for Bionic, as it already provides libssh2 1.8.0.

[1]: https://github.blog/2018-02-01-crypto-removal-notice/