OSS-fuzz has reported a potential out-of-bounds read when processing a
"ng" smart packet:

==1==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6310000249c0 at pc 0x000000493a92 bp 0x7ffddc882cd0 sp 0x7ffddc882480
	READ of size 65529 at 0x6310000249c0 thread T0
	SCARINESS: 26 (multi-byte-read-heap-buffer-overflow)
	#0 0x493a91 in __interceptor_strchr.part.35 /src/llvm/projects/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc:673
	#1 0x813960 in ng_pkt libgit2/src/transports/smart_pkt.c:320:14
	#2 0x810f79 in git_pkt_parse_line libgit2/src/transports/smart_pkt.c:478:9
	#3 0x82c3c9 in git_smart__store_refs libgit2/src/transports/smart_protocol.c:47:12
	#4 0x6373a2 in git_smart__connect libgit2/src/transports/smart.c:251:15
	#5 0x57688f in git_remote_connect libgit2/src/remote.c:708:15
	#6 0x52e59b in LLVMFuzzerTestOneInput /src/download_refs_fuzzer.cc:145:9
	#7 0x52ef3f in ExecuteFilesOnyByOne(int, char**) /src/libfuzzer/afl/afl_driver.cpp:301:5
	#8 0x52f4ee in main /src/libfuzzer/afl/afl_driver.cpp:339:12
	#9 0x7f6c910db82f in __libc_start_main /build/glibc-Cl5G7W/glibc-2.23/csu/libc-start.c:291
	#10 0x41d518 in _start

When parsing an "ng" packet, we keep track of both the current position
as well as the remaining length of the packet itself. But instead of
taking care not to exceed the length, we pass the current pointer's
position to `strchr`, which will search for a certain character until
hitting NUL. It is thus possible to create a crafted packet which
doesn't contain a NUL byte to trigger an out-of-bounds read.

Fix the issue by instead using `memchr`, passing the remaining length as
restriction. Furthermore, verify that we actually have enough bytes left
to produce a match at all.

OSS-Fuzz-Issue: 9406

Remove GIT_PKT_PACK entirely

 ignore: improve `git_ignore_path_is_ignored` description Git analogy

alloc: don't overwrite allocator during init if set

If the allocator has been set before we the library is initialised, we would
replace that setting with the standard allocator contrary to the user's wishes.

C90 standard compliance

In attempt to provide adequate Git command analogy in regards to
ignored files handling, `git_ignore_path_is_ignored` description
mentions doing `git add .` on directory containing the file, and
whether the file in question would be added or not - but behavior of
the two matches for untracked files only, making the comparison
misleading in general sense.

For tracked files, Git doesn't subject them to ignore rules, so even
if a rule applies, `git add .` would actually add the tracked file
changes to index, while `git_ignore_path_is_ignored` would still
consider the file being ignored (as it doesn't check the index, as
documented).

Let's provide `git check-ignore --no-index` as analogous Git command
example instead, being more aligned with what `git_ignore_path_is_ignored`
is about, no matter if the file in question is already tracked or not.

See issue #4720 (git_ignore_path_is_ignored documentation
misleading?, 2018-07-10)[1] for additional information.

[1] https://github.com/libgit2/libgit2/issues/4720

While the aim of libgit2 was to conform to C90 code, we never instructed
the compiler to enforce C90 compliance. Thus, quite a few violations
were able to get into our code base, which have been removed with the
previous commits. As we are now able to build libgit2 with C90 enforced,
we can set the C_STANDARD property for our own build targets.

Note that we explicitly avoid setting the C standard for our third-party
dependencies. At least the zlib target does not build with C90 enforced,
and we do not want to fix them by deviating from upstream. Thus we
simply enforce no standard for them.

The mbedtls headers make direct use of the `inline` attribute to
instruct the compiler to inline functions. As this function is not C90
compliant, this can cause the compiler to error as soon as any of these
files is included and the `-std=c90` flag is being added.

The mbedtls headers declaring functions as inline always have a prelude
which define `inline` as a macro in case it is not yet defined. Thus, we
can easily replace their define with our own define, which simply copies
the logic of our own `GIT_INLINE` macro.

While we want to enforce strict C90 mode, this may cause issues with
system provided header files which are themselves not strictly
conforming. E.g. if a system header has C++ style comments, a compiler
in strict C90 mode would produce an error and abort the build. As the
user most likely doesn't want to change the system header, this would
completely break the build on such systems. One example of this is
mbedtls, which provides such header files.

The problem can be worked around by distinguishing between
system-provided and project-provided include directories. When adding
include directories via "-isystem" instead of "-I", the compiler will
skip certain checks and print out less warnings. To use system includes,
we can simply add the "SYSTEM" flag to CMake's `INCLUDE_DIRECTORIES` and
`TARGET_INCLUDE_DIRECTORIES` functions. Note that we have to split the
include directories into two variables because of this, as we definitely
still want to check for all warnings produced by our own header files.

C++ style comment ("//") are not specified by the ISO C90 standard and
thus do not conform to it. While libgit2 aims to conform to C90, we did
not enforce it until now, which is why quite a lot of these
non-conforming comments have snuck into our codebase. Do a tree-wide
conversion of all C++ style comments to the supported C style comments
to allow us enforcing strict C90 compliance in a later commit.

ISO C90 does not specify the `inline` attribute, and as such we cannot
use it in our code. While we already use `__inline` when building in
Microsoft Visual Studio, we should also be using the `__inline__`
attribute from GCC/Clang. Otherwise, if we're using neither MSVC nor
GCC/Clang, we should simply avoid using `inline` at all and just define
functions as static.

This commit adjusts our own `GIT_INLINE` macro as well as the inline
macros specified by khash and xdiff. This allows us to enable strict C90
mode in a later commit.

Delta OOB access

streams: report OpenSSL errors if global init fails

patch_parse: populate line numbers while parsing diffs

Fix git_worktree_validate failing on bare repositories

git_refspec_transform: Handle NULL dst

In case when the global initialization of the OpenSSL stream fails, the
user is left without any hint as to what went wrong as we do not provide
any error message at all. This commit refactors the init function to
have a common error path, which now also sets an error message including
the error string provided by OpenSSL.

When checking whether a delta base offset and length fit into the base
we have in memory already, we can trigger an overflow which breaks the
check. This would subsequently result in us reading memory from out of
bounds of the base.

The issue is easily fixed by checking for overflow when adding `off` and
`len`, thus guaranteeting that we are never indexing beyond `base_len`.
This corresponds to the git patch 8960844a7 (check patch_delta bounds
more carefully, 2006-04-07), which adds these overflow checks.

Reported-by: Riccardo Schirone <rschiron@redhat.com>

Add a "dirty" state to the index when it has unsaved changes

Add the `GIT_OPT_ENABLE_UNSAVED_INDEX_SAFETY` option, which will cause
commands that reload the on-disk index to fail if the current
`git_index` has changed that have not been saved.  This will prevent
users from - for example - adding a file to the index then calling a
function like `git_checkout` and having that file be silently removed
from the index since it was re-read from disk.

Now calls that would re-read the index will fail if the index is
"dirty", meaning changes have been made to it but have not been written.
Users can either `git_index_read` to discard those changes explicitly,
or `git_index_write` to write them.

When the index is dirty, return GIT_EINDEXDIRTY so that consumers can
identify the exact problem programatically.

If the index is dirty, allow `GIT_CHECKOUT_FORCE` to obliterate unsaved
changes.  This is in keeping with its name and description.

Now that the index has a "dirty" state, where it has changes that have
not yet been committed or rolled back, our tests need to be adapted to
actually commit or rollback the changes instead of assuming that the
index can be operated on in its indeterminate state.

Test that any changes to the index will mark the index as dirty.  Also
ensure that when we initialize a new index, read the index contents
from disk, or write the index contents to disk that we reset the dirty
flag to zero.  Further ensure that an unforced read with dirty contents
(when the on-disk index has not changed) does _not_ reset the dirty
flag as we have not updated the contents of our index and our unsaved
contents remain intact.

Teach the index when it is "dirty", and has unsaved changes.  Consider
the index dirty whenever a caller has added or removed an entry from the
main index, REUC or NAME section, including when the index is completely
cleared.  Similarly, consider the index _not_ dirty immediately after it
is written, or when it is read from the on-disk index.

This allows us to ensure that unsaved changes are not lost when we
automatically refresh the index.

Don't manipulate the repository's index during stash; instead,
manipulate a temporary index and check it out.

This allows us to use the checkout mechanism to update the workdir and
the repository's index, and allows checkout to use its common mechanisms
to write data and handle errors.

Add tests that ensure that we re-read the on-disk image by default
during checkout, but when the `GIT_CHECKOUT_NO_REFRESH` option is
specified, we do _not_ re-read the index.