Our delta code was originally adapted from JGit, which itself adapted it
from git itself. Due to this heritage, we inherited a bug from git.git
in how we compute the delta offset, which was fixed upstream in
48fb7deb5 (Fix big left-shifts of unsigned char, 2009-06-17). As
explained by Linus:

    Shifting 'unsigned char' or 'unsigned short' left can result in sign
    extension errors, since the C integer promotion rules means that the
    unsigned char/short will get implicitly promoted to a signed 'int' due to
    the shift (or due to other operations).

    This normally doesn't matter, but if you shift things up sufficiently, it
    will now set the sign bit in 'int', and a subsequent cast to a bigger type
    (eg 'long' or 'unsigned long') will now sign-extend the value despite the
    original expression being unsigned.

    One example of this would be something like

            unsigned long size;
            unsigned char c;

            size += c << 24;

    where despite all the variables being unsigned, 'c << 24' ends up being a
    signed entity, and will get sign-extended when then doing the addition in
    an 'unsigned long' type.

    Since git uses 'unsigned char' pointers extensively, we actually have this
    bug in a couple of places.

In our delta code, we inherited such a bogus shift when computing the
offset at which the delta base is to be found. Due to the sign extension
we can end up with an offset where all the bits are set. This can allow
an arbitrary memory read, as the addition in `base_len < off + len` can
now overflow if `off` has all its bits set.

Fix the issue by casting the result of `*delta++ << 24UL` to an unsigned
integer again. Add a test with a crafted delta that would actually
succeed with an out-of-bounds read in case where the cast wouldn't
exist.

Reported-by: Riccardo Schirone <rschiron@redhat.com>
Test-provided-by: Riccardo Schirone <rschiron@redhat.com>

Bugfix release v0.27.2

The test submodule::lookup::duplicated_path, which tries to verify that
we detect submodules with duplicated paths, currently relies on the
gitmodules file of "submod2_target". While this file has two gitmodules
with the same path, one of these gitmodules has an empty name and thus
does not pass `git_submodule_name_is_valid`. Because of this, the test
is in fact dependent on the iteration order in which we process the
submodules. In fact the "valid" submodule comes first, the "invalid"
submodule will cause the desired error. In fact the "invalid" submodule
comes first, it will be skipped due to its name being invalid, and we
will not see the desired error. While this works on the master branch
just right due to the refactoring of our config code, where iteration
order is now deterministic, this breaks on all older maintenance
branches.

Fix the issue by simply using `cl_git_rewritefile` to rewrite the
gitmodules file. This greatly simplifies the test and also makes the
intentions of it much clearer.

When loading submodule names, we build a map of submodule paths and
their respective names. While looping over the configuration keys,
we do not check though whether a submodule path was seen already. This
leads to a memory leak in case we have multiple submodules with the same
path, as we just overwrite the old value in the map in that case.

Fix the error by verifying that the path to be added is not yet part of
the string map. Git does not allow to have multiple submodules for a
path anyway, so we now do the same and detect this duplication,
reporting it to the user.

With the recent change of always resolving pkg-config libraries to their
full path, we do not have to manage the LIBGIT2_LIBDIRS variable
anymore. The only other remaining user of LIBGIT2_LIBDIRS is winhttp,
which is a CMake-style library target and can thus be resolved by CMake
automatically.

Remove the variable to simplify our build system a bit.

Libraries found by CMake modules are usually handled with their full
path. This makes linking against those libraries a lot more robust when
it comes to libraries in non-standard locations, as otherwise we might
mix up libraries from different locations when link directories are
given.

One excemption are libraries found by PKG_CHECK_MODULES. Instead of
returning libraries with their complete path, it will return the
variable names as well as a set of link directories. In case where
multiple sets of the same library are installed in different locations,
this can lead the compiler to link against the wrong libraries in the
end, when link directories of other dependencies are added.

To fix this shortcoming, we need to manually resolve library paths
returned by CMake against their respective library directories. This is
an easy task to do with `FIND_LIBRARY`.

Valgrind log :

==17702== 18 bytes in 1 blocks are indirectly lost in loss record 69 of 1,123
==17702==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==17702==    by 0x5FDBB49: strdup (strdup.c:42)
==17702==    by 0x632B3E: git__strdup (util.h:106)
==17702==    by 0x632D2C: git_reference__alloc_symbolic (refs.c:64)
==17702==    by 0x62E0AF: loose_lookup (refdb_fs.c:408)
==17702==    by 0x62E636: refdb_fs_backend__iterator_next (refdb_fs.c:565)
==17702==    by 0x62CD8E: git_refdb_iterator_next (refdb.c:147)
==17702==    by 0x6347F2: git_reference_next (refs.c:838)
==17702==    by 0x6345CB: git_reference_foreach (refs.c:748)
==17702==    by 0x66BE62: local_download_pack (local.c:579)
==17702==    by 0x5DB48F: git_fetch_download_pack (fetch.c:148)
==17702==    by 0x639028: git_remote_download (remote.c:932)
==17702==    by 0x63919A: git_remote_fetch (remote.c:969)
==17702==    by 0x4ABEDD: test_fetchhead_nonetwork__fetch_into_repo_with_symrefs (nonetwork.c:362)
==17702==    by 0x4125D9: clar_run_test (clar.c:222)
==17702==    by 0x41287C: clar_run_suite (clar.c:286)
==17702==    by 0x412DDE: clar_test_run (clar.c:433)
==17702==    by 0x4105E1: main (main.c:24)

As per CID:1378747, we might be called with a NULL repo, which would be deferenced in write_add_refspec

Signed-off-by: Sven Strickroth <email@cs-ware.de>

CID:1383993, "In git_refspec__dwim_one: All paths that lead to this null pointer comparison already dereference the pointer earlier (CWE-476)"

Commit 723e1e97 (appveyor: disable DHE to avoid spurious failures,
2018-03-29) added a workaround to fix spurious test failures due to a
bug in Windows' SChannel implementation. The workaround only worked by
accident, though, as the registry key was in fact mistyped. Fix the
typo.

Our CI builds have intermittent failures in our online tests, e.g. with
the message "A provided buffer was too small". This is not a programming
error in libgit2 but rather an error in the SChannel component of
Windows. Under certain circumstances involving Diffie-Hellman key
exchange, SChannel is unable to correctly handle input from the server.
This bug has already been fixed in recent patches for Windows 10 and
Windows Server 2016, but they are not yet available for AppVeyor.

Manually pamper over that issue by disabling all ciphersuites using DHE
via the registry. While this disables more ciphers than necessary, we
really don't care for that at all but just want to avoid build failures
due to that bug.

See [1], [2] or [3] for additional information.

1: https://github.com/aws/aws-sdk-cpp/issues/671
2: https://github.com/dotnet/corefx/issues/7812
3: https://support.microsoft.com/en-us/help/2992611/ms14-066-vulnerability-in-schannel-could-allow-remote-code-execution-n

There are multiple references to undefined functions in the Microsoft
builds. Add headers to make them known.

LibreSSL 2.7 adds OpenSSL 1.1 API

Signed-off-by: Bernard Spil <brnrd@FreeBSD.org>

This adds the 'T' status character to git_diff_status_char() for diff
entries that change type.

When we want to limit our graphwalk, we use the heuristic of checking
whether the newest limiting (uninteresting) revision is newer than the
oldest interesting revision. We do so by inspecting whether the first
item's commit time of the user-supplied list of revisions is newer than
the last added interesting revision. This is wrong though, as the user
supplied list is in no way guaranteed to be sorted by increasing commit
dates. This could lead us to abort the revwalk early before applying all
relevant limiting revisions, outputting revisions which should in fact
have been hidden.

Fix the heuristic by instead checking whether _any_ of the limiting
commits was made earlier than the last interesting commit. Add a test.

Signed-off-by: Sven Strickroth <email@cs-ware.de>

This fixes a segfault in git_reference_owner on references returned from git_reference__read_head and git_reference_dup ones.

on 32-bit systems with 64-bit file descriptor offsets enabled
(added -D_FILE_OFFSET_BITS=64 when compiling the test suite)

When comparing whether a path matches a directory rule, we pass the
both the path and directory name to `fnmatch` with
`GIT_ATTR_FNMATCH_DIRECTORY` being set. `fnmatch` expects the pattern to
contain no trailing directory '/', which is why we try to always strip
patterns of trailing slashes. We do not handle that case correctly
though when the pattern itself has trailing spaces, causing the match to
fail.

Fix the issue by stripping trailing spaces and tabs for a rule previous
to checking whether the pattern is a directory pattern with a trailing
'/'. This replaces the whitespace-stripping in our ignore file parsing
code, which was stripping whitespaces too late. Add a test to catch
future breakage.

When fetching into a repository which has symbolic references via the
"local" transport we run into an assert. The assert is being triggered
while we negotiate the packfile between the two repositories. When
hiding known revisions from the packbuilder revwalk, we unconditionally
hide all references of the local refdb. In case one of these references
is a symbolic reference, though, this means we're trying to hide a
`NULL` OID, which triggers the assert.

Fix the issue by only hiding OID references from the revwalk. Add a test
to catch this issue in the future.

When a ODB mempack gets free'd, we take no measures at all to free its
contents, most notably the objects added to the database, resulting in a
memory leak. Call `git_mempack_reset` previous to freeing the ODB
structures themselves, which takes care of releasing all associated
data structures.

This updates the version of SHA1DC to c3e1304ea3.

The function `libssh2_session_startup` has been deprecated since libssh2
version 1.2.8 in favor of `libssh2_session_handshake` introduced in the
same version. libssh2 1.2.8 was released in April 2011, so it is already
seven years old. It is available in Debian Wheezy, Ubuntu Trusty and
CentOS 7.4, so the most important and conservative distros already have
it available. As such, it seems safe to just use the new function.

The function `ssh_stream_free` takes over the responsibility of closing
channels and streams just before freeing their memory, but it does not
do so for the session. In fact, we never disconnect the session
ourselves at all, as libssh2 will not do so itself upon freeing the
structure. Quoting the documentation of `libssh2_session_free`:

    > Frees all resources associated with a session instance. Typically
    > called after libssh2_session_disconnect_ex,

The missing disconnect probably stems from a misunderstanding what it
actually does. As we are already closing the TCP socket ourselves, the
assumption was that no additional disconnect is required. But calling
`libssh2_session_disconnect` will notify the server that we are cleanly
closing the connection, such that the server can free his own resources.

Add a call to `libssh2_session_disconnect` to fix that issue.

[1]: https://www.libssh2.org/libssh2_session_free.html

Backport fixes for CVE 2018-11235

We might modify caches due to us trying to load the configuration to figure out
what kinds of filesystem protections we should have.

We still compare case-insensitively to protect more thoroughly as we don't know
what specifics we'll see on the system and it's the behaviour from git.

When dealing with `core.proectNTFS` and `core.protectHFS` we do check
against `.gitmodules` but we still have a failing test as the non-filesystem
codepath does not check for it.