The patch ID is supposed to be mostly context-insignificant and
thus only includes added or deleted lines. As such, we shouldn't honor
end-of-file-without-newline markers in diffs.

Ignore such lines to fix how we compute the patch ID for such diffs.

When parsing a patch from a buffer, we let the patch lines point into
the original buffer. While this is efficient use of resources, this also
ties the lifetime of the parsed patch to the parsed buffer. As this
behaviour is not documented anywhere in our API it is very surprising to
its users.

Untie the lifetime by duplicating the lines into the parsed patch. Add a
test that verifies that lifetimes are indeed independent of each other.

Our online tests are occasionally flaky since they hit real network
endpoints.  Re-run them up to 5 times if they fail, to allow us to
avoid having to fail the whole build.

Our online tests are occasionally flaky since they hit real network
endpoints.  Re-run them up to 5 times if they fail, to allow us to
avoid having to fail the whole build.

Previously, we would fail to correctly truncate the source buffer
if the source has more than one line and ends with a non-newline
character. In the following call, we thus truncate the source
string in the middle of the second line. Without the bug fixed,
we would successfully apply the patch to the source and return
success. With the overflow being fixed, we should return an
error now.

When parsing the patch image from a string, we split the string
by newlines to get a line-based view of it. To split, we use
`memchr` on the buffer and limit the buffer length by the
original length provided by the caller. This works just fine for
the first line, but for every subsequent line we need to actually
subtract the amount of bytes that we have already read.

The above issue can be easily triggered by having a source buffer
with at least two lines, where the second line does _not_ end in
a newline. Given a string "foo\nb", we have an original length of
five bytes. After having extracted the first line, we will point
to 'b' and again try to `memchr(p, '\n', 5)`, resulting in an
out-of-bounds read of four bytes.

Fix the issue by correctly subtracting the amount of bytes
already read.

Security release v0.28.4

Windows/DOS only supports drive letters that are alpha characters A-Z.
However, you can `subst` any one-character as a drive letter, including
numbers or even emoji.  Test that we can identify emoji as drive
letters.

Users may want to turn off core.protectNTFS, perhaps to import (and then
repair) a broken tree.  Ensure that core.protectNTFS=false is honored.

Enable core.protectNTFS by default everywhere and in every codepath, not
just on checkout.

Test that when we enable core.protectNTFS that we cannot add
platform-specific invalid paths to the index.

The name of the `add_invalid_filename` function suggests that we
_want_ to add an invalid filename.  Rename the function to show that
we expect to _fail_ to add the invalid filename.

Ensure that the new protection around .git::$INDEX_ALLOCATION rules are
enabled for using the treebuilder when core.protectNTFS is set.

Ensure that the new protection around .git::$INDEX_ALLOCATION rules are
enabled for adding to the index when core.protectNTFS is set.

The name of the `write_invalid_filename` function suggests that we
_want_ to write an invalid filename.  Rename the function to show that
we expect to _fail_ to write the invalid filename.

The function `only_spaces_and_dots` used to detect the end of the
filename on win32.  Now we look at spaces and dots _before_ the end of
the string _or_ a `:` character, which would signify a win32 alternate
data stream.

Thus, rename the function `ntfs_end_of_filename` to indicate that it
detects the (virtual) end of a filename, that any further characters
would be elided to the given path.

We just safe-guarded `.git` against NTFS Alternate Data Stream-related
attack vectors, and now it is time to do the same for `.gitmodules`.

Note: In the added regression test, we refrain from verifying all kinds
of variations between short names and NTFS Alternate Data Streams: as
the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it
is enough to test one in order to know that all of them are guarded
against.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

A little-known feature of NTFS is that it offers to store metadata in
so-called "Alternate Data Streams" (inspired by Apple's "resource
forks") that are copied together with the file they are associated with.
These Alternate Data Streams can be accessed via `<file name>:<stream
name>:<stream type>`.

Directories, too, have Alternate Data Streams, and they even have a
default stream type `$INDEX_ALLOCATION`. Which means that `abc/` and
`abc::$INDEX_ALLOCATION/` are actually equivalent.

This is of course another attack vector on the Git directory that we
definitely want to prevent.

On Windows, we already do this incidentally, by disallowing colons in
file/directory names.

While it looks as if files'/directories' Alternate Data Streams are not
accessible in the Windows Subsystem for Linux, and neither via
CIFS/SMB-mounted network shares in Linux, it _is_ possible to access
them on SMB-mounted network shares on macOS.

Therefore, let's go the extra mile and prevent this particular attack
_everywhere_. To keep things simple, let's just disallow *any* Alternate
Data Stream of `.git`.

This is libgit2's variant of CVE-2019-1352.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The Windows Subsystem for Linux (WSL) is getting increasingly popular,
in particular because it makes it _so_ easy to run Linux software on
Windows' files, via the auto-mounted Windows drives (`C:\` is mapped to
`/mnt/c/`, no need to set that up manually).

Unfortunately, files/directories on the Windows drives can be accessed
via their _short names_, if that feature is enabled (which it is on the
`C:` drive by default).

Which means that we have to safeguard even our Linux users against the
short name attacks.

Further, while the default options of CIFS/SMB-mounts seem to disallow
accessing files on network shares via their short names on Linux/macOS,
it _is_ possible to do so with the right options.

So let's just safe-guard against short name attacks _everywhere_.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

When we expect a checkout operation to fail, but it succeeds, we
actually do not want to see the error messages that were generated in
the meantime for errors that were handled gracefully by the code (e.g.
when an object could not be found in a pack: in this case, the next
backend would have been given a chance to look up the object, and
probably would have found it because the checkout succeeded, after all).

Which means that in the specific case of `cl_git_fail()`, we actually
want to clear the global error state _after_ evaluating the command: we
know that any still-available error would be bogus, seeing as the
command succeeded (unexpectedly).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The function `commit_quick_parse` provides a way to quickly parse
parts of a commit without storing or verifying most of its
metadata. The first thing it does is calculating the number of
parents by skipping "parent " lines until it finds the first
non-parent line. Afterwards, this parent count is passed to
`alloc_parents`, which will allocate an array to store all the
parent.

To calculate the amount of storage required for the parents
array, `alloc_parents` simply multiplicates the number of parents
with the respective elements's size. This already screams "buffer
overflow", and in fact this problem is getting worse by the
result being cast to an `uint32_t`.

In fact, triggering this is possible: git-hash-object(1) will
happily write a commit with multiple millions of parents for you.
I've stopped at 67,108,864 parents as git-hash-object(1)
unfortunately soaks up the complete object without streaming
anything to disk and thus will cause an OOM situation at a later
point. The point here is: this commit was about 4.1GB of size but
compressed down to 24MB and thus easy to distribute.

The above doesn't yet trigger the buffer overflow, thus. As the
array's elements are all pointers which are 8 bytes on 64 bit, we
need a total of 536,870,912 parents to trigger the overflow to
`0`. The effect is that we're now underallocating the array
and do an out-of-bound writes. As the buffer is kindly provided
by the adversary, this may easily result in code execution.

Extrapolating from the test file with 67m commits to the one with
536m commits results in a factor of 8. Thus the uncompressed
contents would be about 32GB in size and the compressed ones
192MB. While still easily distributable via the network, only
servers will have that amount of RAM and not cause an
out-of-memory condition previous to triggering the overflow. This
at least makes this attack not an easy vector for client-side use
of libgit2.

When the VirtualStore feature is in effect, it is safe to let random
users write into C:\ProgramData because other users won't see those
files. This seemed to be the case when we introduced support for
C:\ProgramData\Git\config.

However, when that feature is not in effect (which seems to be the case
in newer Windows 10 versions), we'd rather not use those files unless
they come from a trusted source, such as an administrator.

This change imitates the strategy chosen by PowerShell's native OpenSSH
port to Windows regarding host key files: if a system file is owned
neither by an administrator, a system account, or the current user, it
is ignored.

Release v0.28.2

(cherry picked from commit 0fd259ed)

(cherry picked from commit 336e98bb)

(cherry picked from commit 37a7adb5)

Opening a default config when ~/.gitconfig doesn't exist, locking it,
and attempting to write to it causes an assertion failure.

Treat non-existent global config file content as an empty string.

(cherry picked from commit e44110db)

When linking against bundled libraries, we include their header
directories by using "-isystem". The reason for that is that we
want to handle our vendored library headers specially, most
importantly to ignore warnings generated by including them. By
using "-isystem", though, we screw up the order of searched
include directories by moving those bundled dependencies towards
the end of the lookup order. Like this, chances are high that any
other specified include directory contains a file that collides
with the actual desired include file.

Fix this by not treating the bundled dependencies' include
directories as system includes. This will move them to the front
of the lookup order and thus cause them to override
system-provided headers. While this may cause the compiler to
generate additional warnings when processing bundled headers,
this is a tradeoff we should make regardless to fix builds on
systems hitting this issue.

(cherry picked from commit ee3d71fb)

We assume that if we are on Win32, Amiga OS, Solaris or SunOS,
that the regcomp(3P) function cannot be provided by the system.
Thus we will in these cases always include our own, bundled regex
sources to make a regcomp implementation available. This test is
obviously very fragile, and we have seen it fail on MSYS2/MinGW
systems, which do in fact provide the regcomp symbol. The effect
is that during compilation, we will use the "regex.h" header
provided by MinGW, but use symbols provided by ourselves. This
in fact may cause subtle memory layout issues, as the structure
made available via MinGW doesn't match what our bundled code
expects.

There's one more problem with our regex detection: on the listed
platforms, we will incorrectly include the bundled regex code
even in case where the system provides regcomp_l(3), but it will
never be used for anything.

Fix the issue by improving our regcomp detection code. Instead of
relying on a fragile listing of platforms, we can just use
`CHECK_FUNCTION_EXISTS` instead. This will not in fact avoid the
header-ordering problem. But we can assume that as soon as a
system-provided "regex.h" header is provided, that
`CHECK_FUNCTION_EXISTS` will now correctly find the desired
symbol and thus not include our bundled regex code.

(cherry picked from commit 13cb9f7a)

git_array_alloc can return NULL if no memory is available, causing
a segmentation fault in memset. This adds GIT_ERROR_CHECK_ALLOC
similar to how other parts of the code base deal with the return
value of git_array_alloc.

Stop traversing the filesystem at the Windows directory root.  We were
calculating the filesystem root for the given directory to create, and
walking up the filesystem hierarchy.  We intended to stop when the
traversal path length is equal to the root path length (ie, stopping at
the root, since no path may be shorter than the root path).

However, on Windows, the root path may be specified in two different
ways, as either `Z:` or `Z:\`, where `Z:` is the current drive letter.
`git_path_dirname_r` returns the path _without_ a trailing slash, even
for the Windows root.  As a result, during traversal, we need to test
that the traversal path is _less than or equal to_ the root path length
to determine if we've hit the root to ensure that we stop when our
traversal path is `Z:` and our calculated root path was `Z:\`.

The function `git_ignore_path_is_ignored` is there to test the
ignore status of paths that need not necessarily exist inside of
a repository. This has the implication that for a given path, we
cannot always decide whether it references a directory or a file,
and we need to distinguish those cases because ignore rules may
treat those differently. E.g. given the following gitignore file:

    *
    !/**/

we'd only want to unignore directories, while keeping files
ignored. But still, calling `git_ignore_path_is_ignored("dir/")`
will say that this directory is ignored because it treats "dir/"
as a file path.

As said, the `is_ignored` function cannot always decide whether
the given path is a file or directory, and thus it may produce
wrong results in some cases. While this is unfixable in the
general case, we can do better when we are being passed a path
name with a trailing path separator (e.g. "dir/") and always
treat them as directories.

Add a test that verifies that we are able to parse patches which add a
new file that has spaces in its path.