Minor doc improvements

attr: Update definition of binary macro

Security fixes for master

Windows/DOS only supports drive letters that are alpha characters A-Z.
However, you can `subst` any one-character as a drive letter, including
numbers or even emoji.  Test that we can identify emoji as drive
letters.

Users may want to turn off core.protectNTFS, perhaps to import (and then
repair) a broken tree.  Ensure that core.protectNTFS=false is honored.

Enable core.protectNTFS by default everywhere and in every codepath, not
just on checkout.

Test that when we enable core.protectNTFS that we cannot add
platform-specific invalid paths to the index.

The name of the `add_invalid_filename` function suggests that we
_want_ to add an invalid filename.  Rename the function to show that
we expect to _fail_ to add the invalid filename.

Ensure that the new protection around .git::$INDEX_ALLOCATION rules are
enabled for using the treebuilder when core.protectNTFS is set.

Ensure that the new protection around .git::$INDEX_ALLOCATION rules are
enabled for adding to the index when core.protectNTFS is set.

The name of the `write_invalid_filename` function suggests that we
_want_ to write an invalid filename.  Rename the function to show that
we expect to _fail_ to write the invalid filename.

The function `only_spaces_and_dots` used to detect the end of the
filename on win32.  Now we look at spaces and dots _before_ the end of
the string _or_ a `:` character, which would signify a win32 alternate
data stream.

Thus, rename the function `ntfs_end_of_filename` to indicate that it
detects the (virtual) end of a filename, that any further characters
would be elided to the given path.

We just safe-guarded `.git` against NTFS Alternate Data Stream-related
attack vectors, and now it is time to do the same for `.gitmodules`.

Note: In the added regression test, we refrain from verifying all kinds
of variations between short names and NTFS Alternate Data Streams: as
the new code disallows _all_ Alternate Data Streams of `.gitmodules`, it
is enough to test one in order to know that all of them are guarded
against.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

A little-known feature of NTFS is that it offers to store metadata in
so-called "Alternate Data Streams" (inspired by Apple's "resource
forks") that are copied together with the file they are associated with.
These Alternate Data Streams can be accessed via `<file name>:<stream
name>:<stream type>`.

Directories, too, have Alternate Data Streams, and they even have a
default stream type `$INDEX_ALLOCATION`. Which means that `abc/` and
`abc::$INDEX_ALLOCATION/` are actually equivalent.

This is of course another attack vector on the Git directory that we
definitely want to prevent.

On Windows, we already do this incidentally, by disallowing colons in
file/directory names.

While it looks as if files'/directories' Alternate Data Streams are not
accessible in the Windows Subsystem for Linux, and neither via
CIFS/SMB-mounted network shares in Linux, it _is_ possible to access
them on SMB-mounted network shares on macOS.

Therefore, let's go the extra mile and prevent this particular attack
_everywhere_. To keep things simple, let's just disallow *any* Alternate
Data Stream of `.git`.

This is libgit2's variant of CVE-2019-1352.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

The Windows Subsystem for Linux (WSL) is getting increasingly popular,
in particular because it makes it _so_ easy to run Linux software on
Windows' files, via the auto-mounted Windows drives (`C:\` is mapped to
`/mnt/c/`, no need to set that up manually).

Unfortunately, files/directories on the Windows drives can be accessed
via their _short names_, if that feature is enabled (which it is on the
`C:` drive by default).

Which means that we have to safeguard even our Linux users against the
short name attacks.

Further, while the default options of CIFS/SMB-mounts seem to disallow
accessing files on network shares via their short names on Linux/macOS,
it _is_ possible to do so with the right options.

So let's just safe-guard against short name attacks _everywhere_.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

When we expect a checkout operation to fail, but it succeeds, we
actually do not want to see the error messages that were generated in
the meantime for errors that were handled gracefully by the code (e.g.
when an object could not be found in a pack: in this case, the next
backend would have been given a chance to look up the object, and
probably would have found it because the checkout succeeded, after all).

Which means that in the specific case of `cl_git_fail()`, we actually
want to clear the global error state _after_ evaluating the command: we
know that any still-available error would be bogus, seeing as the
command succeeded (unexpectedly).

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

There is no git_stash_apply_flags_t above.

I encountered some problematic URLs,
and was delighted to see that they were already fixed.
I figured I may as well add them to the changelog.

For the record, URLs with no path used to be rejected.
That is arguably correct, but command line git accepts them.

URLs with a path of / and a non-standard port
used to have their port completely ignored!

release.md: note that we do two security releases

Note that for security releases, we update the two most recent major release branches.

MSVC: Fix warning C4133 on x64: "function": Incompatible types - from "unsigned long *" to "size_t *"

MSVC: Fix warning C4133 on x64: "function": Incompatible types - from "unsigned long *" to "size_t *"

Signed-off-by: Sven Strickroth <email@cs-ware.de>

ci: only push docs from the libgit2/libgit2 repo

Users may fork libgit2 and run libgit2's CI on that, which is
delightful!  However, if they do, we'll fail the documentation publish
phase, which is correct (because we don't allow them to publish _their_
version of the docs) but regrettable (since it should not fail).

Only run the documentation publish phase when we merge branches into the
libgit2/libgit2 repo.

global: convert to fiber-local storage to fix exit races

Fix copy&paste in git_cherrypick_commit docstring

patch_parse: fix out-of-bounds reads caused by integer underflow

On Windows platforms, we automatically clean up the thread-local storage
upon detaching a thread via `DllMain()`. The thing is that this happens
for every thread of applications that link against the libgit2 DLL, even
those that don't have anything to do with libgit2 itself. As a result,
we cannot assume that these unsuspecting threads make use of our
`git_libgit2_init()` and `git_libgit2_shutdow()` reference counting,
which may lead to racy situations:

    Thread 1                    Thread 2

    git_libgit2_shutdown()
                                DllMain(DETACH_THREAD)
                                git__free_tls_data()
    git_atomic_dec() == 0
    git__free_tls_data()
    TlsFree(_tls_index)
                                TlsGetValue(_tls_index)

Due to the second thread never having executed `git_libgit2_init()`, the
first thread will clean up TLS data and as a result also free the
`_tls_index` variable. When detaching the second thread, we
unconditionally access the now-free'd `_tls_index` variable, which is
obviously not going to work out well.

Fix the issue by converting the code to use fiber-local storage instead
of thread-local storage. While FLS will behave the exact same as TLS if
no fibers are in use, it does allow us to specify a destructor similar
to the one that is accepted by pthread_key_create(3P). Like this, we do
not have to manually free indices anymore, but will let the FLS handle
calling the destructor. This allows us to get rid of `DllMain()`
completely, as we only used it to keep track of when threads were
exiting and results in an overall simplification of TLS cleanup.

tests: fix compiler warning if tracing is disabled

tests: config: only test parsing huge file with GITTEST_INVASIVE_SPEED

The test in config::stress::huge_section_with_many_values takes quite a
long time to execute. Hide it behind the GITTEST_INVASIVE_SPEED
environment varibale to not needlessly blow up execution time of tests.
As this environment variable is being set by the continuous integration,
we will execute it regularly anyway.

The patch format for binary files is a simple Base85 encoding with a
length byte as prefix that encodes the current line's length. For each
line, we thus check whether the line's actual length matches its
expected length in order to not faultily apply a truncated patch. This
also acts as a check to verify that we're not reading outside of the
line's string:

	if (encoded_len > ctx->parse_ctx.line_len - 1) {
		error = git_parse_err(...);
		goto done;
	}

There is the possibility for an integer underflow, though. Given a line
with a single prefix byte, only, `line_len` will be zero when reaching
this check. As a result, subtracting one from that will result in an
integer underflow, causing us to assume that there's a wealth of bytes
available later on. Naturally, this may result in an out-of-bounds read.

Fix the issue by checking both `encoded_len` and `line_len` for a
non-zero value. The binary format doesn't make use of zero-length lines
anyway, so we need to know that there are both encoded bytes and
remaining characters available at all.

This patch also adds a test that works based on the last error message.
Checking error messages is usually too tightly coupled, but in fact
parsing the patch failed even before the change. Thus the only
possibility is to use e.g. Valgrind, but that'd result in us not
catching issues when run without Valgrind. As a result, using the error
message is considered a viable tradeoff as we know that we didn't start
decoding Base85 in the first place.

If building libgit2's test suite with tracing disabled, then the
compiler will emit a warning due to the unused `message_prefix`
function. Fix the issue by wrapping the whole file into ifdef's for
`GIT_TRACE` and providing separate empty function implementations for
both `cl_global_trace_register` and `cl_global_trace_disable`.

diff: complete support for git patchid

Memory optimizations for config entries

Current implementation of patchid is not computing a correct patchid
when given a patch where, for example, a new file is added or removed.
Some more corner cases need to be handled to have same behavior as git
patch-id command.
Add some more tests to cover those corner cases.

Signed-off-by: Gregory Herrero <gregory.herrero@oracle.com>