Execute stale tests

hash: openssl: check return values of SHA1_* functions

diff_generate: avoid excessive stats of .gitattribute files

When generating a diff between two trees, for each file that is to be
diffed we have to determine whether it shall be treated as text or as
binary files. While git has heuristics to determine which kind of diff
to generate, users can also that default behaviour by setting or
unsetting the 'diff' attribute for specific files.

Because of that, we have to query gitattributes in order to determine
how to diff the current files. Instead of hitting the '.gitattributes'
file every time we need to query an attribute, which can get expensive
especially on networked file systems, we try to cache them instead. This
works perfectly fine for every '.gitattributes' file that is found, but
we hit cache invalidation problems when we determine that an attribuse
file is _not_ existing. We do create an entry in the cache for missing
'.gitattributes' files, but as soon as we hit that file again we
invalidate it and stat it again to see if it has now appeared.

In the case of diffing large trees with each other, this behaviour is
very suboptimal. For each pair of files that is to be diffed, we will
repeatedly query every directory component leading towards their
respective location for an attributes file. This leads to thousands or
even hundreds of thousands of wasted syscalls.

The attributes cache already has a mechanism to help in that scenario in
form of the `git_attr_session`. As long as the same attributes session
is still active, we will not try to re-query the gitmodules files at all
but simply retain our currently cached results. To fix our problem, we
can create a session at the top-most level, which is the initialization
of the `git_diff` structure, and use it in order to look up the correct
diff driver. As the `git_diff` structure is used to generate patches for
multiple files at once, this neatly solves our problem by retaining the
session until patches for all files have been generated.

The fix has been tested with linux.git by calling
`git_diff_tree_to_tree` and `git_diff_to_buf` with v4.10^{tree} and
v4.14^{tree}.

                | time    | .gitattributes stats
    without fix | 33.201s | 844614
    with fix    | 30.327s | 4441

While execution only improved by roughly 10%, the stat(3) syscalls for
.gitattributes files decreased by 99.5%. The benchmarks were quite
simple with best-of-three timings on Linux ext4 systems. One can assume
that for network based file systems the performance gain will be a lot
larger due to a much higher latency.

cmake: create a dummy file for Xcode

The function `ERR_error_string` can be invoked without providing a
buffer, in which case OpenSSL will simply return a string printed into a
static buffer. Obviously and as documented in ERR_error_string(3), this
is not thread-safe at all. As libgit2 is a library, though, it is easily
possible that other threads may be using OpenSSL at the same time, which
might lead to clobbered error strings.

Fix the issue by instead using a stack-allocated buffer. According to
the documentation, the caller has to provide a buffer of at least 256
bytes of size. While we do so, make sure that the buffer will never get
overflown by switching to `ERR_error_string_n` to specify the buffer's
size.

The OpenSSL functions `SHA1_Init`, `SHA1_Update` and `SHA1_Final` all
return 1 for success and 0 otherwise, but we never check their return
values. Do so.

Some tests shall be run against our own SSH server we spin up in Travis.
As those need to be run separate from our previous tests which run
against git-daemon, we have to do this in a separate step. Instead of
bundling all that knowledge in the CI script, move it into the test
build instructions by creating a new test target.

Right now, we test our credential callback code twice, once via SSH on
localhost and once via a non-existent GitHub repository. While the first
URL makes sense to be configurable, it does not make sense to hard-code
the non-existing repository, which requires us to call tests multiple
times. Instead, we can just inline the URL into another set of tests.

We support two types of passing credentials to the proxy, either via the
URL or explicitly by specifying user and password. We test these types
by modifying the proxy URL and executing the tests twice, which is
in fact unnecessary and requires us to maintain the list of environment
variables and test executions across multiple CI infrastructures.

To fix the situation, we can just always pass the host, port, user and
password to the tests. The tests can then assemble the complete URL
either with or without included credentials, allowing us to test both
cases in-process.

Our performance tests (or to be more concrete, our single performance
test) are not built by default, as they are always #ifdef'd out. While
it is true that we don't want to run performance tests by default, not
compiling them at all may cause code rot and is thus an unfavorable
approach to handle this.

We can easily improve this situation: this commit removes the #ifdef,
causing the code to always be compiled. Furthermore, we add `-xperf` to
the default command line parameters of `generate.py`, thus causing the
tests to be excluded by default.

Due to this approach, we are now able to execute the performance tests
by passing `-sperf` to `libgit2_clar`. Unfortunately, we cannot execute
the performance tests on Travis or AppVeyor as they rely on history
being available for the libgit2 repository. As both do a shallow clone
only, though, this is not given.

The test `iterator::workdir::filesystem_gunk` is usually not executed,
as it is guarded by the environment variable "GITTEST_INVASIVE_SPEED"
due to its effects on speed. As such, it has become stale and does not
account for new references which have meanwhile been added to the
testrepo, causing it to fail. Fix this by raising the number of expected
references to 15.

When the function `expect_iterator_items` surpasses the number of
expected items, we simply break the loop. This causes us to trigger an
assert later on which has message attached, which is annoying when
trying to locate the root error cause. Instead, directly assert that the
current count is still smaller or equal to the expected count inside of
the loop.

Some function bodies of tests which are not applicable to the Win32
platform are completely #ifdef'd out instead of calling `cl_skip()`.
This leaves us with no indication that these tests are not being
executed at all and may thus cause decreased scrutiny when investigating
skipped tests. Improve the situation by calling `cl_skip()` instead of
just doing nothing.

Our tracing architecture is not built by default, causing the Travis CI
to not execute some code and skip several tests. As AppVeyor has already
enabled the tracing architecture when building the code, we should do
the same for Travis CI to have this code being tested on macOS and
Linux.

Add "-DENABLE_TRACE=ON" to our release-build options of Travis.

docs: git_treebuilder_insert validates entries

tree: standard error messages are lowercase

winhttp: properly support ntlm and negotiate

Our standard error messages begin with a lower case letter so that they
can be prefixed or embedded nicely.

These error messages were missed during the standardization pass since
they use the `tree_error` helper function.

The documentation for `git_treebuilder_insert` erroneously states that
we do not validate that the entry being inserted exists.  We do, as of
https://github.com/libgit2/libgit2/pull/3633.  Update the documentation
to reflect the new reality.

Support using notes via a commit rather than a ref

Transfer fewer objects on push and local fetch

refs: traverse symlinked directories

Inflate large loose blobs

Ensure that we can recurse into directories via symbolic links.

Perform some error checking when examining symlink directories.

 Native Git allows symlinked directories under .git/refs. This
 change allows libgit2 to also look for references that live under
 symlinked directories.

Signed-off-by: Andy Doan <andy@opensourcefoundries.com>

When parsing unauthorized responses, properly parse headers looking for
both NTLM and Negotiate challenges.  Set the HTTP credentials to default
credentials (using a `NULL` username and password) with the schemes
supported by ourselves and the server.

FETCH_HEAD and multiple refspecs

We treat each refspec on its own, but the code currently overwrites the contents
of FETCH_HEAD so we end up with the entries for the last refspec we processed.

Instead, truncate it before performing the updates and append to it when
updating the references.

We want to do this in order to get FETCH_HEAD to be empty when we start updating
it due to fetching from the remote.

patch_parse: fix parsing unquoted filenames with spaces

Fix unpack double free

If an element has been cached, but then the call to
packfile_unpack_compressed() fails, the very next thing that happens is
that its data is freed and then the element is not removed from the
cache, which frees the data again.

This change sets obj->data to NULL to avoid the double-free. It also
stops trying to resolve deltas after two continuous failed rounds of
resolution, and adds a test for this.

Free OpenSSL peer certificate

libFuzzer: Prevent a potential shift overflow

cmake: let USE_ICONV be optional on macOS

Do not attempt to check out submodule as blob when merging a submodule modify/deltete conflict