Configuration file fixes with includes

Patch ID calculation

Modifying variables pulled in by an included file currently succeeds,
but it doesn't actually do what one would expect, as refreshing the
configuration will cause the values to reappear. As we are currently not
really able to support this use case, we will instead just return an
error for deleting and setting variables which were included via an
include.

Right now, we have multiple call sites which initialize a `reader`
structure. As the structure is only actually used inside of
`config_read`, we can instead just move the reader inside of the
`config_read` function. Instead, we can just pass in the configuration
file into `config_read`, which eases code readability.

Currently, we only re-parse the top-level configuration file when it has
changed itself. This can cause problems when an include is changed, as
we were not updating all values correctly.

Instead of conditionally reparsing only refreshed files, the logic
becomes much clearer and easier to follow if we always re-parse the
top-level configuration file when either the file itself or one of its
included configuration files has changed on disk. This commit implements
this logic.

Note that this might impact performance in some cases, as we need to
re-read all configuration files whenever any of the included files
changed. It could increase performance to just re-parse include files
which have actually changed, but this would compromise maintainability
of the code without much gain. The only case where we will gain anything
is when we actually use includes and when only these includes are
updated, which will probably be quite an unusual scenario to actually be
worthwhile to optimize.

The backend passed to `config_read` is never actually used anymore, so
we can remove it from the function and the `parse_data` structure.

Previously, the callbacks passed to `config_parse` got the reader via a
pointer to a pointer. This allowed the callbacks to update the callers
`reader` variable when the array holding it has been reallocated. As the
array is no longer present, we can simply the code by making the reader
a simple pointer.

Current code for configuration files uses the `reader` structure to
parse configuration files and store additional metadata like the file's
path and checksum. These structures are stored within an array in the
backend itself, which causes multiple problems.

First, it does not make sense to keep around the file's contents with
the backend itself. While this data is usually free'd before being added
to the backend, this brings along somewhat intricate lifecycle problems.
A better solution would be to store only the file paths as well as the
checksum of the currently parsed content only.

The second problem is that the `reader` structures are stored inside an
array. When re-parsing configuration files due to changed contents, we
may cause this array to be reallocated, requiring us to update pointers
hold by callers. Furthermore, we do not keep track of includes which
are already associated to a reader inside of this array. This causes us
to add readers multiple times to the backend, e.g. in the scenario of
refreshing configurations.

This commit fixes these shortcomings. We introduce a split between the
parsing data and the configuration file's metadata. The `reader` will
now only hold the file's contents and the parser state and the new
`config_file` structure holds the file's path and checksum. Furthermore,
the new structure is a recursive structure in that it will also hold
references to the files it directly includes. The diskfile is changed to
only store the top-level configuration file.

These changes allow us further refactorings and greatly simplify
understanding the code.

signature: don't leave a dangling pointer to the strings on parse failure

If the signature is invalid but we detect that after allocating the strings, we
free them. We however leave that pointer dangling in the structure the caller
gave us, which can lead to double-free.

Set these pointers to `NULL` after freeing their memory to avoid this.

git_reset_*: pass parameters as const pointers

tests: config: fix missing declaration causing error

On systems where we pull in our distributed version of the regex
library, all tests in config::readonly fail. This error is actually
quite interesting: the test suite is unable to find the declaration of
`git_path_exists` and assumes it has a signature of `int
git_path_exists(const char *)`. But actually, it has a `bool` return
value. Due to this confusion, some wrong conversion is done by the
compiler and the `cl_assert(!git_path_exists("file"))` checks
erroneously fail, even when the function does in fact return the correct
value.

The error is actually introduced by 56893bb9 (cmake: consistently use
TARGET_INCLUDE_DIRECTORIES if available, 2017-06-28), unfortunately
introduced by myself. Due to the delayed addition of include
directories, we will now find the "config.h" header inside of the
"deps/regex" directory instead of inside the "src/" directory, where it
should be. As such, we are missing definitions for the
`git_config_file__ondisk` and `git_path_exists` symbols.

The correct fix here would be to fix the order in which include search
directories are added. But due to the current restructuring of
CMakeBuild.txt, I'm refraining from doing so and delay the proper fix a
bit. Instead, we paper over the issue by explicitly including "path.h"
to fix its prototype. This ignores the issue that
`git_config_file__ondisk` is undeclared, as its signature is correctly
identified by the compiler.

Instead of using INCLUDE_DIRECTORIES again for the libgit2_clar test
suite, we should just be using TARGET_INCLUDE_DIRECTORIES again if the
CMake version is greater than 2.8.11.

Apply `target_include_directories` when CMAKE_VERSION >= 2.8.12

Convert port with htons() in p_getaddrinfo()

Out of tree builds

cmake: Permit disabling external http-parser

The upstream git project provides the ability to calculate a so-called
patch ID. Quoting from git-patch-id(1):

    A "patch ID" is nothing but a sum of SHA-1 of the file diffs
    associated with a patch, with whitespace and line numbers ignored."

Patch IDs can be used to identify two patches which are probably the
same thing, e.g. when a patch has been cherry-picked to another branch.

This commit implements a new function `git_diff_patchid`, which gets a
patch and derives an OID from the diff. Note the different terminology
here: a patch in libgit2 are the differences in a single file and a diff
can contain multiple patches for different files. The implementation
matches the upstream implementation and should derive the same OID for
the same diff. In fact, some code has been directly derived from the
upstream implementation.

The upstream implementation has two different modes to calculate patch
IDs, which is the stable and unstable mode. The old way of calculating
the patch IDs was unstable in a sense that a different ordering the
diffs was leading to different results. This oversight was fixed in git
1.9, but as git tries hard to never break existing workflows, the old
and unstable way is still default. The newer and stable way does not
care for ordering of the diff hunks, and in fact it is the mode that
should probably be used today. So right now, we only implement the
stable way of generating the patch ID.

`sin_port` should be in network byte order.

The test `refs::crashes::double_free` operates on our in-source
"testrepo.git" repository without creating a copy first. As the test
will try to create a new symbolic reference, this will fail when we want
to do a pure out-of-tree build with a read-only source tree.

Fix the issue by creating a sandbox first.

The test `index::tests::can_lock_index` operates on the "testrepo.git"
repository located inside of our source tree. While this is okay for
tests which do read-only operations on these resouces, this specific
test tries to lock the index by creating a lock. This will obviously
fail on out-of-tree builds with read-only source trees.

Fix the issue by creating a sandbox first.

Change the output path of generate.py to generate the clar.suite file
inside of the binary directory. This fixes out of tree builds with
read-only source trees as we now refrain from writing anything into the
source tree.

The source directory should usually not be touched when using
out-of-tree builds. But next to the previously fixed "clar.suite" file, we
are also writing the ".clarcache" into the project's source tree,
breaking the builds. Fix this by also honoring the output directory for
the ".clarcache" file.

The generate.py script will currently always write the generated
clar.suite file into the scanned directory, breaking out-of-tree builds
with read-only source directories. Fix this issue by adding another
option to allow overriding the output path of the generated file.

Our generate.py script is used to extract and write test suite
declarations into the clar.suite file. As is, the script accepts
multiple directories on the command line and will generate this file for
each of these directories.

The generate.py script will always write the clar.suite file into the
directory which is about to be scanned. This actually breaks
out-of-tree builds with libgit2, as the file will be generated in the
source tree instead of in the build tree. This is noticed especially in
the case where the source tree is mounted read-only, rendering us unable
to build unit tests.

Due to us accepting multiple paths which are to be scanned, it is not
trivial to fix though. The first solution which comes into mind would be
to re-create the directory hierarchy at a given output path or use
unique names for the clar.suite files, but this is rather cumbersome and
magical. The second and cleaner solution would be to fold all
directories into a single clar.suite file, but this would probably break
some use-cases.

Seeing that we do not ever pass multiple directories to generate.py, we
will now simply retire support for this. This allows us to later on
introduce an additional option to specify the path where the clar.suite
file will be generated at, defaulting to "clar.suite" inside of the
scanned directory.

When attempting to build libgit2 as an isolated static lib, CMake
gleefully attempts to use the system http-parser.  This is typically
seen on Linux systems which install header files with every package,
such as Gentoo.

Allow developers to forcibly disable using the system http-parser with
the config switch USE_EXT_HTTP_PARSER.  Defaults to ON to maintain
previous behavior.

Signed-off-by: Jason Cooper <jason@lakedaemon.net>

merge: fix potential free of uninitialized memory

The function `merge_diff_mark_similarity_exact` may error our early and,
when it does so, free the `ours_deletes_by_oid` and
`theirs_deletes_by_oid` variables. While the first one can never be
uninitialized due to the first call actually assigning to it, the second
variable can be freed without being initialized.

Fix the issue by initializing both variables to `NULL`.

merge: perform exact rename detection in linear time

travis: upgrade container to Ubuntu 14.04

Fix template dir empty string

Ubuntu 12.04 (Precise Pangolin) reaches end of life on April 28th, 2017.
As such, we should update our build infrastructure to use the next
available LTS release, which is Ubuntu 14.04 LTS (Trusty Tahr). Note
that Trusty is still considered beta quality on Travis. But considering
we are able to correctly build and test libgit2, this seems to be a
non-issue for us.

Switch over our default distribution to Trusty. As Precise still has
extended support for paying customers, add an additional job which
compiles libgit2 on the old release.

Some tests of ours require to be running against an SSH server.
Currently, we simply run against the SSH server provided and started by
Travis itself. As our Linux tests run in a sudo-less environment, we
have no control over its configuration and startup/shutdown procedure.
While this has been no problem until now, it will become a problem as
soon as we migrate over to newer Precise images, as the SSH server does
not have any host keys set up. Luckily, we can simply set up our own
unpriviledged SSH server. This has the benefit of us being able to
modify its configuration even in a sudo-less environment.

This commit sets up the unpriviledged SSH server on port 2222.

All our tests running against a local SSH server usually read the
server's URL from environment variables. But online::clone::ssh_cert
test fails to do so and instead always connects to
"ssh://localhost/foo". This assumption breaks whenever the SSH server is
not running on the standard port, e.g. when it is running as a user.

Fix the issue by using the URL provided by the environment.