In commit b9c5b15a (http: use the new httpclient, 2019-12-22), the HTTP
code got refactored to extract a generic HTTP client that operates
independently of the Git protocol. Part of refactoring was the creation
of a new `git_http_request` struct that encapsulates the generation of
requests. Our Git-specific HTTP transport was converted to use that in
`generate_request`, but during the process we forgot to set up custom
headers for the `git_http_request` and as a result we do not send out
these headers anymore.

Fix the issue by correctly setting up the request's custom headers and
add a test to verify we correctly send them.

azure: tests: re-run flaky proxy tests

While we already do have logic to re-run flaky tests, the FAILED
variable currently does not get reset to "0". As a result, successful
reruns will still cause the test to be registered as failed.

Fix this by resetting the variable accordingly.

The proxy tests regularly fail in our CI environment. Unfortunately,
this is expected due to the network layer. Thus, let's re-try the proxy
tests up to five times in case they fail.

fetchhead: strip credentials from remote URL

If fetching from an anonymous remote via its URL, then the URL gets
written into the FETCH_HEAD reference. This is mainly done to give
valuable context to some commands, like for example git-merge(1), which
will put the URL into the generated MERGE_MSG. As a result, what gets
written into FETCH_HEAD may become public in some cases. This is
especially important considering that URLs may contain credentials, e.g.
when cloning 'https://foo:bar@example.com/repo' we persist the complete
URL into FETCH_HEAD and put it without any kind of sanitization into the
MERGE_MSG. This is obviously bad, as your login data has now just leaked
as soon as you do git-push(1).

When writing the URL into FETCH_HEAD, upstream git does strip
credentials first. Let's do the same by trying to parse the remote URL
as a "real" URL, removing any credentials and then re-formatting the
URL. In case this fails, e.g. when it's a file path or not a valid URL,
we just fall back to using the URL as-is without any sanitization. Add
tests to verify our behaviour.

azure-pipelines: properly expand negotiate passwords

To allow testing against a Kerberos instance, we have added variables
for the Kerberos password to allow authentication against LIBGIT2.ORG in
commit e5fb5fe5 (ci: perform SPNEGO tests, 2019-10-20). To set up the
password, we assign

    "GITTEST_NEGOTIATE_PASSWORD=$(GITTEST_NEGOTIATE_PASSWORD)"

in the environmentVariables section which is then passed through to a
template. As the template does build-time expansion of the environment
variables, it will expand the above line verbosely, and due to the
envVar section not doing any further expansion the password variable
will end up with the value "$(GITTEST_NEGOTIATE_PASSWORD)" in the
container's environment.

Fix this fixed by doing expansion of GITTEST_NEGOTIATE_PASSWORD at
build-time, as well.

cred: change enum to git_credential_t and GIT_CREDENTIAL_*

Update link to libgit2 Julia language binding

We avoid abbreviations where possible; rename git_cred to
git_credential.

In addition, we have standardized on a trailing `_t` for enum types,
instead of using "type" in the name.  So `git_credtype_t` has become
`git_credential_t` and its members have become `GIT_CREDENTIAL` instead
of `GIT_CREDTYPE`.

Finally, the source and header files have been renamed to `credential`
instead of `cred`.

Keep previous name and values as deprecated, and include the new header
files from the previous ones.

Return int from non-free functions

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

Stop returning a void for functions, future-proofing them to allow them
to fail.

HTTP: Support Apache-based servers with Negotiate

Disambiguate between general network problems and HTTP problems in error
codes.

Download poxygit, a debugging git server, and clone from it using NTLM,
both IIS-style (with connection affinity) and Apache-style ("broken",
requiring constant reauthentication).

When tracing is disabled, don't let `git_trace__level` return a void,
since that can't be compared against.

When we're authenticating with a connection-based authentication scheme
(NTLM, Negotiate), we need to make sure that we're still connected
between the initial GET where we did the authentication and the POST
that we're about to send.  Our keep-alive session may have not kept
alive, but more likely, some servers do not authenticate the entire
keep-alive connection and may have "forgotten" that we were
authenticated, namely Apache and nginx.

Send a "probe" packet, that is an HTTP POST request to the upload-pack
or receive-pack endpoint, that consists of an empty git pkt ("0000").
If we're authenticated, we'll get a 200 back.  If we're not, we'll get a
401 back, and then we'll resend that probe packet with the first step of
our authentication (asking to start authentication with the given
scheme).  We expect _yet another_ 401 back, with the authentication
challenge.

Finally, we will send our authentication response with the actual POST
data.  This will allow us to authenticate without draining the POST data
in the initial request that gets us a 401.

Untangle the notion of the http transport from the actual http
implementation.  The http transport now uses the httpclient.

Allow users to opt-in to expect/continue handling when sending a POST
and we're authenticated with a "connection-based" authentication
mechanism like NTLM or Negotiate.

If the response is a 100, return to the caller (to allow them to post
their body).  If the response is *not* a 100, buffer the response for
the caller.

HTTP expect/continue is generally safe, but some legacy servers
have not implemented it correctly.  Require it to be opt-in.

Fully support HTTP proxies, in particular CONNECT proxies, that allow us
to speak TLS through a proxy.

Detect responses that are sent with Transfer-Encoding: chunked, and
record that information so that we can consume the entire message body.

Store the last-seen credential challenges (eg, all the
'WWW-Authenticate' headers in a response message).  Given some
credentials, find the best (first) challenge whose mechanism supports
these credentials.  (eg, 'Basic' supports username/password credentials,
'Negotiate' supports default credentials).

Set up an authentication context for this mechanism and these
credentials.  Continue exchanging challenge/responses until we're
authenticated.

Introduce a function to format the path and query string for a URL,
suitable for creating an HTTP request.

When sending a new request, ensure that we got the entirety of the
response body.  Our caller may have decided that they were done reading.
If we were not at the end of the message, this means that we need to
tear down the connection and cannot do keep-alive.

However, if the caller read all of the message, but we still have a
final end-of-response chunk signifier (ie, "0\r\n\r\n") on the socket,
then we should consider that the response was successfully copmleted.

If we're asked to send a new request, try to read from the socket, just
to clear out that end-of-chunk message, marking ourselves as
disconnected on any errors.

The CLAR_TRACE_LEVEL environment variable was supported when building
with GIT_TRACE.  Now we always build with GIT_TRACE, but that variable
is not provided to tests.  Simply support clar tracing always.

Teach httpclient how to support chunking when POSTing request bodies.