In the iterator, distinguish between ignores and empty directories
so that diff and status can ignore empty directories, but checkout
and stash can treat them as untracked items.

When diff finds an untracked directory, it emulates Git behavior
by looking inside the directory to see if there are any untracked
items inside it. If there are only ignored items inside the dir,
then diff considers it ignored, even if there is no direct ignore
rule for it.

Checkout was not copying this behavior - when it found an untracked
directory, it just treated it as untracked.  Unfortunately, when
combined with GIT_CHECKOUT_REMOVE_UNTRACKED, this made is seem that
checkout (and stash, which uses checkout) was removing ignored
items when you had only asked it to remove untracked ones.

This commit moves the logic for advancing past an untracked dir
while scanning for non-ignored items into an iterator helper fn,
and uses that for both diff and checkout.

To emulate git, stash should not remove untracked git repositories
inside the parent repo, and checkout's REMOVE_UNTRACKED should
also skip over these items.

`git stash` actually prints a warning message for these items.
That should be possible with a checkout notify callback if you
wanted to, although it would require a bit of extra logic as things
are at the moment.

A few niceties for binding authors

Allow the credentials callback to return GIT_PASSTHROUGH to make the
transports code behave as though none was set.

This should make it easier for bindings to behave closer to the C code
when there is no credentials callback set at their level.

This should make it easier for bindings to dynamically override their
own callbacks.

Fix some issues from the last Coverity scan

Fix reset for staged deletes

Check for compiler flag support instead of checking for a platform

Reported by coverity.

Returning an error cleared the buf, but this operation does not free the
memory associated with it. Use git_buf_free() instead.

Add support for SSH keyboard-interactive authentication

Fix several ignore and attribute file behavior bugs

Only apply LEADING_DIR pattern munging to patterns in ignore and
attribute files, not to pathspecs used to select files to operate
on.  Also, allow internal macro definitions to be evaluated before
loading all external ones (important so that external ones can
make use of internal `binary` definition).

Ignore patterns that ended with a trailing '/*' were still needing
to match against another actual '/' character in the full path.
This is not the same behavior as core Git.

Instead, we strip a trailing '/*' off of any patterns that were
matching and just take it to imply the FNM_LEADING_DIR behavior.

There was a latent bug where files that use macro definitions
could be parsed before the macro definitions were loaded.  Because
of attribute file caching, preloading files that are going to be
used doesn't add a significant amount of overhead, so let's always
preload any files that could contain macros before we assemble the
actual vector of files to scan for attributes.

When traversing the directory structure, the iterator pushes and
pops ignore files using a vector.  Some directories don't have
ignore files, so it uses a path comparison to decide when it is
right to actually pop the last ignore file.  This was only
comparing directory suffixes, though, so a subdirectory with the
same name as a parent could result in the parent's .gitignore
being popped off the list ignores too early.  This changes the
logic to compare the entire relative path of the ignore file.

This simplifies platform/compiler dependent checks where we optionally
enable features or disable warnings.

Introduce core.safecrlf handling

This allows for keyboard-interactive based SSH authentication

cred: tighten username rules

The ssh-specific credentials allow the username to be missing. The idea
being that the ssh transport will then use the username provided in the
url, if it's available. There are two main issues with this.

The credential callback already knows what username was provided by the
url and needs to figure out whether it wants to ask the user for it or
it can reuse it, so passing NULL as the username means the credential
callback is suspicious.

The username provided in the url is not in fact used by the
transport. The only time it even considers it is for the user/pass
credential, which asserts the existence of a username in its
constructor. For the ssh-specific ones, it passes in the username stored
in the credential, which is NULL. The libssh2 macro we use runs strlen()
against this value (which is no different from what we would be doing
ourselves), so we then crash.

As the documentation doesn't suggest to leave out the username, assert
the need for a username in the code, which removes this buggy behavior
and removes implicit state.

git_cred_has_username() becomes a blacklist of credential types that do
not have a username. The only one at the moment is the 'default' one,
which is meant to call up some Microsoft magic.

Make index iterator thread safe

We treat this as a NUL-terminated string, so make sure that we add the
terminator.

The checks to see if files were out of date in the attibute cache
was wrong because the cache-breaker data wasn't getting stored
correctly.  Additionally, when the cache-breaker triggered, the
old file data was being leaked.

I don't love this approach, but achieving thread-safety for
attribute and ignore data while reloading files would require a
larger rewrite in order to avoid this.  If an attribute or ignore
file is out of date, this holds a lock on the file while we are
reloading the data so that another thread won't try to reload the
data at the same time.

In the threading tests, I was still seeing a race condition where
the same item could end up being inserted multiple times into the
index.  Preserving the sorted-ness of the index outside of the
`index_insert` call fixes the issue.

This is a big refactoring of the attribute file cache to be a bit
simpler which in turn makes it easier to enforce a lock around any
updates to the cache so that it can be used in a threaded env.
Tons of changes to the attributes and ignores code.