Move some checkout utility functions into a shared file and fix
some crlf filtering issues when verifying file contents.

Fix checkout corner cases, index handling, and reset hard behavior

There are a couple of checkout bugs fixed here.  One is with
untracked working directory entries that are prefixes of tree
entries but not in a meaningful way (i.e. "read" is a prefix of
"readme.txt" but doesn't interfere in any way).  The second bug
is actually a redo of 07edfa0fc640f85f95507c3101e77accd7d2bf0d
where directory entries in the index that are not in the diff
were not being removed correctly.  That fix remedied one case
but broke another.

This adds the git_index_remove_directory API plus tests.

I made a small change to the behavior of this code and apparently
got it wrong. Sigh.

When checking out with the GIT_CHECKOUT_REMOVE_UNTRACKED option
and there was an entire tree in the working directory and in the
index that is not in the baseline nor target commit, the tree was
correctly(?) removed from the working directory but was not
successfully removed from the index.  This fixes that and adds a
test of the functionality.

This adds a bunch of new checkout tests and in the process I found
a bug in the GIT_CHECKOUT_UPDATE_ONLY flag which I fixed.

This moves a lot of the detailed checkout documentation into a new
file (docs/checkout-internals.md) and simplifies the public docs
for the checkout API.

For clone to work as expected, it should be using a SAFE_CREATE
checkout (i.e. create files that are missing, even if the target
tree matches the current HEAD).

Test a number of other cases, including intentionally forced
conflicts and deeper inspection that trees get created properly.

There is a still a bug in checkout because the first test here
(i.e. test_checkout_typechange__checkout_typechanges_safe) should
be able to pass with GIT_CHECKOUT_SAFE as a strategy, but it will
not because of some lingering submodule checkout issues.

There were a bunch of small bugs in the checkout code where I was
assuming that a typechange was always from a tree to a blob or
vice versa.  This fixes up most of those cases.  Also, there were
circumstances where the submodule definitions were changed by the
checkout and the submodule data was not getting reloaded properly
before the new submodules were checked out.

The notifications were broken from the various iterations over
this code and were not returning working dir item data correctly.
Also, workdir items that were alphabetically after the last item
in diff were not being processed.

The spoolandsort iterator changes got sort-of cherry picked out of
this branch and so I dropped the commit when rebasing; however,
there were a few small changes that got dropped as well (since the
version merged upstream wasn't quite the same as what I dropped).

This adds a new API to the submodule interface that just returns
where information about the submodule was found (e.g. config file
only or in the HEAD, index, or working directory).

Also, the old "refresh" call was potentially keeping some stale
submodule data around, so this simplfies that code and literally
discards the old cache, then reallocates.

Stash was sometimes obscuring the actual error code, replacing it
with a -1 when there was more descriptive value.  This updates
stash to preserve the original error code more reliably along
with a variety of other error handling tweaks.

I believe this is an improvement, but arguably, preserving the
underlying error code may result in values that are harder to
interpret by the caller who does not understand the internals.
Discussion is welcome!

Previously a NULL oid was handled like an empty buffer and
returned a status empty string.  This makes git_oid_tostr()
set the output buffer to the empty string instead.

Make checkout update entries in the index for all files that are
updated and/or removed, unless flag GIT_CHECKOUT_DONT_UPDATE_INDEX
is given.  To do this, iterators were extended to allow a little
more introspection into the index being iterated over, etc.

This flips checkout back to be driven off the changes between
the baseline and the target trees.  This reinstates the complex
code for tracking the contents of the working directory, but
overall, I think the resulting logic is easier to follow.

I've tried to map out the detailed behaviors of checkout and make
sure that we're handling the various cases correctly, along with
providing options to allow us to emulate "git checkout" and "git
checkout-index" with the various flags.  I've thrown away flags
in the checkout API that seemed like clutter and added some new
ones.  Also, I've converted the conflict callback to a general
notification callback so we can emulate "git checkout" output and
display "dirty" files.

As of this commit, the new behavior is not working 100% but some
of that is probably baked into tests that are not testing the
right thing.  This is a decent snapshot point, I think, along the
way to getting the update done.

This corrects the order of operations in git reset so that the
checkout to reset the working directory content is done before
the HEAD is moved.  This allows us to use the HEAD and the index
content to know what files can / should safely be reset.

Unfortunately, there are still some cases where the behavior of
this revision differs from core git.  Notable, a file which has
been added to the index but is not present in the HEAD is
considered to be tracked by core git (and thus removable by a
reset command) whereas since this loads the target state into
the index prior to resetting, it will consider such a file to be
untracked and won't touch it.  That is a larger fix that I'll
defer to a future commit.

This adds a failure reporting function that is called by
cl_git_pass which captures the actual error return code and
the error message if available in the failure report.

Prevent soft reset when index contains conflicts

Add jGit license block to derrived tests

Add the jGit license block to tests derrived from jGit as per the
terms of the BSD license.

Conflicts:
	src/pqueue.c
	src/pqueue.h
	src/util.c
	tests-clar/merge/setup.c