Add O_CLOEXEC to open calls

In loose objects backend, constrain mkdir calls to avoid extra mkdirs

In theory, p_stat should never return an S_ISLNK result, but due
to the current implementation on Windows with mount points it is
possible that it will.  For now, work around that by allowing a
link in the path to a directory being created.  If it is really a
problem, then the issue will be caught on the next iteration of
the loop, but typically this will be the right thing to do.

Do not redefine WC_ERR_INVALID_CHARS

This updates the calls that make the subdirectories for objects
to use a base directory above which git_futils_mkdir won't walk
any higher.  This prevents attempts to mkdir all the way up to
the root of the filesystem.

Also, this moves the objects_dir into the loose backend structure
and removes the separate allocation, plus does some preformatting
of the objects_dir value to guarantee a trailing slash, etc.

WC_ERR_INVALID_CHARS might be already defined by the Windows SDK.

Signed-off-by: Sven Strickroth <email@cs-ware.de>

Add target directory to checkout options

This adds additonal tests of the checkout target directory option
including using it to dump data from bare repos.

With the new target directory option to checkout, the non-bareness
of the repository should be checked much later in the parameter
validation process - actually that check was already in place, but
I was doing it redundantly in the checkout APIs.

This removes the now unnecessary early check for bare repos.  It
also adds some other parameter validation and makes it so that
implied parameters can actually be passed as NULL (i.e. if you
pass a git_index, you don't have to pass the git_repository - we
can get it from index).

This adds the ability for checkout to write to a target directory
instead of having to use the working directory of the repository.
This makes it easier to do exports of repository data and the like.

This is similar to, but not quite the same as, the --prefix option
to `git checkout-index` (this will always be treated as a directory
name, not just as a simple text prefix).

As part of this, the workdir iterator was extended to take the
path to the working directory as a parameter and fallback on the
git_repository_workdir result only if it's not specified.

Fixes #1332

Checkout should not recreate deleted files - with fix

This fixes the checkout case when a file is modified between the
baseline and the target and yet missing in the working directory.
The logic for that case appears to have been wrong.

This also adds a useful checkout notify callback to the checkout
test helpers that will count notifications and also has a debug
mode to visualize what checkout thinks that it's doing.

Command line status example (with bug fixes)

Add test for bug fixed in 852ded96
Sorry, I wrote that bug fix and forgot to check in a test at the
same time.  Here is one that fails on the old version of the code
and now works.

Files in status will, be default, be sorted according to the case
insensitivity of the filesystem that we're running on.  However,
in some cases, this is not desirable.  Even on case insensitive
file systems, 'git status' at the command line will generally use
a case sensitive sort (like 'ls').  Some GUIs prefer to display a
list of file case insensitively even on case-sensitive platforms.

This adds two new flags: GIT_STATUS_OPT_SORT_CASE_SENSITIVELY
and GIT_STATUS_OPT_SORT_CASE_INSENSITIVELY that will override the
default sort order of the status output and give the user control.
This includes tests for exercising these new options and makes
the examples/status.c program emulate core Git and always use a
case sensitive sort.

When scanning untracked directories looking for non-ignored files
there was a bug where an empty directory would generate a false
error.

Index operations using globs

This adds some tests for updating the index and having it remove
items to make sure that the iteration over the index still works
even as earlier items are removed.

In testing with valgrind, this found a path that would use the
path string from the index entry after it had been freed.  The
bug fix is simply to copy the path of the index entry before
doing any actual index manipulation.

This adds three new public APIs for manipulating the index:

1. `git_index_add_all` is similar to `git add -A` and will add
   files in the working directory that match a pathspec to the
   index while honoring ignores, etc.
2. `git_index_remove_all` removes files from the index that match
   a pathspec.
3. `git_index_update_all` updates entries in the index based on
   the current contents of the working directory, either added
   the new information or removing the entry from the index.

Command line Git sometimes generates an error message if given a
pathspec that contains an exact match to an ignored file (provided
--force isn't also given).  This adds an internal function that
makes it easy to check it that has happened.  Right now, I'm not
creating a public API for this because that would get a little
more complicated with a need for callbacks for all invalid paths.

Right now, setting up a pathspec to be parsed and processed
requires several data structures and a couple of API calls.  This
adds a new high level data structure that contains all the items
that you'll need and high-level APIs that do all of the setup and
all of the teardown.  This will make it easier to use pathspecs
in various places with less repeated code.

CMakeLists: fix zlib linker setup

b53671ae (Search for zlib unconditional, 2012-12-18) changed things
around to always (even on windows, that's what the subject refers to)
call FIND_PACKAGE(ZLIB).

However, it did not correctly handle the case where ZLIB_LIBRARY is
cached, either by the user setting it manually or by an earlier
search.  In that case, the IF(ZLIB_FOUND) would not trigger (that
variable is not cached) and we'd instead use the built-in version.

000e6896 (CMake: don't try to use bundled zlib when the system's path
is in the cache, 2013-05-12) tried to fix that, but it actually made
the problem worse: now with ZLIB_LIBRARY cached, _neither_ of the
blocks would execute, resulting in a linker error for me when trying
to build such a doubly-configured setup.

To fix the issue, we just trust CMake to do the right thing.  If
ZLIB_LIBRARY is set (either from user or cache) then the find_library
in FindZLIB.cmake will use that instead of searching again.  So we can
unconditionally (for real this time) call FIND_PACKAGE(ZLIB), and just
check its result.

Diff rename detection cycle fixes

This makes the diff rename tracking code more careful about the
order in which it processes renames and more thorough in updating
the mapping of correct renames when an earlier rename update
alters the index of a later matched pair.

Add "as_path" parameters to blob and buffer diff APIs

This adds parameters to the four functions that allow for blob-to-
blob and blob-to-buffer differencing (either via callbacks or by
making a git_diff_patch object).  These parameters let you say
that filename we should pretend the blob has while doing the diff.
If you pass NULL, there should be no change from the existing
behavior, which is to skip using attributes for file type checks
and just look at content.  With the parameters, you can plug into
the new diff driver functionality and get binary or non-binary
behavior, plus function context regular expressions, etc.

This commit also fixes things so that the git_diff_delta that is
generated by these functions will actually be populated with the
data that we know about the blobs (or buffers) so you can use it
appropriately.  It also fixes a bug in generating patches from
the git_diff_patch objects created via these functions.

Lastly, there is one other behavior change that may matter.  If
there is no difference between the two blobs, these functions no
longer generate any diff callbacks / patches unless you have
passed in GIT_DIFF_INCLUDE_UNMODIFIED.  This is pretty natural,
but could potentially change the behavior of existing usage.

Status indexed updates

This fixes a couple objects I forgot to free, and also updates
the valgrind suppressions file on the Mac to cover a few more
cases that had crept in.

A tree to index rename with no changes was getting erased by
the iteration routine (if the routine actually loaded the data
for the unmodified file).  This invokes the code path that was
previously messing up the diff and iterates twice to make sure
that the iteration process itself doesn't modify the data.

This changes the behavior of the status RENAMED flags so that they
will be combined with the MODIFIED flags if appropriate.  If a file
is modified in the index and also renamed, then the status code
will have both the GIT_STATUS_INDEX_MODIFIED and INDEX_RENAMED bits
set.  If it is renamed but the OID has not changed, then just the
GIT_STATUS_INDEX_RENAMED bit will be set.  Similarly, the flags
GIT_STATUS_WT_MODIFIED and GIT_STATUS_WT_RENAMED can both be set
independently of one another.

This fixes a serious bug where the check for unmodified files that
was done at data load time could end up erasing the RENAMED state
of a file that was renamed with no changes.

Lastly, this contains a bunch of new tests for status with renames,
including tests where the only rename changes are case changes.
The expected results of these tests have to vary by whether the
platform uses a case sensitive filesystem or not, so the expected
data covers those platform differences separately.