test_helpers: do not rely on assert

References! References! References!

This new version of the references code is significantly faster and
hopefully easier to read.

External API stays the same. A new method `git_reference_reload()` has
been added to force updating a memory reference from disk. In-memory
references are no longer updated automagically -- this was killing us.

If a reference is deleted externally and the user doesn't reload the
memory object, nothing critical happens: any functions using that
reference should fail gracefully (e.g. deletion, renaming, and so on).

All generated references from the API are read only and must be free'd
by the user. There is no reference counting and no traces of generated
references are kept in the library.

There is no longer an internal representation for references. There is
only one reference struct `git_reference`, and symbolic/oid targets are
stored inside an union.

Packfile references are stored using an optimized struct with flex array
for reference names. This should significantly reduce the memory cost of
loading the packfile from disk.

git_reference_rename() didn't properly cleanup old references given by
the user to not break some ugly old tests. Since references don't point
to libgit's internal cache anymore we can cleanup git_reference_rename()
to be somewhat less messy.

Signed-off-by: schu <schu-github@schulog.org>

Since references are not owned by the repository anymore we have to free
them manually now.

Signed-off-by: schu <schu-github@schulog.org>

Signed-off-by: schu <schu-github@schulog.org>

Currently libgit2 shares pointers to its internal reference cache with
the user. This leads to several problems like invalidation of reference
pointers when reordering the cache or manipulation of the cache from
user side.

Give each user its own git_reference instead of leaking the internal
representation (struct reference).

Add the following new API functions:

	* git_reference_free
	* git_reference_is_packed

Signed-off-by: schu <schu-github@schulog.org>

The functions loose_object_mode and loose_object_dir_mode call stat
inside an assert statement which isn't evaluated when compiling in
Release mode (NDEBUG) and leads to failing tests. Replace it.

Signed-off-by: schu <schu-github@schulog.org>

Status: fix segfault (#465) and order issues

This ensures that entries from the working directory are retrieved according to the following rules:

 - The file "subdir" should appear before the file "subdir.txt"
 - The folder "subdir" should appear after the file "subdir.txt"

status: Add a file in the test repository to cover the correct sorting of entries when the working folder is being read

In this case, "subdir.txt" should be listed before the "subdir" directory.

Fixes #465

Create objects, indexes, and directories with the right file permissions

Ensure that all memory related functions (malloc, calloc, strdup, free,
etc) are using their respective `git__` wrappers.

Their actual values have no meaning, so pack them in an enum.

Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>

Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>

The only caller has been changed to treat a NULL tree as a special
case and use the existing git_tree_entry_byindex.

Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>

This function is already implemented (better) as git_index_get. Change
the only caller to use that function.

Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>

Fixed crash in config parser when empty value is encountered.

Example:

key1 = value1
key2 =

In this config the value will be a bad pointer which config object will attempt to free() causing a crash.

tests-clay: move t01-rawobj.c to clay

Our previous assumption that all paths in Windows are encoded in UTF-8
is rather weak, specially when considering that Git is
encoding-agnostic.

These set of functions allow the user to change the library's active
codepage globally, so it is possible to access paths and files on all
international versions of Windows.

Note that the default encoding here is UTF-8 because we assume that 99%
of all Git repositories will be in UTF-8.

Also, if you use non-ascii characters in paths, anywhere, please burn on
a fire.

Signed-off-by: schu <schu-github@schulog.org>

Tolerate zlib deflation with window size < 32Kb

libgit2 currently identifies loose objects as corrupt if they've been
deflated using a window size less than 32Kb, because the
is_zlib_compressed_data() function doesn't recognise the header
byte as a zlib header. This patch makes the method tolerant of
all valid window sizes (15-bit to 8-bit) - but doesn't sacrifice
it's accuracy in distingushing the standard loose-object format
from the experimental (now abandoned) format. It's based on a patch
which has been merged into C-Git master branch:

https://github.com/git/git/commit/7f684a2aff636f44a506

On memory constrained systems zlib may use a much smaller window
size - working on Agit, I found that Android uses a 4KB window;
giving a header byte of 0x48, not 0x78. Consequently all loose
objects generated by the Android platform appear 'corrupt' :(

It might appear that this patch changes isStandardFormat() to the
point where it could incorrectly identify the experimental format as
the standard one, but the two criteria (bitmask & checksum) can only
give a false result for an experimental object where both of the
following are true:

1) object size is exactly 8 bytes when uncompressed (bitmask)
2) [single-byte in-pack git type&size header] * 256
   + [1st byte of the following zlib header] % 31 = 0 (checksum)

As it happens, for all possible combinations of valid object type
(1-4) and window bits (0-7), the only time when the checksum will be
divisible by 31 is for 0x1838 - ie object type *1*, a Commit - which,
due the fields all Commit objects must contain, could never be as
small as 8 bytes in size.

Given this, the combination of the two criteria (bitmask & checksum)
always correctly determines the buffer format, and is more tolerant
than the previous version.

References:

Android uses a 4KB window for deflation:
http://android.git.kernel.org/?p=platform/libcore.git;a=blob;f=luni/src/main/native/java_util_zip_Deflater.cpp;h=c0b2feff196e63a7b85d97cf9ae5bb258

Code snippet searching for false positives with the zlib checksum:
https://gist.github.com/1118177

Change-Id: Ifd84cd2bd6b46f087c9984fb4cbd8309f483dec0

Otherwise, GCC optimizes variables away and gdb can't tell us what's
in them.

Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>

Double-pass binary search. Jeez.

Remove a wrong call to git_mwindow_close which caused a segfault if it
ever did run. In that same piece of code, if the LRU was from the
first wiindow in the list in a different file, we didn't update that
list, so the first element had been freed.

Fix these two issues.

Signed-off-by: Carlos Martín Nieto <carlos@cmartin.tk>

tests: propagate errors from open_temp_repo() instead of exiting

This makes it slightly easier to debug test failures when one test
opens a repo, has a failure, and doesn't get a chance to close it for
the next test. Now, instead of getting no feedback, we at least see
test failure information.

The following files now have 0444 permissions:

- loose objects
- pack indexes
- pack files
- packs downloaded by fetch
- packs downloaded by the HTTP transport

And the following files now have 0666 permissions:

- config files
- repository indexes
- reflogs
- refs

This brings libgit2 more in line with Git.

Note that git_filebuf_commit() and git_filebuf_commit_at() have both
gained a new mode parameter.

The latter change fixes an important issue where filebufs created with
GIT_FILEBUF_TEMPORARY received 0600 permissions (due to mkstemp(3)
usage). Now we chmod() the file before renaming it into place.

Tests have been added to confirm that new commit, tag, and tree
objects are created with the right permissions. I don't have access to
Windows, so for now I've guarded the tests with "#ifndef GIT_WIN32".

To further match how Git behaves, this change makes most of the
directories libgit2 creates in a git repo have a file mode of
0777. Specifically:

- Intermediate directories created with git_futils_mkpath2file() have
  0777 permissions. This affects odb_loose, reflog, and refs.

- The top level folder for bare repos is created with 0777
  permissions.

- The top level folder for non-bare repos is created with 0755
  permissions.

- /objects/info/, /objects/pack/, /refs/heads/, and /refs/tags/ are
  created with 0777 permissions.

Additionally, the following changes have been made:

- fileops functions that create intermediate directories have grown a
  new dirmode parameter. The only exception to this is filebuf's
  lock_file(), which unconditionally creates intermediate directories
  with 0777 permissions when GIT_FILEBUF_FORCE is set.

- The test runner now sets the umask to 0 before running any
  tests. This ensurses all file mode checks are consistent across
  systems.

- t09-tree.c now does a directory permissions check. I've avoided
  adding this check to other tests that might reuse existing
  directories from the prefabricated test repos. Because they're
  checked into the repo, they have 0755 permissions.

- Other assorted directories created by tests have 0777 permissions.

Note: Functions exported from fileops take const mode_t, while the
underlying POSIX wrappers take mode_t.

Update Makefile.embed with http-parser dependency