Merge pull request #1118 from arrbee/fix-reset-hard

Fix checkout corner cases, index handling, and reset hard behavior

docs/checkout-internals.md

+Checkout Internals
+==================
+
+Checkout has to handle a lot of different cases.  It examines the
+differences between the target tree, the baseline tree and the working
+directory, plus the contents of the index, and groups files into five
+categories:
+
+1. UNMODIFIED - Files that match in all places.
+2. SAFE - Files where the working directory and the baseline content
+   match that can be safely updated to the target.
+3. DIRTY/MISSING - Files where the working directory differs from the
+   baseline but there is no conflicting change with the target.  One
+   example is a file that doesn't exist in the working directory - no
+   data would be lost as a result of writing this file.  Which action
+   will be taken with these files depends on the options you use.
+4. CONFLICTS - Files where changes in the working directory conflict
+   with changes to be applied by the target.  If conflicts are found,
+   they prevent any other modifications from being made (although there
+   are options to override that and force the update, of course).
+5. UNTRACKED/IGNORED - Files in the working directory that are untracked
+   or ignored (i.e. only in the working directory, not the other places).
+
+Right now, this classification is done via 3 iterators (for the three
+trees), with a final lookup in the index.  At some point, this may move to
+a 4 iterator version to incorporate the index better.
+
+The actual checkout is done in five phases (at least right now).
+
+1. The diff between the baseline and the target tree is used as a base
+   list of possible updates to be applied.
+2. Iterate through the diff and the working directory, building a list of
+   actions to be taken (and sending notifications about conflicts and
+   dirty files).
+3. Remove any files / directories as needed (because alphabetical
+   iteration means that an untracked directory will end up sorted *after*
+   a blob that should be checked out with the same name).
+4. Update all blobs.
+5. Update all submodules (after 4 in case a new .gitmodules blob was
+   checked out)
+
+Checkout could be driven either off a target-to-workdir diff or a
+baseline-to-target diff.  There are pros and cons of each.
+
+Target-to-workdir means the diff includes every file that could be
+modified, which simplifies bookkeeping, but the code to constantly refer
+back to the baseline gets complicated.
+
+Baseline-to-target has simpler code because the diff defines the action to
+take, but needs special handling for untracked and ignored files, if they
+need to be removed.
+
+The current checkout implementation is based on a baseline-to-target diff.
+
+
+Picking Actions
+===============
+
+The most interesting aspect of this is phase 2, picking the actions that
+should be taken.  There are a lot of corner cases, so it may be easier to
+start by looking at the rules for a simple 2-iterator diff:
+
+Key
+---
+- B1,B2,B3 - blobs with different SHAs,
+- Bi       - ignored blob (WD only)
+- T1,T2,T3 - trees with different SHAs,
+- Ti       - ignored tree (WD only)
+- x        - nothing
+
+Diff with 2 non-workdir iterators
+---------------------------------
+
+    Old New
+    --- ---
+  0   x   x - nothing
+  1   x  B1 - added blob
+  2   x  T1 - added tree
+  3  B1   x - removed blob
+  4  B1  B1 - unmodified blob
+  5  B1  B2 - modified blob
+  6  B1  T1 - typechange blob -> tree
+  7  T1   x - removed tree
+  8  T1  B1 - typechange tree -> blob
+  9  T1  T1 - unmodified tree
+ 10  T1  T2 - modified tree (implies modified/added/removed blob inside)
+
+
+Now, let's make the "New" iterator into a working directory iterator, so
+we replace "added" items with either untracked or ignored, like this:
+
+Diff with non-work & workdir iterators
+--------------------------------------
+
+    Old New-WD
+    --- ------
+  0   x   x - nothing
+  1   x  B1 - untracked blob
+  2   x  Bi - ignored file
+  3   x  T1 - untracked tree
+  4   x  Ti - ignored tree
+  5  B1   x - removed blob
+  6  B1  B1 - unmodified blob
+  7  B1  B2 - modified blob
+  8  B1  T1 - typechange blob -> tree
+  9  B1  Ti - removed blob AND ignored tree as separate items
+ 10  T1   x - removed tree
+ 11  T1  B1 - typechange tree -> blob
+ 12  T1  Bi - removed tree AND ignored blob as separate items
+ 13  T1  T1 - unmodified tree
+ 14  T1  T2 - modified tree (implies modified/added/removed blob inside)
+
+Note: if there is a corresponding entry in the old tree, then a working
+directory item won't be ignored (i.e. no Bi or Ti for tracked items).
+
+
+Now, expand this to three iterators: a baseline tree, a target tree, and
+an actual working directory tree:
+
+Checkout From 3 Iterators (2 not workdir, 1 workdir)
+----------------------------------------------------
+
+(base == old HEAD; target == what to checkout; actual == working dir)
+
+      base target actual/workdir
+      ---- ------ ------
+  0      x      x      x - nothing
+  1      x      x B1/Bi/T1/Ti - untracked/ignored blob/tree (SAFE)
+  2+     x     B1      x - add blob (SAFE)
+  3      x     B1     B1 - independently added blob (FORCEABLE-2)
+  4*     x     B1 B2/Bi/T1/Ti - add blob with content conflict (FORCEABLE-2)
+  5+     x     T1      x - add tree (SAFE)
+  6*     x     T1  B1/Bi - add tree with blob conflict (FORCEABLE-2)
+  7      x     T1   T1/i - independently added tree (SAFE+MISSING)
+  8     B1      x      x - independently deleted blob (SAFE+MISSING)
+  9-    B1      x     B1 - delete blob (SAFE)
+ 10-    B1      x     B2 - delete of modified blob (FORCEABLE-1)
+ 11     B1      x  T1/Ti - independently deleted blob AND untrack/ign tree (SAFE+MISSING !!!)
+ 12     B1     B1      x - locally deleted blob (DIRTY || SAFE+CREATE)
+ 13+    B1     B2      x - update to deleted blob (SAFE+MISSING)
+ 14     B1     B1     B1 - unmodified file (SAFE)
+ 15     B1     B1     B2 - locally modified file (DIRTY)
+ 16+    B1     B2     B1 - update unmodified blob (SAFE)
+ 17     B1     B2     B2 - independently updated blob (FORCEABLE-1)
+ 18+    B1     B2     B3 - update to modified blob (FORCEABLE-1)
+ 19     B1     B1  T1/Ti - locally deleted blob AND untrack/ign tree (DIRTY)
+ 20*    B1     B2  T1/Ti - update to deleted blob AND untrack/ign tree (F-1)
+ 21+    B1     T1      x - add tree with locally deleted blob (SAFE+MISSING)
+ 22*    B1     T1     B1 - add tree AND deleted blob (SAFE)
+ 23*    B1     T1     B2 - add tree with delete of modified blob (F-1)
+ 24     B1     T1     T1 - add tree with deleted blob (F-1)
+ 25     T1      x      x - independently deleted tree (SAFE+MISSING)
+ 26     T1      x  B1/Bi - independently deleted tree AND untrack/ign blob (F-1)
+ 27-    T1      x     T1 - deleted tree (MAYBE SAFE)
+ 28+    T1     B1      x - deleted tree AND added blob (SAFE+MISSING)
+ 29     T1     B1     B1 - independently typechanged tree -> blob (F-1)
+ 30+    T1     B1     B2 - typechange tree->blob with conflicting blob (F-1)
+ 31*    T1     B1  T1/T2 - typechange tree->blob (MAYBE SAFE)
+ 32+    T1     T1      x - restore locally deleted tree (SAFE+MISSING)
+ 33     T1     T1  B1/Bi - locally typechange tree->untrack/ign blob (DIRTY)
+ 34     T1     T1  T1/T2 - unmodified tree (MAYBE SAFE)
+ 35+    T1     T2      x - update locally deleted tree (SAFE+MISSING)
+ 36*    T1     T2  B1/Bi - update to tree with typechanged tree->blob conflict (F-1)
+ 37     T1     T2 T1/T2/T3 - update to existing tree (MAYBE SAFE)
+
+The number is followed by ' ' if no change is needed or '+' if the case
+needs to write to disk or '-' if something must be deleted and '*' if
+there should be a delete followed by an write.
+
+There are four tiers of safe cases:
+
+- SAFE         == completely safe to update
+- SAFE+MISSING == safe except the workdir is missing the expect content
+- MAYBE SAFE   == safe if workdir tree matches (or is missing) baseline
+                  content, which is unknown at this point
+- FORCEABLE == conflict unless FORCE is given
+- DIRTY     == no conflict but change is not applied unless FORCE
+
+Some slightly unusual circumstances:
+
+  8 - parent dir is only deleted when file is, so parent will be left if
+      empty even though it would be deleted if the file were present
+ 11 - core git does not consider this a conflict but attempts to delete T1
+      and gives "unable to unlink file" error yet does not skip the rest
+      of the operation
+ 12 - without FORCE file is left deleted (i.e. not restored) so new wd is
+      dirty (and warning message "D file" is printed), with FORCE, file is
+      restored.
+ 24 - This should be considered MAYBE SAFE since effectively it is 7 and 8
+      combined, but core git considers this a conflict unless forced.
+ 26 - This combines two cases (1 & 25) (and also implied 8 for tree content)
+      which are ok on their own, but core git treat this as a conflict.
+      If not forced, this is a conflict.  If forced, this actually doesn't
+      have to write anything and leaves the new blob as an untracked file.
+ 32 - This is the only case where the baseline and target values match
+      and yet we will still write to the working directory.  In all other
+      cases, if baseline == target, we don't touch the workdir (it is
+      either already right or is "dirty").  However, since this case also
+      implies that a ?/B1/x case will exist as well, it can be skipped.
+
+Cases 3, 17, 24, 26, and 29 are all considered conflicts even though
+none of them will require making any updates to the working directory.
+

include/git2/index.h

@@ -313,6 +313,17 @@ GIT_EXTERN(const git_index_entry *) git_index_get_bypath(
 GIT_EXTERN(int) git_index_remove(git_index *index, const char *path, int stage);
 
 /**
+ * Remove all entries from the index under a given directory
+ *
+ * @param index an existing index object
+ * @param dir container directory path
+ * @param stage stage to search
+ * @return 0 or an error code
+ */
+GIT_EXTERN(int) git_index_remove_directory(
+	git_index *index, const char *dir, int stage);
+
+/**
  * Add or update an index entry from an in-memory struct
  *
  * If a previous index entry exists that has the same path and stage

include/git2/submodule.h

@@ -504,6 +504,24 @@ GIT_EXTERN(int) git_submodule_status(
 	unsigned int *status,
 	git_submodule *submodule);
 
+/**
+ * Get the locations of submodule information.
+ *
+ * This is a bit like a very lightweight version of `git_submodule_status`.
+ * It just returns a made of the first four submodule status values (i.e.
+ * the ones like GIT_SUBMODULE_STATUS_IN_HEAD, etc) that tell you where the
+ * submodule data comes from (i.e. the HEAD commit, gitmodules file, etc.).
+ * This can be useful if you want to know if the submodule is present in the
+ * working directory at this point in time, etc.
+ *
+ * @param status Combination of first four `GIT_SUBMODULE_STATUS` flags
+ * @param submodule Submodule for which to get status
+ * @return 0 on success, <0 on error
+ */
+GIT_EXTERN(int) git_submodule_location(
+	unsigned int *location_status,
+	git_submodule *submodule);
+
 /** @} */
 GIT_END_DECL
 #endif

src/attr.c

@@ -572,9 +572,11 @@ static int collect_attr_files(
 		error = git_futils_find_system_file(&dir, GIT_ATTR_FILE_SYSTEM);
 		if (!error)
 			error = push_attr_file(repo, files, NULL, dir.ptr);
-		else if (error == GIT_ENOTFOUND)
+		else if (error == GIT_ENOTFOUND) {
+			giterr_clear();
 			error = 0;
 		}
+	}
 
  cleanup:
 	if (error < 0)

src/checkout.h

+/*
+ * Copyright (C) 2009-2012 the libgit2 contributors
+ *
+ * This file is part of libgit2, distributed under the GNU GPL v2 with
+ * a Linking Exception. For full terms see the included COPYING file.
+ */
+#ifndef INCLUDE_checkout_h__
+#define INCLUDE_checkout_h__
+
+#include "git2/checkout.h"
+#include "iterator.h"
+
+#define GIT_CHECKOUT__NOTIFY_CONFLICT_TREE (1u << 12)
+
+/**
+ * Update the working directory to match the target iterator.  The
+ * expected baseline value can be passed in via the checkout options
+ * or else will default to the HEAD commit.
+ */
+extern int git_checkout_iterator(
+	git_iterator *target,
+	git_checkout_opts *opts);
+
+#endif

src/clone.c

@@ -362,7 +362,7 @@ static bool should_checkout(
 	if (!opts)
 		return false;
 
-	if (opts->checkout_strategy == GIT_CHECKOUT_DEFAULT)
+	if (opts->checkout_strategy == GIT_CHECKOUT_NONE)
 		return false;
 
 	return !git_repository_head_orphan(repo);

src/diff.c

@@ -164,6 +164,11 @@ static git_diff_delta *diff_delta__last_for_item(
 		if (git_oid_cmp(&delta->new_file.oid, &item->oid) == 0)
 			return delta;
 		break;
+	case GIT_DELTA_UNTRACKED:
+		if (diff->strcomp(delta->new_file.path, item->path) == 0 &&
+			git_oid_cmp(&delta->new_file.oid, &item->oid) == 0)
+			return delta;
+		break;
 	case GIT_DELTA_MODIFIED:
 		if (git_oid_cmp(&delta->old_file.oid, &item->oid) == 0 ||
 			git_oid_cmp(&delta->new_file.oid, &item->oid) == 0)
@@ -531,14 +536,14 @@ static bool entry_is_prefixed(
 {
 	size_t pathlen;
 
-	if (!prefix_item || diff->pfxcomp(prefix_item->path, item->path))
+	if (!item || diff->pfxcomp(item->path, prefix_item->path) != 0)
 		return false;
 
-	pathlen = strlen(item->path);
+	pathlen = strlen(prefix_item->path);
 
-	return (item->path[pathlen - 1] == '/' ||
-			prefix_item->path[pathlen] == '\0' ||
-			prefix_item->path[pathlen] == '/');
+	return (prefix_item->path[pathlen - 1] == '/' ||
+			item->path[pathlen] == '\0' ||
+			item->path[pathlen] == '/');
 }
 
 static int diff_list_init_from_iterators(
@@ -584,8 +589,7 @@ int git_diff__from_iterators(
 
 	*diff_ptr = NULL;
 
-	if (!diff ||
-		diff_list_init_from_iterators(diff, old_iter, new_iter) < 0)
+	if (!diff || diff_list_init_from_iterators(diff, old_iter, new_iter) < 0)
 		goto fail;
 
 	if (diff->opts.flags & GIT_DIFF_DELTAS_ARE_ICASE) {
@@ -616,7 +620,7 @@ int git_diff__from_iterators(
 			 * instead of just generating a DELETE record
 			 */
 			if ((diff->opts.flags & GIT_DIFF_INCLUDE_TYPECHANGE_TREES) != 0 &&
-				entry_is_prefixed(diff, oitem, nitem))
+				entry_is_prefixed(diff, nitem, oitem))
 			{
 				/* this entry has become a tree! convert to TYPECHANGE */
 				git_diff_delta *last = diff_delta__last_for_item(diff, oitem);
@@ -624,6 +628,17 @@ int git_diff__from_iterators(
 					last->status = GIT_DELTA_TYPECHANGE;
 					last->new_file.mode = GIT_FILEMODE_TREE;
 				}
+
+				/* If new_iter is a workdir iterator, then this situation
+				 * will certainly be followed by a series of untracked items.
+				 * Unless RECURSE_UNTRACKED_DIRS is set, skip over them...
+				 */
+				if (S_ISDIR(nitem->mode) &&
+					!(diff->opts.flags & GIT_DIFF_RECURSE_UNTRACKED_DIRS))
+				{
+					if (git_iterator_advance(new_iter, &nitem) < 0)
+						goto fail;
+				}
 			}
 
 			if (git_iterator_advance(old_iter, &oitem) < 0)
@@ -635,6 +650,7 @@ int git_diff__from_iterators(
 		 */
 		else if (cmp > 0) {
 			git_delta_t delta_type = GIT_DELTA_UNTRACKED;
+			bool contains_oitem = entry_is_prefixed(diff, oitem, nitem);
 
 			/* check if contained in ignored parent directory */
 			if (git_buf_len(&ignore_prefix) &&
@@ -646,14 +662,12 @@ int git_diff__from_iterators(
 				 * it or if the user requested the contents of untracked
 				 * directories and it is not under an ignored directory.
 				 */
-				bool contains_tracked =
-					entry_is_prefixed(diff, nitem, oitem);
 				bool recurse_untracked =
 					(delta_type == GIT_DELTA_UNTRACKED &&
 					 (diff->opts.flags & GIT_DIFF_RECURSE_UNTRACKED_DIRS) != 0);
 
 				/* do not advance into directories that contain a .git file */
-				if (!contains_tracked && recurse_untracked) {
+				if (!contains_oitem && recurse_untracked) {
 					git_buf *full = NULL;
 					if (git_iterator_current_workdir_path(new_iter, &full) < 0)
 						goto fail;
@@ -661,7 +675,7 @@ int git_diff__from_iterators(
 						recurse_untracked = false;
 				}
 
-				if (contains_tracked || recurse_untracked) {
+				if (contains_oitem || recurse_untracked) {
 					/* if this directory is ignored, remember it as the
 					 * "ignore_prefix" for processing contained items
 					 */
@@ -707,14 +721,14 @@ int git_diff__from_iterators(
 				goto fail;
 
 			/* if we are generating TYPECHANGE records then check for that
-			 * instead of just generating an ADD/UNTRACKED record
+			 * instead of just generating an ADDED/UNTRACKED record
 			 */
 			if (delta_type != GIT_DELTA_IGNORED &&
 				(diff->opts.flags & GIT_DIFF_INCLUDE_TYPECHANGE_TREES) != 0 &&
-				entry_is_prefixed(diff, nitem, oitem))
+				contains_oitem)
 			{
-				/* this entry was a tree! convert to TYPECHANGE */
-				git_diff_delta *last = diff_delta__last_for_item(diff, oitem);
+				/* this entry was prefixed with a tree - make TYPECHANGE */
+				git_diff_delta *last = diff_delta__last_for_item(diff, nitem);
 				if (last) {
 					last->status = GIT_DELTA_TYPECHANGE;
 					last->old_file.mode = GIT_FILEMODE_TREE;

src/fileops.c

@@ -352,6 +352,7 @@ int git_futils_mkdir_r(const char *path, const char *base, const mode_t mode)
 
 typedef struct {
 	const char *base;
+	size_t baselen;
 	uint32_t flags;
 	int error;
 } futils__rmdir_data;
@@ -443,9 +444,13 @@ static int futils__rmdir_recurs_foreach(void *opaque, git_buf *path)
 
 static int futils__rmdir_empty_parent(void *opaque, git_buf *path)
 {
-	int error = p_rmdir(path->ptr);
+	futils__rmdir_data *data = opaque;
+	int error;
 
-	GIT_UNUSED(opaque);
+	if (git_buf_len(path) <= data->baselen)
+		return GIT_ITEROVER;
+
+	error = p_rmdir(git_buf_cstr(path));
 
 	if (error) {
 		int en = errno;
@@ -457,7 +462,7 @@ static int futils__rmdir_empty_parent(void *opaque, git_buf *path)
 			giterr_clear();
 			error = GIT_ITEROVER;
 		} else {
-			futils__error_cannot_rmdir(path->ptr, NULL);
+			futils__error_cannot_rmdir(git_buf_cstr(path), NULL);
 		}
 	}
 
@@ -476,6 +481,7 @@ int git_futils_rmdir_r(
 		return -1;
 
 	data.base    = base ? base : "";
+	data.baselen = base ? strlen(base) : 0;
 	data.flags   = flags;
 	data.error   = 0;
 

src/index.c

@@ -794,6 +794,44 @@ int git_index_remove(git_index *index, const char *path, int stage)
 	return error;
 }
 
+int git_index_remove_directory(git_index *index, const char *dir, int stage)
+{
+	git_buf pfx = GIT_BUF_INIT;
+	int error = 0;
+	size_t pos;
+	git_index_entry *entry;
+
+	if (git_buf_sets(&pfx, dir) < 0 || git_path_to_dir(&pfx) < 0)
+		return -1;
+
+	git_vector_sort(&index->entries);
+
+	pos = git_index__prefix_position(index, pfx.ptr);
+
+	while (1) {
+		entry = git_vector_get(&index->entries, pos);
+		if (!entry || git__prefixcmp(entry->path, pfx.ptr) != 0)
+			break;
+
+		if (index_entry_stage(entry) != stage) {
+			++pos;
+			continue;
+		}
+
+		git_tree_cache_invalidate_path(index->tree, entry->path);
+
+		if ((error = git_vector_remove(&index->entries, pos)) < 0)
+			break;
+		index_entry_free(entry);
+
+		/* removed entry at 'pos' so we don't need to increment it */
+	}
+
+	git_buf_free(&pfx);
+
+	return error;
+}
+
 static int index_find(git_index *index, const char *path, int stage)
 {
 	struct entry_srch_key srch_key;
@@ -814,7 +852,10 @@ int git_index_find(git_index *index, const char *path)
 
 	if ((pos = git_vector_bsearch2(
 			&index->entries, index->entries_search_path, path)) < 0)
+	{
+		giterr_set(GITERR_INDEX, "Index does not contain %s", path);
 		return pos;
+	}
 
 	/* Since our binary search only looked at path, we may be in the
 	 * middle of a list of stages.

src/iterator.c

@@ -30,8 +30,8 @@
 	(P)->base.start   = start ? git__strdup(start) : NULL; \
 	(P)->base.end     = end ? git__strdup(end) : NULL; \
 	(P)->base.ignore_case = false; \
-	if ((start && !(P)->base.start) || (end && !(P)->base.end)) \
-		return -1; \
+	if ((start && !(P)->base.start) || (end && !(P)->base.end)) { \
+		git__free(P); return -1; } \
 	} while (0)
 
 static int iterator__reset_range(
@@ -988,6 +988,26 @@ fail:
 	return -1;
 }
 
+git_index *git_iterator_index_get_index(git_iterator *iter)
+{
+	if (iter->type == GIT_ITERATOR_INDEX)
+		return ((index_iterator *)iter)->index;
+
+	if (iter->type == GIT_ITERATOR_SPOOLANDSORT &&
+		((spoolandsort_callbacks *)iter->cb)->orig_type == GIT_ITERATOR_INDEX)
+		return ((index_iterator *)iter)->index;
+
+	return NULL;
+}
+
+git_iterator_type_t git_iterator_inner_type(git_iterator *iter)
+{
+	if (iter->type == GIT_ITERATOR_SPOOLANDSORT)
+		return ((spoolandsort_callbacks *)iter->cb)->orig_type;
+
+	return iter->type;
+}
+
 int git_iterator_current_tree_entry(
 	git_iterator *iter, const git_tree_entry **tree_entry)
 {
@@ -1058,8 +1078,8 @@ int git_iterator_advance_into_directory(
 
 	if (iter->type == GIT_ITERATOR_WORKDIR &&
 		wi->entry.path &&
-		S_ISDIR(wi->entry.mode) &&
-		!S_ISGITLINK(wi->entry.mode))
+		(wi->entry.mode == GIT_FILEMODE_TREE ||
+		 wi->entry.mode == GIT_FILEMODE_COMMIT))
 	{
 		if (workdir_iterator__expand_dir(wi) < 0)
 			/* if error loading or if empty, skip the directory. */

src/iterator.h

@@ -193,4 +193,9 @@ extern int git_iterator_cmp(
 extern int git_iterator_current_workdir_path(
 	git_iterator *iter, git_buf **path);
 
+
+extern git_index *git_iterator_index_get_index(git_iterator *iter);
+
+extern git_iterator_type_t git_iterator_inner_type(git_iterator *iter);
+
 #endif

src/oid.c

@@ -95,11 +95,14 @@ char *git_oid_tostr(char *out, size_t n, const git_oid *oid)
 {
 	char str[GIT_OID_HEXSZ];
 
-	if (!out || n == 0 || !oid)
+	if (!out || n == 0)
 		return "";
 
 	n--; /* allow room for terminating NUL */
 
+	if (oid == NULL)
+		n = 0;
+
 	if (n > 0) {
 		git_oid_fmt(str, oid);
 		if (n > GIT_OID_HEXSZ)

src/reset.c

@@ -62,13 +62,10 @@ int git_reset(
 	git_object *commit = NULL;
 	git_index *index = NULL;
 	git_tree *tree = NULL;
-	int error;
+	int error = 0;
 	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
 
 	assert(repo && target);
-	assert(reset_type == GIT_RESET_SOFT
-		|| reset_type == GIT_RESET_MIXED
-		|| reset_type == GIT_RESET_HARD);
 
 	if (git_object_owner(target) != repo) {
 		giterr_set(GITERR_OBJECT,
@@ -76,57 +73,51 @@ int git_reset(
 		return -1;
 	}
 
-	if (reset_type != GIT_RESET_SOFT
-		&& (error = git_repository__ensure_not_bare(
-			repo,
+	if (reset_type != GIT_RESET_SOFT &&
+		(error = git_repository__ensure_not_bare(repo,
 			reset_type == GIT_RESET_MIXED ? "reset mixed" : "reset hard")) < 0)
 		return error;
 
-	if ((error = git_object_peel(&commit, target, GIT_OBJ_COMMIT)) < 0)
-		goto cleanup;
-
-	if ((error = git_repository_index(&index, repo)) < 0)
+	if ((error = git_object_peel(&commit, target, GIT_OBJ_COMMIT)) < 0 ||
+		(error = git_repository_index(&index, repo)) < 0 ||
+		(error = git_commit_tree(&tree, (git_commit *)commit)) < 0)
 		goto cleanup;
 
 	if (reset_type == GIT_RESET_SOFT &&
 		(git_repository_state(repo) == GIT_REPOSITORY_STATE_MERGE ||
-		git_index_has_conflicts(index))) {
-			giterr_set(GITERR_OBJECT, "%s (soft) while in the middle of a merge.", ERROR_MSG);
+		 git_index_has_conflicts(index)))
+	{
+		giterr_set(GITERR_OBJECT, "%s (soft) in the middle of a merge.", ERROR_MSG);
 		error = GIT_EUNMERGED;
 		goto cleanup;
 	}
 
+	/* move HEAD to the new target */
 	if ((error = update_head(repo, commit)) < 0)
 		goto cleanup;
 
-	if (reset_type == GIT_RESET_SOFT) {
-		error = 0;
-		goto cleanup;
-	}
+	if (reset_type == GIT_RESET_HARD) {
+		/* overwrite working directory with HEAD */
+		opts.checkout_strategy = GIT_CHECKOUT_FORCE;
 
-	if ((error = git_commit_tree(&tree, (git_commit *)commit)) < 0)
+		if ((error = git_checkout_tree(repo, (git_object *)tree, &opts)) < 0)
 			goto cleanup;
+	}
 
-	if ((error = git_index_read_tree(index, tree)) < 0)
-		goto cleanup;
+	if (reset_type > GIT_RESET_SOFT) {
+		/* reset index to the target content */
 
-	if ((error = git_index_write(index)) < 0)
+		if ((error = git_repository_index(&index, repo)) < 0 ||
+			(error = git_index_read_tree(index, tree)) < 0 ||
+			(error = git_index_write(index)) < 0)
 			goto cleanup;
 
 		if ((error = git_repository_merge_cleanup(repo)) < 0) {
-		giterr_set(GITERR_REPOSITORY, "%s - Failed to clean up merge data.", ERROR_MSG);
+			giterr_set(GITERR_INDEX, "%s - failed to clean up merge data", ERROR_MSG);
 			goto cleanup;
 		}
-
-	if (reset_type == GIT_RESET_MIXED) {
-		error = 0;
-		goto cleanup;
 	}
 
-	opts.checkout_strategy = GIT_CHECKOUT_FORCE;
-
-	error = git_checkout_index(repo, NULL, &opts);
-
 cleanup:
 	git_object_free(commit);
 	git_index_free(index);

src/submodule.c

@@ -66,7 +66,7 @@ __KHASH_IMPL(
 	str, static kh_inline, const char *, void *, 1,
 	str_hash_no_trailing_slash, str_equal_no_trailing_slash);
 
-static int load_submodule_config(git_repository *repo, bool force);
+static int load_submodule_config(git_repository *repo);
 static git_config_backend *open_gitmodules(git_repository *, bool, const git_oid *);
 static int lookup_head_remote(git_buf *url, git_repository *repo);
 static int submodule_get(git_submodule **, git_repository *, const char *, const char *);
@@ -106,7 +106,7 @@ int git_submodule_lookup(
 
 	assert(repo && name);
 
-	if ((error = load_submodule_config(repo, false)) < 0)
+	if ((error = load_submodule_config(repo)) < 0)
 		return error;
 
 	pos = git_strmap_lookup_index(repo->submodules, name);
@@ -148,7 +148,7 @@ int git_submodule_foreach(
 
 	assert(repo && callback);
 
-	if ((error = load_submodule_config(repo, false)) < 0)
+	if ((error = load_submodule_config(repo)) < 0)
 		return error;
 
 	git_strmap_foreach_value(repo->submodules, sm, {
@@ -708,7 +708,8 @@ int git_submodule_open(
 int git_submodule_reload_all(git_repository *repo)
 {
 	assert(repo);
-	return load_submodule_config(repo, true);
+	git_submodule_config_free(repo);
+	return load_submodule_config(repo);
 }
 
 int git_submodule_reload(git_submodule *submodule)
@@ -829,6 +830,20 @@ int git_submodule_status(
 	return error;
 }
 
+int git_submodule_location(
+	unsigned int *location_status,
+	git_submodule *submodule)
+{
+	assert(location_status && submodule);
+
+	*location_status = submodule->flags &
+		(GIT_SUBMODULE_STATUS_IN_HEAD | GIT_SUBMODULE_STATUS_IN_INDEX |
+		 GIT_SUBMODULE_STATUS_IN_CONFIG | GIT_SUBMODULE_STATUS_IN_WD);
+
+	return 0;
+}
+
+
 /*
  * INTERNAL FUNCTIONS
  */
@@ -1225,14 +1240,14 @@ static git_config_backend *open_gitmodules(
 	return mods;
 }
 
-static int load_submodule_config(git_repository *repo, bool force)
+static int load_submodule_config(git_repository *repo)
 {
 	int error;
 	git_oid gitmodules_oid;
 	git_buf path = GIT_BUF_INIT;
 	git_config_backend *mods = NULL;
 
-	if (repo->submodules && !force)
+	if (repo->submodules)
 		return 0;
 
 	memset(&gitmodules_oid, 0, sizeof(gitmodules_oid));

tests-clar/checkout/head.c

 #include "clar_libgit2.h"
 #include "refs.h"
 #include "repo/repo_helpers.h"
+#include "path.h"
+#include "fileops.h"
 
 static git_repository *g_repo;
 
@@ -14,9 +16,48 @@ void test_checkout_head__cleanup(void)
 	cl_git_sandbox_cleanup();
 }
 
-void test_checkout_head__checking_out_an_orphaned_head_returns_GIT_EORPHANEDHEAD(void)
+void test_checkout_head__orphaned_head_returns_GIT_EORPHANEDHEAD(void)
 {
 	make_head_orphaned(g_repo, NON_EXISTING_HEAD);
 
 	cl_assert_equal_i(GIT_EORPHANEDHEAD, git_checkout_head(g_repo, NULL));
 }
+
+void test_checkout_head__with_index_only_tree(void)
+{
+	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+	git_index *index;
+
+	/* let's start by getting things into a known state */
+
+	opts.checkout_strategy = GIT_CHECKOUT_FORCE;
+	cl_git_pass(git_checkout_head(g_repo, &opts));
+
+	/* now let's stage some new stuff including a new directory */
+
+	cl_git_pass(git_repository_index(&index, g_repo));
+
+	p_mkdir("testrepo/newdir", 0777);
+    cl_git_mkfile("testrepo/newdir/newfile.txt", "new file\n");
+
+	cl_git_pass(git_index_add_from_workdir(index, "newdir/newfile.txt"));
+	cl_git_pass(git_index_write(index));
+
+	cl_assert(git_path_isfile("testrepo/newdir/newfile.txt"));
+	cl_assert(git_index_get_bypath(index, "newdir/newfile.txt", 0) != NULL);
+
+	git_index_free(index);
+
+	/* okay, so now we have staged this new file; let's see if we can remove */
+
+	opts.checkout_strategy = GIT_CHECKOUT_FORCE | GIT_CHECKOUT_REMOVE_UNTRACKED;
+	cl_git_pass(git_checkout_head(g_repo, &opts));
+
+	cl_git_pass(git_repository_index(&index, g_repo));
+	cl_git_pass(git_index_read(index)); /* reload if needed */
+
+	cl_assert(!git_path_isfile("testrepo/newdir/newfile.txt"));
+	cl_assert(git_index_get_bypath(index, "newdir/newfile.txt", 0) == NULL);
+
+	git_index_free(index);
+}

tests-clar/checkout/tree.c

@@ -2,6 +2,8 @@
 
 #include "git2/checkout.h"
 #include "repository.h"
+#include "buffer.h"
+#include "fileops.h"
 
 static git_repository *g_repo;
 static git_checkout_opts g_opts;
@@ -12,7 +14,7 @@ void test_checkout_tree__initialize(void)
 	g_repo = cl_git_sandbox_init("testrepo");
 
 	GIT_INIT_STRUCTURE(&g_opts, GIT_CHECKOUT_OPTS_VERSION);
-	g_opts.checkout_strategy = GIT_CHECKOUT_SAFE;
+	g_opts.checkout_strategy = GIT_CHECKOUT_SAFE_CREATE;
 }
 
 void test_checkout_tree__cleanup(void)
@@ -48,6 +50,34 @@ void test_checkout_tree__can_checkout_a_subdirectory_from_a_commit(void)
 	cl_assert_equal_i(true, git_path_isfile("./testrepo/ab/de/fgh/1.txt"));
 }
 
+void test_checkout_tree__can_checkout_and_remove_directory(void)
+{
+	cl_assert_equal_i(false, git_path_isdir("./testrepo/ab/"));
+
+	/* Checkout brach "subtrees" and update HEAD, so that HEAD matches the
+	 * current working tree
+	 */
+	cl_git_pass(git_revparse_single(&g_object, g_repo, "subtrees"));
+	cl_git_pass(git_checkout_tree(g_repo, g_object, &g_opts));
+
+	cl_git_pass(git_repository_set_head(g_repo, "refs/heads/subtrees"));
+
+	cl_assert_equal_i(true, git_path_isdir("./testrepo/ab/"));
+	cl_assert_equal_i(true, git_path_isfile("./testrepo/ab/de/2.txt"));
+	cl_assert_equal_i(true, git_path_isfile("./testrepo/ab/de/fgh/1.txt"));
+
+	/* Checkout brach "master" and update HEAD, so that HEAD matches the
+	 * current working tree
+	 */
+	cl_git_pass(git_revparse_single(&g_object, g_repo, "master"));
+	cl_git_pass(git_checkout_tree(g_repo, g_object, &g_opts));
+
+	cl_git_pass(git_repository_set_head(g_repo, "refs/heads/master"));
+
+	/* This directory should no longer exist */
+	cl_assert_equal_i(false, git_path_isdir("./testrepo/ab/"));
+}
+
 void test_checkout_tree__can_checkout_a_subdirectory_from_a_subtree(void)
 {
 	char *entries[] = { "de/" };
@@ -85,3 +115,193 @@ void test_checkout_tree__calls_progress_callback(void)
 
 	cl_assert_equal_i(was_called, true);
 }
+
+void test_checkout_tree__doesnt_write_unrequested_files_to_worktree(void)
+{
+  git_oid master_oid;
+  git_oid chomped_oid;
+  git_commit* p_master_commit;
+  git_commit* p_chomped_commit;
+  git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+
+  git_oid_fromstr(&master_oid, "a65fedf39aefe402d3bb6e24df4d4f5fe4547750");
+  git_oid_fromstr(&chomped_oid, "e90810b8df3e80c413d903f631643c716887138d");
+  cl_git_pass(git_commit_lookup(&p_master_commit, g_repo, &master_oid));
+  cl_git_pass(git_commit_lookup(&p_chomped_commit, g_repo, &chomped_oid));
+
+  /* GIT_CHECKOUT_NONE should not add any file to the working tree from the
+   * index as it is supposed to be a dry run.
+   */
+  opts.checkout_strategy = GIT_CHECKOUT_NONE;
+  git_checkout_tree(g_repo, (git_object*)p_chomped_commit, &opts);
+  cl_assert_equal_i(false, git_path_isfile("testrepo/readme.txt"));
+}
+
+static void assert_on_branch(git_repository *repo, const char *branch)
+{
+	git_reference *head;
+	git_buf bname = GIT_BUF_INIT;
+
+	cl_git_pass(git_reference_lookup(&head, repo, GIT_HEAD_FILE));
+	cl_assert_(git_reference_type(head) == GIT_REF_SYMBOLIC, branch);
+
+	cl_git_pass(git_buf_joinpath(&bname, "refs/heads", branch));
+	cl_assert_equal_s(bname.ptr, git_reference_symbolic_target(head));
+
+	git_reference_free(head);
+	git_buf_free(&bname);
+}
+
+void test_checkout_tree__can_switch_branches(void)
+{
+	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+	git_oid oid;
+	git_object *obj = NULL;
+
+	assert_on_branch(g_repo, "master");
+
+	/* do first checkout with FORCE because we don't know if testrepo
+	 * base data is clean for a checkout or not
+	 */
+	opts.checkout_strategy = GIT_CHECKOUT_FORCE;
+
+	cl_git_pass(git_reference_name_to_id(&oid, g_repo, "refs/heads/dir"));
+	cl_git_pass(git_object_lookup(&obj, g_repo, &oid, GIT_OBJ_ANY));
+
+	cl_git_pass(git_checkout_tree(g_repo, obj, &opts));
+	cl_git_pass(git_repository_set_head(g_repo, "refs/heads/dir"));
+
+	cl_assert(git_path_isfile("testrepo/README"));
+	cl_assert(git_path_isfile("testrepo/branch_file.txt"));
+	cl_assert(git_path_isfile("testrepo/new.txt"));
+	cl_assert(git_path_isfile("testrepo/a/b.txt"));
+
+	cl_assert(!git_path_isdir("testrepo/ab"));
+
+	assert_on_branch(g_repo, "dir");
+
+	git_object_free(obj);
+
+	/* do second checkout safe because we should be clean after first */
+	opts.checkout_strategy = GIT_CHECKOUT_SAFE;
+
+	cl_git_pass(git_reference_name_to_id(&oid, g_repo, "refs/heads/subtrees"));
+	cl_git_pass(git_object_lookup(&obj, g_repo, &oid, GIT_OBJ_ANY));
+
+	cl_git_pass(git_checkout_tree(g_repo, obj, &opts));
+	cl_git_pass(git_repository_set_head(g_repo, "refs/heads/subtrees"));
+
+	cl_assert(git_path_isfile("testrepo/README"));
+	cl_assert(git_path_isfile("testrepo/branch_file.txt"));
+	cl_assert(git_path_isfile("testrepo/new.txt"));
+	cl_assert(git_path_isfile("testrepo/ab/4.txt"));
+	cl_assert(git_path_isfile("testrepo/ab/c/3.txt"));
+	cl_assert(git_path_isfile("testrepo/ab/de/2.txt"));
+	cl_assert(git_path_isfile("testrepo/ab/de/fgh/1.txt"));
+
+	cl_assert(!git_path_isdir("testrepo/a"));
+
+	assert_on_branch(g_repo, "subtrees");
+
+	git_object_free(obj);
+}
+
+void test_checkout_tree__can_remove_untracked(void)
+{
+	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+
+	opts.checkout_strategy = GIT_CHECKOUT_SAFE | GIT_CHECKOUT_REMOVE_UNTRACKED;
+
+	cl_git_mkfile("testrepo/untracked_file", "as you wish");
+	cl_assert(git_path_isfile("testrepo/untracked_file"));
+
+	cl_git_pass(git_checkout_head(g_repo, &opts));
+
+	cl_assert(!git_path_isfile("testrepo/untracked_file"));
+}
+
+void test_checkout_tree__can_remove_ignored(void)
+{
+	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+	int ignored = 0;
+
+	opts.checkout_strategy = GIT_CHECKOUT_SAFE | GIT_CHECKOUT_REMOVE_IGNORED;
+
+	cl_git_mkfile("testrepo/ignored_file", "as you wish");
+
+	cl_git_pass(git_ignore_add_rule(g_repo, "ignored_file\n"));
+
+	cl_git_pass(git_ignore_path_is_ignored(&ignored, g_repo, "ignored_file"));
+	cl_assert_equal_i(1, ignored);
+
+	cl_assert(git_path_isfile("testrepo/ignored_file"));
+
+	cl_git_pass(git_checkout_head(g_repo, &opts));
+
+	cl_assert(!git_path_isfile("testrepo/ignored_file"));
+}
+
+/* this is essentially the code from git__unescape modified slightly */
+static void strip_cr_from_buf(git_buf *buf)
+{
+	char *scan, *pos = buf->ptr;
+
+	for (scan = pos; *scan; pos++, scan++) {
+		if (*scan == '\r')
+			scan++; /* skip '\r' */
+		if (pos != scan)
+			*pos = *scan;
+	}
+
+	*pos = '\0';
+	buf->size = (pos - buf->ptr);
+}
+
+void test_checkout_tree__can_update_only(void)
+{
+	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+	git_oid oid;
+	git_object *obj = NULL;
+	git_buf buf = GIT_BUF_INIT;
+
+	/* first let's get things into a known state - by checkout out the HEAD */
+
+	assert_on_branch(g_repo, "master");
+
+	opts.checkout_strategy = GIT_CHECKOUT_FORCE;
+	cl_git_pass(git_checkout_head(g_repo, &opts));
+
+	cl_assert(!git_path_isdir("testrepo/a"));
+
+	cl_git_pass(git_futils_readbuffer(&buf, "testrepo/branch_file.txt"));
+	strip_cr_from_buf(&buf);
+	cl_assert_equal_s("hi\nbye!\n", buf.ptr);
+	git_buf_free(&buf);
+
+	/* now checkout branch but with update only */
+
+	opts.checkout_strategy = GIT_CHECKOUT_SAFE | GIT_CHECKOUT_UPDATE_ONLY;
+
+	cl_git_pass(git_reference_name_to_id(&oid, g_repo, "refs/heads/dir"));
+	cl_git_pass(git_object_lookup(&obj, g_repo, &oid, GIT_OBJ_ANY));
+
+	cl_git_pass(git_checkout_tree(g_repo, obj, &opts));
+	cl_git_pass(git_repository_set_head(g_repo, "refs/heads/dir"));
+
+	assert_on_branch(g_repo, "dir");
+
+	/* this normally would have been created (which was tested separately in
+	 * the test_checkout_tree__can_switch_branches test), but with
+	 * UPDATE_ONLY it will not have been created.
+	 */
+	cl_assert(!git_path_isdir("testrepo/a"));
+
+	/* but this file still should have been updated */
+	cl_git_pass(git_futils_readbuffer(&buf, "testrepo/branch_file.txt"));
+	strip_cr_from_buf(&buf);
+	cl_assert_equal_s("hi\n", buf.ptr);
+
+	git_buf_free(&buf);
+
+	git_object_free(obj);
+}

tests-clar/checkout/typechange.c

@@ -2,6 +2,7 @@
 #include "git2/checkout.h"
 #include "path.h"
 #include "posix.h"
+#include "fileops.h"
 
 static git_repository *g_repo = NULL;
 
@@ -34,28 +35,97 @@ void test_checkout_typechange__cleanup(void)
 	cl_fixture_cleanup("submod2_target");
 }
 
-void test_checkout_typechange__checkout_typechanges(void)
+static void assert_file_exists(const char *path)
+{
+	cl_assert_(git_path_isfile(path), path);
+}
+
+static void assert_dir_exists(const char *path)
+{
+	cl_assert_(git_path_isdir(path), path);
+}
+
+static void assert_workdir_matches_tree(
+	git_repository *repo, const git_oid *id, const char *root, bool recurse)
+{
+	git_object *obj;
+	git_tree *tree;
+	size_t i, max_i;
+	git_buf path = GIT_BUF_INIT;
+
+	if (!root)
+		root = git_repository_workdir(repo);
+	cl_assert(root);
+
+	cl_git_pass(git_object_lookup(&obj, repo, id, GIT_OBJ_ANY));
+	cl_git_pass(git_object_peel((git_object **)&tree, obj, GIT_OBJ_TREE));
+	git_object_free(obj);
+
+	max_i = git_tree_entrycount(tree);
+
+	for (i = 0; i < max_i; ++i) {
+		const git_tree_entry *te = git_tree_entry_byindex(tree, i);
+		cl_assert(te);
+
+		cl_git_pass(git_buf_joinpath(&path, root, git_tree_entry_name(te)));
+
+		switch (git_tree_entry_type(te)) {
+		case GIT_OBJ_COMMIT:
+			assert_dir_exists(path.ptr);
+			break;
+		case GIT_OBJ_TREE:
+			assert_dir_exists(path.ptr);
+			if (recurse)
+				assert_workdir_matches_tree(
+					repo, git_tree_entry_id(te), path.ptr, true);
+			break;
+		case GIT_OBJ_BLOB:
+			switch (git_tree_entry_filemode(te)) {
+			case GIT_FILEMODE_BLOB:
+			case GIT_FILEMODE_BLOB_EXECUTABLE:
+				assert_file_exists(path.ptr);
+				/* because of cross-platform, don't confirm exec bit yet */
+				break;
+			case GIT_FILEMODE_LINK:
+				cl_assert_(git_path_exists(path.ptr), path.ptr);
+				/* because of cross-platform, don't confirm link yet */
+				break;
+			default:
+				cl_assert(false); /* really?! */
+			}
+			break;
+		default:
+			cl_assert(false); /* really?!! */
+		}
+	}
+
+	git_tree_free(tree);
+	git_buf_free(&path);
+}
+
+void test_checkout_typechange__checkout_typechanges_safe(void)
 {
 	int i;
 	git_object *obj;
 	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
 
+	for (i = 0; g_typechange_oids[i] != NULL; ++i) {
+		cl_git_pass(git_revparse_single(&obj, g_repo, g_typechange_oids[i]));
+
 		opts.checkout_strategy = GIT_CHECKOUT_FORCE;
 
-	/* if you don't include GIT_CHECKOUT_REMOVE_UNTRACKED then on the final
-	 * checkout which is supposed to remove all the files, we will not
-	 * actually remove them!
+		/* There are bugs in some submodule->tree changes that prevent
+		 * SAFE from passing here, even though the following should work:
 		 */
-
-	for (i = 0; g_typechange_oids[i] != NULL; ++i) {
-		cl_git_pass(git_revparse_single(&obj, g_repo, g_typechange_oids[i]));
-		/* fprintf(stderr, "checking out '%s'\n", g_typechange_oids[i]); */
+		/* !i ? GIT_CHECKOUT_FORCE : GIT_CHECKOUT_SAFE; */
 
 		cl_git_pass(git_checkout_tree(g_repo, obj, &opts));
 
 		cl_git_pass(
 			git_repository_set_head_detached(g_repo, git_object_id(obj)));
 
+		assert_workdir_matches_tree(g_repo, git_object_id(obj), NULL, true);
+
 		git_object_free(obj);
 
 		if (!g_typechange_empty[i]) {
@@ -75,3 +145,96 @@ void test_checkout_typechange__checkout_typechanges(void)
 		}
 	}
 }
+
+typedef struct {
+	int conflicts;
+	int dirty;
+	int updates;
+	int untracked;
+	int ignored;
+} notify_counts;
+
+static int notify_counter(
+	git_checkout_notify_t why,
+	const char *path,
+	const git_diff_file *baseline,
+	const git_diff_file *target,
+	const git_diff_file *workdir,
+	void *payload)
+{
+	notify_counts *cts = payload;
+
+	GIT_UNUSED(path);
+	GIT_UNUSED(baseline);
+	GIT_UNUSED(target);
+	GIT_UNUSED(workdir);
+
+	switch (why) {
+	case GIT_CHECKOUT_NOTIFY_CONFLICT:  cts->conflicts++; break;
+	case GIT_CHECKOUT_NOTIFY_DIRTY:     cts->dirty++;     break;
+	case GIT_CHECKOUT_NOTIFY_UPDATED:   cts->updates++;   break;
+	case GIT_CHECKOUT_NOTIFY_UNTRACKED: cts->untracked++; break;
+	case GIT_CHECKOUT_NOTIFY_IGNORED:   cts->ignored++;   break;
+	default: break;
+	}
+
+	return 0;
+}
+
+static void force_create_file(const char *file)
+{
+	int error = git_futils_rmdir_r(file, NULL,
+		GIT_RMDIR_REMOVE_FILES | GIT_RMDIR_REMOVE_BLOCKERS);
+	cl_assert(!error || error == GIT_ENOTFOUND);
+	cl_git_pass(git_futils_mkpath2file(file, 0777));
+	cl_git_rewritefile(file, "yowza!");
+}
+
+void test_checkout_typechange__checkout_with_conflicts(void)
+{
+	int i;
+	git_object *obj;
+	git_checkout_opts opts = GIT_CHECKOUT_OPTS_INIT;
+	notify_counts cts = {0};
+
+	opts.notify_flags =
+		GIT_CHECKOUT_NOTIFY_CONFLICT | GIT_CHECKOUT_NOTIFY_UNTRACKED;
+	opts.notify_cb = notify_counter;
+	opts.notify_payload = &cts;
+
+	for (i = 0; g_typechange_oids[i] != NULL; ++i) {
+		cl_git_pass(git_revparse_single(&obj, g_repo, g_typechange_oids[i]));
+
+		force_create_file("typechanges/a/blocker");
+		force_create_file("typechanges/b");
+		force_create_file("typechanges/c/sub/sub/file");
+		git_futils_rmdir_r("typechanges/d", NULL, GIT_RMDIR_REMOVE_FILES);
+		p_mkdir("typechanges/d", 0777); /* intentionally empty dir */
+		force_create_file("typechanges/untracked");
+
+		opts.checkout_strategy = GIT_CHECKOUT_SAFE_CREATE;
+		memset(&cts, 0, sizeof(cts));
+
+		cl_git_fail(git_checkout_tree(g_repo, obj, &opts));
+		cl_assert(cts.conflicts > 0);
+		cl_assert(cts.untracked > 0);
+
+		opts.checkout_strategy =
+			GIT_CHECKOUT_FORCE | GIT_CHECKOUT_REMOVE_UNTRACKED;
+		memset(&cts, 0, sizeof(cts));
+
+		cl_assert(git_path_exists("typechanges/untracked"));
+
+		cl_git_pass(git_checkout_tree(g_repo, obj, &opts));
+		cl_assert_equal_i(0, cts.conflicts);
+
+		cl_assert(!git_path_exists("typechanges/untracked"));
+
+		cl_git_pass(
+			git_repository_set_head_detached(g_repo, git_object_id(obj)));
+
+		assert_workdir_matches_tree(g_repo, git_object_id(obj), NULL, true);
+
+		git_object_free(obj);
+	}
+}

tests-clar/clar_libgit2.c

@@ -2,6 +2,16 @@
 #include "posix.h"
 #include "path.h"
 
+void cl_git_report_failure(
+	int error, const char *file, int line, const char *fncall)
+{
+	char msg[4096];
+	const git_error *last = giterr_last();
+	p_snprintf(msg, 4096, "error %d - %s",
+		error, last ? last->message : "<no message>");
+	clar__assert(0, file, line, fncall, msg, 1);
+}
+
 void cl_git_mkfile(const char *filename, const char *content)
 {
 	int fd;

tests-clar/clar_libgit2.h

@@ -6,17 +6,17 @@
 #include "common.h"
 
 /**
- * Special wrapper for `clar_must_pass` that passes
- * the last library error as the test failure message.
+ * Replace for `clar_must_pass` that passes the last library error as the
+ * test failure message.
  *
- * Use this wrapper around all `git_` library calls that
- * return error codes!
+ * Use this wrapper around all `git_` library calls that return error codes!
  */
 #define cl_git_pass(expr) do { \
+	int _lg2_error; \
 	giterr_clear(); \
-	if ((expr) != 0) \
-		clar__assert(0, __FILE__, __LINE__, "Function call failed: " #expr, giterr_last() ? giterr_last()->message : NULL, 1); \
-	} while(0)
+	if ((_lg2_error = (expr)) != 0) \
+		cl_git_report_failure(_lg2_error, __FILE__, __LINE__, "Function call failed: " #expr); \
+	} while (0)
 
 /**
  * Wrapper for `clar_must_fail` -- this one is
@@ -25,6 +25,10 @@
  */
 #define cl_git_fail(expr) cl_must_fail(expr)
 
+#define cl_git_fail_with(expr, error) cl_assert_equal_i(error,expr)
+
+void cl_git_report_failure(int, const char *, int, const char *);
+
 #define cl_assert_equal_sz(sz1,sz2) cl_assert((sz1) == (sz2))
 
 /*

tests-clar/index/tests.c

@@ -290,3 +290,86 @@ void test_index_tests__write_invalid_filename(void)
 
 	cl_fixture_cleanup("read_tree");
 }
+
+void test_index_tests__remove_entry(void)
+{
+	git_repository *repo;
+	git_index *index;
+
+	p_mkdir("index_test", 0770);
+
+	cl_git_pass(git_repository_init(&repo, "index_test", 0));
+	cl_git_pass(git_repository_index(&index, repo));
+	cl_assert(git_index_entrycount(index) == 0);
+
+	cl_git_mkfile("index_test/hello", NULL);
+	cl_git_pass(git_index_add_from_workdir(index, "hello"));
+	cl_git_pass(git_index_write(index));
+
+	cl_git_pass(git_index_read(index)); /* reload */
+	cl_assert(git_index_entrycount(index) == 1);
+	cl_assert(git_index_get_bypath(index, "hello", 0) != NULL);
+
+	cl_git_pass(git_index_remove(index, "hello", 0));
+	cl_git_pass(git_index_write(index));
+
+	cl_git_pass(git_index_read(index)); /* reload */
+	cl_assert(git_index_entrycount(index) == 0);
+	cl_assert(git_index_get_bypath(index, "hello", 0) == NULL);
+
+	git_index_free(index);
+	git_repository_free(repo);
+	cl_fixture_cleanup("index_test");
+}
+
+void test_index_tests__remove_directory(void)
+{
+	git_repository *repo;
+	git_index *index;
+
+	p_mkdir("index_test", 0770);
+
+	cl_git_pass(git_repository_init(&repo, "index_test", 0));
+	cl_git_pass(git_repository_index(&index, repo));
+	cl_assert_equal_i(0, (int)git_index_entrycount(index));
+
+	p_mkdir("index_test/a", 0770);
+	cl_git_mkfile("index_test/a/1.txt", NULL);
+	cl_git_mkfile("index_test/a/2.txt", NULL);
+	cl_git_mkfile("index_test/a/3.txt", NULL);
+	cl_git_mkfile("index_test/b.txt", NULL);
+
+	cl_git_pass(git_index_add_from_workdir(index, "a/1.txt"));
+	cl_git_pass(git_index_add_from_workdir(index, "a/2.txt"));
+	cl_git_pass(git_index_add_from_workdir(index, "a/3.txt"));
+	cl_git_pass(git_index_add_from_workdir(index, "b.txt"));
+	cl_git_pass(git_index_write(index));
+
+	cl_git_pass(git_index_read(index)); /* reload */
+	cl_assert_equal_i(4, (int)git_index_entrycount(index));
+	cl_assert(git_index_get_bypath(index, "a/1.txt", 0) != NULL);
+	cl_assert(git_index_get_bypath(index, "a/2.txt", 0) != NULL);
+	cl_assert(git_index_get_bypath(index, "b.txt", 0) != NULL);
+
+	cl_git_pass(git_index_remove(index, "a/1.txt", 0));
+	cl_git_pass(git_index_write(index));
+
+	cl_git_pass(git_index_read(index)); /* reload */
+	cl_assert_equal_i(3, (int)git_index_entrycount(index));
+	cl_assert(git_index_get_bypath(index, "a/1.txt", 0) == NULL);
+	cl_assert(git_index_get_bypath(index, "a/2.txt", 0) != NULL);
+	cl_assert(git_index_get_bypath(index, "b.txt", 0) != NULL);
+
+	cl_git_pass(git_index_remove_directory(index, "a", 0));
+	cl_git_pass(git_index_write(index));
+
+	cl_git_pass(git_index_read(index)); /* reload */
+	cl_assert_equal_i(1, (int)git_index_entrycount(index));
+	cl_assert(git_index_get_bypath(index, "a/1.txt", 0) == NULL);
+	cl_assert(git_index_get_bypath(index, "a/2.txt", 0) == NULL);
+	cl_assert(git_index_get_bypath(index, "b.txt", 0) != NULL);
+
+	git_index_free(index);
+	git_repository_free(repo);
+	cl_fixture_cleanup("index_test");
+}

tests-clar/object/raw/convert.c

@@ -21,9 +21,9 @@ void test_object_raw_convert__succeed_on_oid_to_string_conversion(void)
 	str = git_oid_tostr(out, 0, &in);
 	cl_assert(str && *str == '\0' && str != out);
 
-	/* NULL oid pointer, returns static empty string */
+	/* NULL oid pointer, sets existing buffer to empty string */
 	str = git_oid_tostr(out, sizeof(out), NULL);
-	cl_assert(str && *str == '\0' && str != out);
+	cl_assert(str && *str == '\0' && str == out);
 
 	/* n == 1, returns out as an empty string */
 	str = git_oid_tostr(out, 1, &in);

tests-clar/online/clone.c

@@ -91,7 +91,7 @@ void test_online_clone__can_checkout_a_cloned_repo(void)
 	bool checkout_progress_cb_was_called = false,
 		  fetch_progress_cb_was_called = false;
 
-	g_options.checkout_opts.checkout_strategy = GIT_CHECKOUT_SAFE;
+	g_options.checkout_opts.checkout_strategy = GIT_CHECKOUT_SAFE_CREATE;
 	g_options.checkout_opts.progress_cb = &checkout_progress;
 	g_options.checkout_opts.progress_payload = &checkout_progress_cb_was_called;
 	g_options.fetch_progress_cb = &fetch_progress;

tests-clar/reset/hard.c

@@ -54,9 +54,7 @@ void test_reset_hard__resetting_reverts_modified_files(void)
 	static const char *after[4] = {
 		"current_file\n",
 		"modified_file\n",
-		/* wrong value because reset is still slightly incorrect */
-		"staged_new_file\n",
-		/* right value: NULL, */
+		NULL,
 		"staged_changes_modified_file\n"
 	};
 	const char *wd = git_repository_workdir(repo);

tests-clar/stash/drop.c

@@ -36,15 +36,22 @@ static void push_three_states(void)
 	cl_git_pass(git_repository_index(&index, repo));
 	cl_git_pass(git_index_add_from_workdir(index, "zero.txt"));
 	commit_staged_files(&oid, index, signature);
+	cl_assert(git_path_exists("stash/zero.txt"));
 
 	cl_git_mkfile("stash/one.txt", "content\n");
 	cl_git_pass(git_stash_save(&oid, repo, signature, "First", GIT_STASH_INCLUDE_UNTRACKED));
+	cl_assert(!git_path_exists("stash/one.txt"));
+	cl_assert(git_path_exists("stash/zero.txt"));
 
 	cl_git_mkfile("stash/two.txt", "content\n");
 	cl_git_pass(git_stash_save(&oid, repo, signature, "Second", GIT_STASH_INCLUDE_UNTRACKED));
+	cl_assert(!git_path_exists("stash/two.txt"));
+	cl_assert(git_path_exists("stash/zero.txt"));
 
 	cl_git_mkfile("stash/three.txt", "content\n");
 	cl_git_pass(git_stash_save(&oid, repo, signature, "Third", GIT_STASH_INCLUDE_UNTRACKED));
+	cl_assert(!git_path_exists("stash/three.txt"));
+	cl_assert(git_path_exists("stash/zero.txt"));
 
 	git_index_free(index);
 }