There are a few cases where diff should leave directories in
the diff list if we want to match core git, such as when the
directory contains a .git dir.  That feature was lost when I
introduced some of the new submodule handling.

This restores that and then fixes a couple of related to diff
output that are triggered by having diffs with directories in
them.

Also, this adds a new flag that can be passed to diff if you
want diff output to actually include the file content of any
untracked files.