When we're not doing pathspec matching, we let the iterator handle
file matching for us.  However, we can only trust the iterator to
return *files* that match the pattern, because the iterator must
return directories that are not strictly in the pathlist, but that
are the parents of files that match the pattern, so that diff can
later recurse into them.

Thus, diff must examine non-files explicitly before including them
in the delta list.