This commit fixes a regression introduced by

        20a2b02d

The commit introduced an optimization for finding references using a
glob: rather than iterating over all references and matching each one
against the glob, we would iterate only over references within the
directory common to all possible references which may match against the
glob.

However, contrary to the `ref/` directory, which was the previous entry
point for the iteration, this directory may not exist. In this case, the
optimization causes an error (`ENOENT`) rather than the iterator simply
yielding no references.

This patch fixes the regression by checkign for this specific case.