The GIT_MODE_TYPE macro was looking at all bits above the
permissions, but it should really just look at the top bits so
that it will give the right results for a setgid or setuid entry.

Since we're now using these macros in the tests, this was causing
a test failure on platforms that don't support setgid.