When the end of the host is reached, and we're at the colon separating the host with the port (ie, there is no numeric port) then do not error. This is allowed by RFC 3986.
