Commit 6777db6d by Neil Booth Committed by Zack Weinberg

cpplib.h: "~=" is not a single pp-token.

2000-04-26  Neil Booth  <NeilB@earthling.net>

	* cpplib.h: "~=" is not a single pp-token.
	* cpplex.c: Correct commentary.

From-SVN: r33463
parent b4593d17
2000-04-26 Neil Booth <NeilB@earthling.net>
* cpplib.h: "~=" is not a single pp-token.
* cpplex.c: Correct commentary.
2000-04-26 Richard Henderson <rth@cygnus.com> 2000-04-26 Richard Henderson <rth@cygnus.com>
* flow.c (mark_set_1): New arguments code and flags; update all * flow.c (mark_set_1): New arguments code and flags; update all
......
...@@ -2056,24 +2056,24 @@ _cpp_init_input_buffer (pfile) ...@@ -2056,24 +2056,24 @@ _cpp_init_input_buffer (pfile)
that replaced trigraphs and deleted esacped newlines, and a second that replaced trigraphs and deleted esacped newlines, and a second
pass that tokenized the result of the first pass. Tokenisation was pass that tokenized the result of the first pass. Tokenisation was
performed by peeking at the next character in the input stream. For performed by peeking at the next character in the input stream. For
example, if the input stream contained "~=", the handler for the ~ example, if the input stream contained "!=", the handler for the !
character would peek at the next character, and if it were a '=' character would peek at the next character, and if it were a '='
would skip over it, and return a "~=" token, otherwise it would would skip over it, and return a "!=" token, otherwise it would
return just the "~" token. return just the "!" token.
To implement a single-pass lexer, this peeking ahead is unworkable. To implement a single-pass lexer, this peeking ahead is unworkable.
An arbitrary number of escaped newlines, and trigraphs (in particular An arbitrary number of escaped newlines, and trigraphs (in particular
??/ which translates to the escape \), could separate the '~' and '=' ??/ which translates to the escape \), could separate the '!' and '='
in the input stream, yet the next token is still a "~=". in the input stream, yet the next token is still a "!=".
Suppose instead that we lex by one logical line at a time, producing Suppose instead that we lex by one logical line at a time, producing
a token list or stack for each logical line, and when seeing the '~' a token list or stack for each logical line, and when seeing the '!'
push a CPP_COMPLEMENT token on the list. Then if the '~' is part of push a CPP_NOT token on the list. Then if the '!' is part of a
a longer token ("~=") we know we must see the remainder of the token longer token ("!=") we know we must see the remainder of the token by
by the time we reach the end of the logical line. Thus we can have the time we reach the end of the logical line. Thus we can have the
the '=' handler look at the previous token (at the end of the list / '=' handler look at the previous token (at the end of the list / top
top of the stack) and see if it is a "~" token, and if so, instead of of the stack) and see if it is a "!" token, and if so, instead of
pushing a "=" token revise the existing token to be a "~=" token. pushing a "=" token revise the existing token to be a "!=" token.
This works in the presence of escaped newlines, because the '\' would This works in the presence of escaped newlines, because the '\' would
have been pushed on the top of the stack as a CPP_BACKSLASH. The have been pushed on the top of the stack as a CPP_BACKSLASH. The
...@@ -2091,7 +2091,7 @@ _cpp_init_input_buffer (pfile) ...@@ -2091,7 +2091,7 @@ _cpp_init_input_buffer (pfile)
To the preprocessor, whitespace is only significant to the point of To the preprocessor, whitespace is only significant to the point of
knowing whether whitespace precedes a particular token. For example, knowing whether whitespace precedes a particular token. For example,
the '=' handler needs to know whether there was whitespace between it the '=' handler needs to know whether there was whitespace between it
and a "~" token on the top of the stack, to make the token conversion and a "!" token on the top of the stack, to make the token conversion
decision correctly. So each token has a PREV_WHITESPACE flag to decision correctly. So each token has a PREV_WHITESPACE flag to
indicate this - the standard permits consecutive whitespace to be indicate this - the standard permits consecutive whitespace to be
regarded as a single space. The compiler front ends are not regarded as a single space. The compiler front ends are not
......
...@@ -61,10 +61,10 @@ typedef struct cpp_name cpp_name; ...@@ -61,10 +61,10 @@ typedef struct cpp_name cpp_name;
T(CPP_AND, "&") /* bit ops */ \ T(CPP_AND, "&") /* bit ops */ \
T(CPP_OR, "|") \ T(CPP_OR, "|") \
T(CPP_XOR, "^") \ T(CPP_XOR, "^") \
T(CPP_COMPL, "~") \
T(CPP_RSHIFT, ">>") \ T(CPP_RSHIFT, ">>") \
T(CPP_LSHIFT, "<<") \ T(CPP_LSHIFT, "<<") \
\ \
T(CPP_COMPL, "~") \
T(CPP_AND_AND, "&&") /* logical */ \ T(CPP_AND_AND, "&&") /* logical */ \
T(CPP_OR_OR, "||") \ T(CPP_OR_OR, "||") \
T(CPP_QUERY, "?") \ T(CPP_QUERY, "?") \
...@@ -85,7 +85,6 @@ typedef struct cpp_name cpp_name; ...@@ -85,7 +85,6 @@ typedef struct cpp_name cpp_name;
T(CPP_AND_EQ, "&=") /* bit ops */ \ T(CPP_AND_EQ, "&=") /* bit ops */ \
T(CPP_OR_EQ, "|=") \ T(CPP_OR_EQ, "|=") \
T(CPP_XOR_EQ, "^=") \ T(CPP_XOR_EQ, "^=") \
T(CPP_COMPL_EQ, "~=") \
T(CPP_RSHIFT_EQ, ">>=") \ T(CPP_RSHIFT_EQ, ">>=") \
T(CPP_LSHIFT_EQ, "<<=") \ T(CPP_LSHIFT_EQ, "<<=") \
/* Digraphs together, beginning with CPP_FIRST_DIGRAPH. */ \ /* Digraphs together, beginning with CPP_FIRST_DIGRAPH. */ \
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment