Commit 26aea073 by Neil Booth Committed by Neil Booth

cppfiles.c (ENABLE_VALGRIND_CHECKING, [...]): Remove.

	* cppfiles.c (ENABLE_VALGRIND_CHECKING, VALGRIND_DISCARD,
	MMAP_THRESHOLD, TEST_THRESHOLD, SHOULD_MMAP): Remove.
	(struct include_file): Remove fefcnt, mapped members.
	(open_file, stack_include_file, _cpp_pop_file_buffer): Disable caching.
	(read_include_file): Don't use mmap, terminate buffers in '\r'.
	(purge_cache): Don't use munmap.
	* cpphash.h (CPP_BUF_COLUMN): Update.
	(lexer_state): Remove lexing_comment.
	(struct _cpp_line_note): New.
	(struct cpp_buffer): New members cur_note, notes_used, notes_cap,
	next_line and need_line.  Remove col_adjust and saved_flags.
	(_cpp_process_line_notes, _cpp_clean_line, _cpp_get_fresh_line,
	_cpp_skip_block_comment, scan_out_logical_line): New.
	(_cpp_init_mbchar): Remove.
	* cppinit.c (init_library): Remove call to _cpp_init_mbchar.
	(cpp_read_main_file): Set line to 1 earlier.
	(post_options): -traditional-cpp doesn't want trigraphs.
	* cpplex.c (MULTIBYTE_CHARS): Remove code predicated on this.
	(add_line_note, _cpp_clean_line, _cpp_process_line_notes,
	_cpp_get_fresh_line): New.
	(handle_newline, skip_escaped_newlines, trigraph_p,
	continue_after_nul, _cpp_init_mbchar): Remove.
	(get_effective_char): Update.
	(_cpp_skip_block_comment): Rename from skip_block_comment, simplify.
	(skip_line_comment): Simplify.
	(skip_whitespace, parse_identifier, parse_slow, parse_number,
	parse_string): Update.
	(cpp_lex_direct): Use clean lines and process line notes.  Update.
	(cpp_interpret_charconst): No MULTIBYTE_CHARS.
	* cpplib.c (prepare_directive_trad): Call scan_out_logical_line
	directly.
	(_cpp_handle_directive): Don't set saved_flags.
	(run_directive, destringize_and_run, cpp_define, cpp_define_builtin,
	cpp_undef, handle_assertion, cpp_push_buffer): Update.
	(_cpp_pop_buffer): Free notes.
	* cppmacro.c (builtin_macro, paste_tokens): \n terminate buffer.
	* cpppch.c (cpp_read_state): \n terminate buffer.
	* cpptrad.c (skip_escaped_newlines, handle_newline): Remove.
	(copy_comment): Use _cpp_skip_block_comment.
	(skip_whitespace, lex_identifier, _cpp_read_logical_line_trad):
	Simplify.
	(_cpp_overlay_buffer, _cpp_remove_overlay, push_replacement_text,
	save_replacement_text): Update.
	(scan_out_logical_line): Update to use clean lines and process
	line notes.
	* fix-header.c (read_scan_file): Update.
testsuite:
	* gcc.dg/cpp/_Pragma4.c: Remove stray space.
	* gcc.dg/cpp/trad/escaped-eof.c: Correct line number.

From-SVN: r65808
parent 606c9991
2003-04-19 Neil Booth <neil@daikokuya.co.uk>
* cppfiles.c (ENABLE_VALGRIND_CHECKING, VALGRIND_DISCARD,
MMAP_THRESHOLD, TEST_THRESHOLD, SHOULD_MMAP): Remove.
(struct include_file): Remove fefcnt, mapped members.
(open_file, stack_include_file, _cpp_pop_file_buffer): Disable caching.
(read_include_file): Don't use mmap, terminate buffers in '\r'.
(purge_cache): Don't use munmap.
* cpphash.h (CPP_BUF_COLUMN): Update.
(lexer_state): Remove lexing_comment.
(struct _cpp_line_note): New.
(struct cpp_buffer): New members cur_note, notes_used, notes_cap,
next_line and need_line. Remove col_adjust and saved_flags.
(_cpp_process_line_notes, _cpp_clean_line, _cpp_get_fresh_line,
_cpp_skip_block_comment, scan_out_logical_line): New.
(_cpp_init_mbchar): Remove.
* cppinit.c (init_library): Remove call to _cpp_init_mbchar.
(cpp_read_main_file): Set line to 1 earlier.
(post_options): -traditional-cpp doesn't want trigraphs.
* cpplex.c (MULTIBYTE_CHARS): Remove code predicated on this.
(add_line_note, _cpp_clean_line, _cpp_process_line_notes,
_cpp_get_fresh_line): New.
(handle_newline, skip_escaped_newlines, trigraph_p,
continue_after_nul, _cpp_init_mbchar): Remove.
(get_effective_char): Update.
(_cpp_skip_block_comment): Rename from skip_block_comment, simplify.
(skip_line_comment): Simplify.
(skip_whitespace, parse_identifier, parse_slow, parse_number,
parse_string): Update.
(cpp_lex_direct): Use clean lines and process line notes. Update.
(cpp_interpret_charconst): No MULTIBYTE_CHARS.
* cpplib.c (prepare_directive_trad): Call scan_out_logical_line
directly.
(_cpp_handle_directive): Don't set saved_flags.
(run_directive, destringize_and_run, cpp_define, cpp_define_builtin,
cpp_undef, handle_assertion, cpp_push_buffer): Update.
(_cpp_pop_buffer): Free notes.
* cppmacro.c (builtin_macro, paste_tokens): \n terminate buffer.
* cpppch.c (cpp_read_state): \n terminate buffer.
* cpptrad.c (skip_escaped_newlines, handle_newline): Remove.
(copy_comment): Use _cpp_skip_block_comment.
(skip_whitespace, lex_identifier, _cpp_read_logical_line_trad):
Simplify.
(_cpp_overlay_buffer, _cpp_remove_overlay, push_replacement_text,
save_replacement_text): Update.
(scan_out_logical_line): Update to use clean lines and process
line notes.
* fix-header.c (read_scan_file): Update.
2003-04-18 Douglas B Rupp <rupp@gnat.com> 2003-04-18 Douglas B Rupp <rupp@gnat.com>
* unwind-dw2-fde.c (__register_frame_info_bases): Check for * unwind-dw2-fde.c (__register_frame_info_bases): Check for
......
...@@ -30,47 +30,6 @@ Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. */ ...@@ -30,47 +30,6 @@ Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. */
#include "intl.h" #include "intl.h"
#include "mkdeps.h" #include "mkdeps.h"
#include "splay-tree.h" #include "splay-tree.h"
#ifdef ENABLE_VALGRIND_CHECKING
# ifdef HAVE_MEMCHECK_H
# include <memcheck.h>
# else
# include <valgrind.h>
# endif
#else
/* Avoid #ifdef:s when we can help it. */
#define VALGRIND_DISCARD(x)
#endif
#ifdef HAVE_MMAP_FILE
# include <sys/mman.h>
# ifndef MMAP_THRESHOLD
# define MMAP_THRESHOLD 3 /* Minimum page count to mmap the file. */
# endif
# if MMAP_THRESHOLD
# define TEST_THRESHOLD(size, pagesize) \
(size / pagesize >= MMAP_THRESHOLD && (size % pagesize) != 0)
/* Use mmap if the file is big enough to be worth it (controlled
by MMAP_THRESHOLD) and if we can safely count on there being
at least one readable NUL byte after the end of the file's
contents. This is true for all tested operating systems when
the file size is not an exact multiple of the page size. */
# ifndef __CYGWIN__
# define SHOULD_MMAP(size, pagesize) TEST_THRESHOLD (size, pagesize)
# else
# define WIN32_LEAN_AND_MEAN
# include <windows.h>
/* Cygwin can't correctly emulate mmap under Windows 9x style systems so
disallow use of mmap on those systems. Windows 9x does not zero fill
memory at EOF and beyond, as required. */
# define SHOULD_MMAP(size, pagesize) ((GetVersion() & 0x80000000) \
? 0 : TEST_THRESHOLD (size, pagesize))
# endif
# endif
#else /* No MMAP_FILE */
# undef MMAP_THRESHOLD
# define MMAP_THRESHOLD 0
#endif
#ifndef O_BINARY #ifndef O_BINARY
# define O_BINARY 0 # define O_BINARY 0
...@@ -102,8 +61,6 @@ struct include_file { ...@@ -102,8 +61,6 @@ struct include_file {
int fd; /* fd open on file (short term storage only) */ int fd; /* fd open on file (short term storage only) */
int err_no; /* errno obtained if opening a file failed */ int err_no; /* errno obtained if opening a file failed */
unsigned short include_count; /* number of times file has been read */ unsigned short include_count; /* number of times file has been read */
unsigned short refcnt; /* number of stacked buffers using this file */
unsigned char mapped; /* file buffer is mmapped */
unsigned char pch; /* 0: file not known to be a PCH. unsigned char pch; /* 0: file not known to be a PCH.
1: file is a PCH 1: file is a PCH
(on return from find_include_file). (on return from find_include_file).
...@@ -278,7 +235,7 @@ open_file (pfile, filename) ...@@ -278,7 +235,7 @@ open_file (pfile, filename)
return file; return file;
/* Don't reopen one which is already loaded. */ /* Don't reopen one which is already loaded. */
if (file->buffer != NULL) if (0 && file->buffer != NULL)
return file; return file;
/* We used to open files in nonblocking mode, but that caused more /* We used to open files in nonblocking mode, but that caused more
...@@ -446,7 +403,7 @@ stack_include_file (pfile, inc) ...@@ -446,7 +403,7 @@ stack_include_file (pfile, inc)
} }
/* Not in cache? */ /* Not in cache? */
if (! inc->buffer) if (1 || ! inc->buffer)
{ {
if (read_include_file (pfile, inc)) if (read_include_file (pfile, inc))
{ {
...@@ -471,7 +428,6 @@ stack_include_file (pfile, inc) ...@@ -471,7 +428,6 @@ stack_include_file (pfile, inc)
fp = cpp_push_buffer (pfile, inc->buffer, inc->st.st_size, fp = cpp_push_buffer (pfile, inc->buffer, inc->st.st_size,
/* from_stage3 */ CPP_OPTION (pfile, preprocessed), 0); /* from_stage3 */ CPP_OPTION (pfile, preprocessed), 0);
fp->inc = inc; fp->inc = inc;
fp->inc->refcnt++;
/* Initialize controlling macro state. */ /* Initialize controlling macro state. */
pfile->mi_valid = true; pfile->mi_valid = true;
...@@ -507,9 +463,6 @@ read_include_file (pfile, inc) ...@@ -507,9 +463,6 @@ read_include_file (pfile, inc)
{ {
ssize_t size, offset, count; ssize_t size, offset, count;
uchar *buf; uchar *buf;
#if MMAP_THRESHOLD
static int pagesize = -1;
#endif
if (S_ISREG (inc->st.st_mode)) if (S_ISREG (inc->st.st_mode))
{ {
...@@ -528,25 +481,6 @@ read_include_file (pfile, inc) ...@@ -528,25 +481,6 @@ read_include_file (pfile, inc)
} }
size = inc->st.st_size; size = inc->st.st_size;
inc->mapped = 0;
#if MMAP_THRESHOLD
if (pagesize == -1)
pagesize = getpagesize ();
if (SHOULD_MMAP (size, pagesize))
{
buf = (uchar *) mmap (0, size, PROT_READ, MAP_PRIVATE, inc->fd, 0);
if (buf == (uchar *) -1)
goto perror_fail;
/* We must tell Valgrind that the byte at buf[size] is actually
readable. Discard the handle to avoid handle leak. */
VALGRIND_DISCARD (VALGRIND_MAKE_READABLE (buf + size, 1));
inc->mapped = 1;
}
else
#endif
{ {
buf = (uchar *) xmalloc (size + 1); buf = (uchar *) xmalloc (size + 1);
offset = 0; offset = 0;
...@@ -567,8 +501,8 @@ read_include_file (pfile, inc) ...@@ -567,8 +501,8 @@ read_include_file (pfile, inc)
} }
offset += count; offset += count;
} }
/* The lexer requires that the buffer be NUL-terminated. */ /* The lexer requires that the buffer be \n-terminated. */
buf[size] = '\0'; buf[size] = '\n';
} }
} }
else if (S_ISBLK (inc->st.st_mode)) else if (S_ISBLK (inc->st.st_mode))
...@@ -600,8 +534,8 @@ read_include_file (pfile, inc) ...@@ -600,8 +534,8 @@ read_include_file (pfile, inc)
if (offset + 1 < size) if (offset + 1 < size)
buf = xrealloc (buf, offset + 1); buf = xrealloc (buf, offset + 1);
/* The lexer requires that the buffer be NUL-terminated. */ /* The lexer requires that the buffer be \n-terminated. */
buf[offset] = '\0'; buf[offset] = '\n';
inc->st.st_size = offset; inc->st.st_size = offset;
} }
...@@ -614,26 +548,14 @@ read_include_file (pfile, inc) ...@@ -614,26 +548,14 @@ read_include_file (pfile, inc)
return 1; return 1;
} }
/* Drop INC's buffer from memory, if we are unlikely to need it again. */ /* Drop INC's buffer from memory. */
static void static void
purge_cache (inc) purge_cache (inc)
struct include_file *inc; struct include_file *inc;
{ {
if (inc->buffer) if (inc->buffer)
{ {
#if MMAP_THRESHOLD free ((PTR) inc->buffer);
if (inc->mapped)
{
/* Undo the previous annotation for the
known-zero-byte-after-mmap. Discard the handle to avoid
handle leak. */
VALGRIND_DISCARD (VALGRIND_MAKE_NOACCESS (inc->buffer
+ inc->st.st_size, 1));
munmap ((PTR) inc->buffer, inc->st.st_size);
}
else
#endif
free ((PTR) inc->buffer);
inc->buffer = NULL; inc->buffer = NULL;
} }
} }
...@@ -929,9 +851,7 @@ _cpp_pop_file_buffer (pfile, inc) ...@@ -929,9 +851,7 @@ _cpp_pop_file_buffer (pfile, inc)
/* Invalidate control macros in the #including file. */ /* Invalidate control macros in the #including file. */
pfile->mi_valid = false; pfile->mi_valid = false;
inc->refcnt--; purge_cache (inc);
if (inc->refcnt == 0 && DO_NOT_REREAD (inc))
purge_cache (inc);
} }
/* Returns the first place in the include chain to start searching for /* Returns the first place in the include chain to start searching for
......
...@@ -45,7 +45,7 @@ typedef unsigned char uchar; ...@@ -45,7 +45,7 @@ typedef unsigned char uchar;
#define CPP_OPTION(PFILE, OPTION) ((PFILE)->opts.OPTION) #define CPP_OPTION(PFILE, OPTION) ((PFILE)->opts.OPTION)
#define CPP_BUFFER(PFILE) ((PFILE)->buffer) #define CPP_BUFFER(PFILE) ((PFILE)->buffer)
#define CPP_BUF_COLUMN(BUF, CUR) ((CUR) - (BUF)->line_base + (BUF)->col_adjust) #define CPP_BUF_COLUMN(BUF, CUR) ((CUR) - (BUF)->line_base)
#define CPP_BUF_COL(BUF) CPP_BUF_COLUMN(BUF, (BUF)->cur) #define CPP_BUF_COL(BUF) CPP_BUF_COLUMN(BUF, (BUF)->cur)
/* Maximum nesting of cpp_buffers. We use a static limit, partly for /* Maximum nesting of cpp_buffers. We use a static limit, partly for
...@@ -212,9 +212,6 @@ struct lexer_state ...@@ -212,9 +212,6 @@ struct lexer_state
all directives apart from #define. */ all directives apart from #define. */
unsigned char save_comments; unsigned char save_comments;
/* Nonzero if we're mid-comment. */
unsigned char lexing_comment;
/* Nonzero if lexing __VA_ARGS__ is valid. */ /* Nonzero if lexing __VA_ARGS__ is valid. */
unsigned char va_args_ok; unsigned char va_args_ok;
...@@ -240,17 +237,37 @@ struct spec_nodes ...@@ -240,17 +237,37 @@ struct spec_nodes
cpp_hashnode *n__VA_ARGS__; /* C99 vararg macros */ cpp_hashnode *n__VA_ARGS__; /* C99 vararg macros */
}; };
typedef struct _cpp_line_note _cpp_line_note;
struct _cpp_line_note
{
/* Location in the clean line the note refers to. */
const uchar *pos;
/* Type of note. */
enum { NOTE_ESC_NL = 0,
NOTE_ESC_SPACE_NL,
NOTE_TRIGRAPH,
NOTE_NEWLINE } type;
};
/* Represents the contents of a file cpplib has read in. */ /* Represents the contents of a file cpplib has read in. */
struct cpp_buffer struct cpp_buffer
{ {
const unsigned char *cur; /* current position */ const uchar *cur; /* Current location. */
const unsigned char *backup_to; /* if peeked character is not wanted */ const uchar *line_base; /* Start of current physical line. */
const unsigned char *rlimit; /* end of valid data */ const uchar *next_line; /* Start of to-be-cleaned logical line. */
const unsigned char *line_base; /* start of current line */
const uchar *buf; /* Entire character buffer. */
const uchar *rlimit; /* Writable byte at end of file. */
_cpp_line_note *notes; /* Array of notes. */
unsigned int cur_note; /* Next note to process. */
unsigned int notes_used; /* Number of notes. */
unsigned int notes_cap; /* Size of allocated array. */
struct cpp_buffer *prev; struct cpp_buffer *prev;
const unsigned char *buf; /* Entire character buffer. */ const unsigned char *backup_to; /* Soon to die. */
/* Pointer into the include table; non-NULL if this is a file /* Pointer into the include table; non-NULL if this is a file
buffer. Used for include_next and to record control macros. */ buffer. Used for include_next and to record control macros. */
...@@ -260,15 +277,8 @@ struct cpp_buffer ...@@ -260,15 +277,8 @@ struct cpp_buffer
Used to prohibit unmatched #endif (etc) in an include file. */ Used to prohibit unmatched #endif (etc) in an include file. */
struct if_stack *if_stack; struct if_stack *if_stack;
/* Token column position adjustment owing to tabs in whitespace. */ /* True if we need to get the next clean line. */
unsigned int col_adjust; bool need_line;
/* Contains PREV_WHITE and/or AVOID_LPASTE. */
unsigned char saved_flags;
/* Because of the way the lexer works, -Wtrigraphs can sometimes
warn twice for the same trigraph. This helps prevent that. */
const unsigned char *last_Wtrigraphs;
/* True if we have already warned about C++ comments in this file. /* True if we have already warned about C++ comments in this file.
The warning happens only for C89 extended mode with -pedantic on, The warning happens only for C89 extended mode with -pedantic on,
...@@ -503,13 +513,16 @@ extern bool _cpp_parse_expr PARAMS ((cpp_reader *)); ...@@ -503,13 +513,16 @@ extern bool _cpp_parse_expr PARAMS ((cpp_reader *));
extern struct op *_cpp_expand_op_stack PARAMS ((cpp_reader *)); extern struct op *_cpp_expand_op_stack PARAMS ((cpp_reader *));
/* In cpplex.c */ /* In cpplex.c */
extern void _cpp_process_line_notes PARAMS ((cpp_reader *, int));
extern void _cpp_clean_line PARAMS ((cpp_reader *));
extern bool _cpp_get_fresh_line PARAMS ((cpp_reader *));
extern bool _cpp_skip_block_comment PARAMS ((cpp_reader *));
extern cpp_token *_cpp_temp_token PARAMS ((cpp_reader *)); extern cpp_token *_cpp_temp_token PARAMS ((cpp_reader *));
extern const cpp_token *_cpp_lex_token PARAMS ((cpp_reader *)); extern const cpp_token *_cpp_lex_token PARAMS ((cpp_reader *));
extern cpp_token *_cpp_lex_direct PARAMS ((cpp_reader *)); extern cpp_token *_cpp_lex_direct PARAMS ((cpp_reader *));
extern int _cpp_equiv_tokens PARAMS ((const cpp_token *, extern int _cpp_equiv_tokens PARAMS ((const cpp_token *,
const cpp_token *)); const cpp_token *));
extern void _cpp_init_tokenrun PARAMS ((tokenrun *, unsigned int)); extern void _cpp_init_tokenrun PARAMS ((tokenrun *, unsigned int));
extern void _cpp_init_mbchar PARAMS ((void));
/* In cppinit.c. */ /* In cppinit.c. */
extern void _cpp_maybe_push_include_file PARAMS ((cpp_reader *)); extern void _cpp_maybe_push_include_file PARAMS ((cpp_reader *));
...@@ -529,6 +542,7 @@ extern void _cpp_do_file_change PARAMS ((cpp_reader *, enum lc_reason, ...@@ -529,6 +542,7 @@ extern void _cpp_do_file_change PARAMS ((cpp_reader *, enum lc_reason,
extern void _cpp_pop_buffer PARAMS ((cpp_reader *)); extern void _cpp_pop_buffer PARAMS ((cpp_reader *));
/* In cpptrad.c. */ /* In cpptrad.c. */
extern bool scan_out_logical_line PARAMS ((cpp_reader *, cpp_macro *));
extern bool _cpp_read_logical_line_trad PARAMS ((cpp_reader *)); extern bool _cpp_read_logical_line_trad PARAMS ((cpp_reader *));
extern void _cpp_overlay_buffer PARAMS ((cpp_reader *pfile, const uchar *, extern void _cpp_overlay_buffer PARAMS ((cpp_reader *pfile, const uchar *,
size_t)); size_t));
......
...@@ -125,8 +125,6 @@ init_library () ...@@ -125,8 +125,6 @@ init_library ()
we were compiled with a compiler that supports C99 designated we were compiled with a compiler that supports C99 designated
initializers. */ initializers. */
init_trigraph_map (); init_trigraph_map ();
_cpp_init_mbchar ();
} }
} }
...@@ -167,7 +165,6 @@ cpp_create_reader (lang, table) ...@@ -167,7 +165,6 @@ cpp_create_reader (lang, table)
/* Initialize the line map. Start at logical line 1, so we can use /* Initialize the line map. Start at logical line 1, so we can use
a line number of zero for special states. */ a line number of zero for special states. */
init_line_maps (&pfile->line_maps); init_line_maps (&pfile->line_maps);
pfile->line = 1;
/* Initialize lexer state. */ /* Initialize lexer state. */
pfile->state.save_comments = ! CPP_OPTION (pfile, discard_comments); pfile->state.save_comments = ! CPP_OPTION (pfile, discard_comments);
...@@ -454,6 +451,7 @@ cpp_read_main_file (pfile, fname) ...@@ -454,6 +451,7 @@ cpp_read_main_file (pfile, fname)
} }
/* Open the main input file. */ /* Open the main input file. */
pfile->line = 1;
if (!_cpp_read_file (pfile, fname)) if (!_cpp_read_file (pfile, fname))
return NULL; return NULL;
...@@ -556,7 +554,11 @@ post_options (pfile) ...@@ -556,7 +554,11 @@ post_options (pfile)
CPP_OPTION (pfile, traditional) = 0; CPP_OPTION (pfile, traditional) = 0;
} }
/* Traditional CPP does not accurately track column information. */
if (CPP_OPTION (pfile, traditional)) if (CPP_OPTION (pfile, traditional))
CPP_OPTION (pfile, show_column) = 0; {
/* Traditional CPP does not accurately track column information. */
CPP_OPTION (pfile, show_column) = 0;
CPP_OPTION (pfile, trigraphs) = 0;
CPP_OPTION (pfile, warn_trigraphs) = 0;
}
} }
...@@ -4,7 +4,6 @@ ...@@ -4,7 +4,6 @@
Based on CCCP program by Paul Rubin, June 1986 Based on CCCP program by Paul Rubin, June 1986
Adapted to ANSI C, Richard Stallman, Jan 1987 Adapted to ANSI C, Richard Stallman, Jan 1987
Broken out to separate file, Zack Weinberg, Mar 2000 Broken out to separate file, Zack Weinberg, Mar 2000
Single-pass line tokenization by Neil Booth, April 2000
This program is free software; you can redistribute it and/or modify it This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the under the terms of the GNU General Public License as published by the
...@@ -27,11 +26,6 @@ Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. */ ...@@ -27,11 +26,6 @@ Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. */
#include "cpplib.h" #include "cpplib.h"
#include "cpphash.h" #include "cpphash.h"
#ifdef MULTIBYTE_CHARS
#include "mbchar.h"
#include <locale.h>
#endif
/* Tokens with SPELL_STRING store their spelling in the token list, /* Tokens with SPELL_STRING store their spelling in the token list,
and it's length in the token->val.name.len. */ and it's length in the token->val.name.len. */
enum spell_type enum spell_type
...@@ -63,24 +57,19 @@ static const struct token_spelling token_spellings[N_TTYPES] = { TTYPE_TABLE }; ...@@ -63,24 +57,19 @@ static const struct token_spelling token_spellings[N_TTYPES] = { TTYPE_TABLE };
#define TOKEN_NAME(token) (token_spellings[(token)->type].name) #define TOKEN_NAME(token) (token_spellings[(token)->type].name)
#define BACKUP() do {buffer->cur = buffer->backup_to;} while (0) #define BACKUP() do {buffer->cur = buffer->backup_to;} while (0)
static void handle_newline PARAMS ((cpp_reader *)); static void add_line_note PARAMS ((cpp_buffer *, const uchar *, unsigned int));
static cppchar_t skip_escaped_newlines PARAMS ((cpp_reader *));
static cppchar_t get_effective_char PARAMS ((cpp_reader *)); static cppchar_t get_effective_char PARAMS ((cpp_reader *));
static int skip_block_comment PARAMS ((cpp_reader *));
static int skip_line_comment PARAMS ((cpp_reader *)); static int skip_line_comment PARAMS ((cpp_reader *));
static void adjust_column PARAMS ((cpp_reader *)); static void skip_whitespace PARAMS ((cpp_reader *, cppchar_t));
static int skip_whitespace PARAMS ((cpp_reader *, cppchar_t));
static cpp_hashnode *parse_identifier PARAMS ((cpp_reader *)); static cpp_hashnode *parse_identifier PARAMS ((cpp_reader *));
static uchar *parse_slow PARAMS ((cpp_reader *, const uchar *, int, static uchar *parse_slow PARAMS ((cpp_reader *, const uchar *, int,
unsigned int *)); unsigned int *));
static void parse_number PARAMS ((cpp_reader *, cpp_string *, int)); static void parse_number PARAMS ((cpp_reader *, cpp_string *, int));
static int unescaped_terminator_p PARAMS ((cpp_reader *, const uchar *)); static int unescaped_terminator_p PARAMS ((cpp_reader *, const uchar *));
static void parse_string PARAMS ((cpp_reader *, cpp_token *, cppchar_t)); static void parse_string PARAMS ((cpp_reader *, cpp_token *, cppchar_t));
static bool trigraph_p PARAMS ((cpp_reader *));
static void save_comment PARAMS ((cpp_reader *, cpp_token *, const uchar *, static void save_comment PARAMS ((cpp_reader *, cpp_token *, const uchar *,
cppchar_t)); cppchar_t));
static bool continue_after_nul PARAMS ((cpp_reader *));
static int name_p PARAMS ((cpp_reader *, const cpp_string *)); static int name_p PARAMS ((cpp_reader *, const cpp_string *));
static int maybe_read_ucs PARAMS ((cpp_reader *, const unsigned char **, static int maybe_read_ucs PARAMS ((cpp_reader *, const unsigned char **,
const unsigned char *, cppchar_t *)); const unsigned char *, cppchar_t *));
...@@ -89,15 +78,6 @@ static tokenrun *next_tokenrun PARAMS ((tokenrun *)); ...@@ -89,15 +78,6 @@ static tokenrun *next_tokenrun PARAMS ((tokenrun *));
static unsigned int hex_digit_value PARAMS ((unsigned int)); static unsigned int hex_digit_value PARAMS ((unsigned int));
static _cpp_buff *new_buff PARAMS ((size_t)); static _cpp_buff *new_buff PARAMS ((size_t));
/* Change to the native locale for multibyte conversions. */
void
_cpp_init_mbchar ()
{
#ifdef MULTIBYTE_CHARS
setlocale (LC_CTYPE, "");
GET_ENVIRONMENT (literal_codeset, "LANG");
#endif
}
/* Utility routine: /* Utility routine:
...@@ -114,132 +94,158 @@ cpp_ideq (token, string) ...@@ -114,132 +94,158 @@ cpp_ideq (token, string)
return !ustrcmp (NODE_NAME (token->val.node), (const uchar *) string); return !ustrcmp (NODE_NAME (token->val.node), (const uchar *) string);
} }
/* Call when meeting a newline, assumed to be in buffer->cur[-1]. /* Record a note TYPE at byte POS into the current cleaned logical
Returns with buffer->cur pointing to the character immediately line. */
following the newline (combination). */
static void static void
handle_newline (pfile) add_line_note (buffer, pos, type)
cpp_reader *pfile; cpp_buffer *buffer;
const uchar *pos;
unsigned int type;
{ {
cpp_buffer *buffer = pfile->buffer; if (buffer->notes_used == buffer->notes_cap)
{
/* Handle CR-LF and LF-CR. Most other implementations (e.g. java) buffer->notes_cap = buffer->notes_cap * 2 + 200;
only accept CR-LF; maybe we should fall back to that behavior? */ buffer->notes = (_cpp_line_note *)
if (buffer->cur[-1] + buffer->cur[0] == '\r' + '\n') xrealloc (buffer->notes, buffer->notes_cap * sizeof (_cpp_line_note));
buffer->cur++; }
buffer->line_base = buffer->cur; buffer->notes[buffer->notes_used].pos = pos;
buffer->col_adjust = 0; buffer->notes[buffer->notes_used].type = type;
pfile->line++; buffer->notes_used++;
} }
/* Subroutine of skip_escaped_newlines; called when a 3-character /* Returns with a logical line that contains no escaped newlines or
sequence beginning with "??" is encountered. buffer->cur points to trigraphs. This is a time-critical inner loop. */
the second '?'. void
_cpp_clean_line (pfile)
Warn if necessary, and returns true if the sequence forms a
trigraph and the trigraph should be honored. */
static bool
trigraph_p (pfile)
cpp_reader *pfile; cpp_reader *pfile;
{ {
cpp_buffer *buffer = pfile->buffer; cpp_buffer *buffer;
cppchar_t from_char = buffer->cur[1]; const uchar *s;
bool accept; uchar c, *d, *p;
if (!_cpp_trigraph_map[from_char])
return false;
accept = CPP_OPTION (pfile, trigraphs); buffer = pfile->buffer;
buffer->cur_note = buffer->notes_used = 0;
buffer->cur = buffer->line_base = buffer->next_line;
buffer->need_line = false;
s = buffer->next_line - 1;
/* Don't warn about trigraphs in comments. */ if (!buffer->from_stage3)
if (CPP_OPTION (pfile, warn_trigraphs) && !pfile->state.lexing_comment)
{ {
if (accept) d = (uchar *) s;
cpp_error_with_line (pfile, DL_WARNING,
pfile->line, CPP_BUF_COL (buffer) - 1, for (;;)
"trigraph ??%c converted to %c",
(int) from_char,
(int) _cpp_trigraph_map[from_char]);
else if (buffer->cur != buffer->last_Wtrigraphs)
{ {
buffer->last_Wtrigraphs = buffer->cur; c = *++s;
cpp_error_with_line (pfile, DL_WARNING, *++d = c;
pfile->line, CPP_BUF_COL (buffer) - 1,
"trigraph ??%c ignored", (int) from_char); if (c == '\n' || c == '\r')
{
/* Handle DOS line endings. */
if (c == '\r' && s != buffer->rlimit && s[1] == '\n')
s++;
if (s == buffer->rlimit)
break;
/* Escaped? */
p = d;
while (p != buffer->next_line && is_nvspace (p[-1]))
p--;
if (p == buffer->next_line || p[-1] != '\\')
break;
add_line_note (buffer, p - 1,
p != d ? NOTE_ESC_SPACE_NL: NOTE_ESC_NL);
d = p - 2;
buffer->next_line = p - 1;
}
else if (c == '?' && s[1] == '?' && _cpp_trigraph_map[s[2]])
{
/* Add a note regardless, for the benefit of -Wtrigraphs. */
add_line_note (buffer, d, NOTE_TRIGRAPH);
if (CPP_OPTION (pfile, trigraphs))
{
*d = _cpp_trigraph_map[s[2]];
s += 2;
}
}
} }
} }
else
{
do
s++;
while (*s != '\n' && *s != '\r');
d = (uchar *) s;
/* Handle DOS line endings. */
if (*s == '\r' && s != buffer->rlimit && s[1] == '\n')
s++;
}
return accept; *d = '\n';
add_line_note (buffer, d + 1, NOTE_NEWLINE);
buffer->next_line = s + 1;
} }
/* Skips any escaped newlines introduced by '?' or a '\\', assumed to /* Process the notes created by add_line_note as far as the current
lie in buffer->cur[-1]. Returns the next byte, which will be in location. */
buffer->cur[-1]. This routine performs preprocessing stages 1 and void
2 of the ISO C standard. */ _cpp_process_line_notes (pfile, in_comment)
static cppchar_t
skip_escaped_newlines (pfile)
cpp_reader *pfile; cpp_reader *pfile;
int in_comment;
{ {
cpp_buffer *buffer = pfile->buffer; cpp_buffer *buffer = pfile->buffer;
cppchar_t next = buffer->cur[-1];
/* Only do this if we apply stages 1 and 2. */ for (;;)
if (!buffer->from_stage3)
{ {
const unsigned char *saved_cur; _cpp_line_note *note = &buffer->notes[buffer->cur_note];
cppchar_t next1; unsigned int col;
do if (note->pos > buffer->cur)
{ break;
if (next == '?')
{
if (buffer->cur[0] != '?' || !trigraph_p (pfile))
break;
/* Translate the trigraph. */
next = _cpp_trigraph_map[buffer->cur[1]];
buffer->cur += 2;
if (next != '\\')
break;
}
if (buffer->cur == buffer->rlimit) buffer->cur_note++;
break; col = CPP_BUF_COLUMN (buffer, note->pos + 1);
/* We have a backslash, and room for at least one more switch (note->type)
character. Skip horizontal whitespace. */ {
saved_cur = buffer->cur; case NOTE_NEWLINE:
do /* This note is a kind of sentinel we should never reach. */
next1 = *buffer->cur++; abort ();
while (is_nvspace (next1) && buffer->cur < buffer->rlimit);
if (!is_vspace (next1)) case NOTE_TRIGRAPH:
if (!in_comment && CPP_OPTION (pfile, warn_trigraphs))
{ {
buffer->cur = saved_cur; if (CPP_OPTION (pfile, trigraphs))
break; cpp_error_with_line (pfile, DL_WARNING, pfile->line, col,
"trigraph converted to %c",
(int) note->pos[0]);
else
cpp_error_with_line (pfile, DL_WARNING, pfile->line, col,
"trigraph ??%c ignored",
(int) note->pos[2]);
} }
break;
if (saved_cur != buffer->cur - 1 case NOTE_ESC_SPACE_NL:
&& !pfile->state.lexing_comment) if (!in_comment)
cpp_error (pfile, DL_WARNING, cpp_error_with_line (pfile, DL_WARNING, pfile->line, col,
"backslash and newline separated by space"); "backslash and newline separated by space");
/* Fall through... */
handle_newline (pfile); case NOTE_ESC_NL:
buffer->backup_to = buffer->cur; if (buffer->next_line > buffer->rlimit)
if (buffer->cur == buffer->rlimit)
{ {
cpp_error (pfile, DL_PEDWARN, cpp_error_with_line (pfile, DL_PEDWARN, pfile->line, col,
"backslash-newline at end of file"); "backslash-newline at end of file");
next = EOF; /* Prevent "no newline at end of file" warning. */
buffer->next_line = buffer->rlimit;
} }
else
next = *buffer->cur++; buffer->line_base = note->pos;
pfile->line++;
} }
while (next == '\\' || next == '?');
} }
return next;
} }
/* Obtain the next character, after trigraph conversion and skipping /* Obtain the next character, after trigraph conversion and skipping
...@@ -251,42 +257,34 @@ static cppchar_t ...@@ -251,42 +257,34 @@ static cppchar_t
get_effective_char (pfile) get_effective_char (pfile)
cpp_reader *pfile; cpp_reader *pfile;
{ {
cppchar_t next;
cpp_buffer *buffer = pfile->buffer; cpp_buffer *buffer = pfile->buffer;
buffer->backup_to = buffer->cur; buffer->backup_to = buffer->cur;
next = *buffer->cur++; return *buffer->cur++;
if (__builtin_expect (next == '?' || next == '\\', 0))
next = skip_escaped_newlines (pfile);
return next;
} }
/* Skip a C-style block comment. We find the end of the comment by /* Skip a C-style block comment. We find the end of the comment by
seeing if an asterisk is before every '/' we encounter. Returns seeing if an asterisk is before every '/' we encounter. Returns
nonzero if comment terminated by EOF, zero otherwise. */ nonzero if comment terminated by EOF, zero otherwise. */
static int bool
skip_block_comment (pfile) _cpp_skip_block_comment (pfile)
cpp_reader *pfile; cpp_reader *pfile;
{ {
cpp_buffer *buffer = pfile->buffer; cpp_buffer *buffer = pfile->buffer;
cppchar_t c = EOF, prevc = EOF; cppchar_t c;
pfile->state.lexing_comment = 1; if (*buffer->cur == '/')
while (buffer->cur != buffer->rlimit) buffer->cur++;
{
prevc = c, c = *buffer->cur++;
/* FIXME: For speed, create a new character class of characters for (;;)
of interest inside block comments. */ {
if (c == '?' || c == '\\') c = *buffer->cur++;
c = skip_escaped_newlines (pfile);
/* People like decorating comments with '*', so check for '/' /* People like decorating comments with '*', so check for '/'
instead for efficiency. */ instead for efficiency. */
if (c == '/') if (c == '/')
{ {
if (prevc == '*') if (buffer->cur[-2] == '*')
break; break;
/* Warn about potential nested comments, but not if the '/' /* Warn about potential nested comments, but not if the '/'
...@@ -298,14 +296,18 @@ skip_block_comment (pfile) ...@@ -298,14 +296,18 @@ skip_block_comment (pfile)
pfile->line, CPP_BUF_COL (buffer), pfile->line, CPP_BUF_COL (buffer),
"\"/*\" within comment"); "\"/*\" within comment");
} }
else if (is_vspace (c)) else if (c == '\n')
handle_newline (pfile); {
else if (c == '\t') buffer->cur--;
adjust_column (pfile); _cpp_process_line_notes (pfile, true);
if (buffer->next_line >= buffer->rlimit)
return true;
_cpp_clean_line (pfile);
pfile->line++;
}
} }
pfile->state.lexing_comment = 0; return false;
return c != '/' || prevc != '*';
} }
/* Skip a C++ line comment, leaving buffer->cur pointing to the /* Skip a C++ line comment, leaving buffer->cur pointing to the
...@@ -317,72 +319,16 @@ skip_line_comment (pfile) ...@@ -317,72 +319,16 @@ skip_line_comment (pfile)
{ {
cpp_buffer *buffer = pfile->buffer; cpp_buffer *buffer = pfile->buffer;
unsigned int orig_line = pfile->line; unsigned int orig_line = pfile->line;
cppchar_t c;
#ifdef MULTIBYTE_CHARS
wchar_t wc;
int char_len;
#endif
pfile->state.lexing_comment = 1;
#ifdef MULTIBYTE_CHARS
/* Reset multibyte conversion state. */
(void) local_mbtowc (NULL, NULL, 0);
#endif
do
{
if (buffer->cur == buffer->rlimit)
goto at_eof;
#ifdef MULTIBYTE_CHARS
char_len = local_mbtowc (&wc, (const char *) buffer->cur,
buffer->rlimit - buffer->cur);
if (char_len == -1)
{
cpp_error (pfile, DL_WARNING,
"ignoring invalid multibyte character");
char_len = 1;
c = *buffer->cur++;
}
else
{
buffer->cur += char_len;
c = wc;
}
#else
c = *buffer->cur++;
#endif
if (c == '?' || c == '\\')
c = skip_escaped_newlines (pfile);
}
while (!is_vspace (c));
/* Step back over the newline, except at EOF. */ while (*buffer->cur != '\n')
buffer->cur--; buffer->cur++;
at_eof:
pfile->state.lexing_comment = 0; _cpp_process_line_notes (pfile, true);
return orig_line != pfile->line; return orig_line != pfile->line;
} }
/* pfile->buffer->cur is one beyond the \t character. Update /* Skips whitespace, saving the next non-whitespace character. */
col_adjust so we track the column correctly. */
static void static void
adjust_column (pfile)
cpp_reader *pfile;
{
cpp_buffer *buffer = pfile->buffer;
unsigned int col = CPP_BUF_COL (buffer) - 1; /* Zero-based column. */
/* Round it up to multiple of the tabstop, but subtract 1 since the
tab itself occupies a character position. */
buffer->col_adjust += (CPP_OPTION (pfile, tabstop)
- col % CPP_OPTION (pfile, tabstop)) - 1;
}
/* Skips whitespace, saving the next non-whitespace character.
Adjusts pfile->col_adjust to account for tabs. Without this,
tokens might be assigned an incorrect column. */
static int
skip_whitespace (pfile, c) skip_whitespace (pfile, c)
cpp_reader *pfile; cpp_reader *pfile;
cppchar_t c; cppchar_t c;
...@@ -393,15 +339,11 @@ skip_whitespace (pfile, c) ...@@ -393,15 +339,11 @@ skip_whitespace (pfile, c)
do do
{ {
/* Horizontal space always OK. */ /* Horizontal space always OK. */
if (c == ' ') if (c == ' ' || c == '\t')
; ;
else if (c == '\t')
adjust_column (pfile);
/* Just \f \v or \0 left. */ /* Just \f \v or \0 left. */
else if (c == '\0') else if (c == '\0')
{ {
if (buffer->cur - 1 == buffer->rlimit)
return 0;
if (!warned) if (!warned)
{ {
cpp_error (pfile, DL_WARNING, "null character(s) ignored"); cpp_error (pfile, DL_WARNING, "null character(s) ignored");
...@@ -420,7 +362,6 @@ skip_whitespace (pfile, c) ...@@ -420,7 +362,6 @@ skip_whitespace (pfile, c)
while (is_nvspace (c)); while (is_nvspace (c));
buffer->cur--; buffer->cur--;
return 1;
} }
/* See if the characters of a number token are valid in a name (no /* See if the characters of a number token are valid in a name (no
...@@ -461,7 +402,7 @@ parse_identifier (pfile) ...@@ -461,7 +402,7 @@ parse_identifier (pfile)
cur++; cur++;
/* Check for slow-path cases. */ /* Check for slow-path cases. */
if (*cur == '?' || *cur == '\\' || *cur == '$') if (*cur == '$')
{ {
unsigned int len; unsigned int len;
...@@ -532,8 +473,6 @@ parse_slow (pfile, cur, number_p, plen) ...@@ -532,8 +473,6 @@ parse_slow (pfile, cur, number_p, plen)
{ {
/* Potential escaped newline? */ /* Potential escaped newline? */
buffer->backup_to = buffer->cur - 1; buffer->backup_to = buffer->cur - 1;
if (c == '?' || c == '\\')
c = skip_escaped_newlines (pfile);
if (!is_idchar (c)) if (!is_idchar (c))
{ {
...@@ -590,7 +529,7 @@ parse_number (pfile, number, leading_period) ...@@ -590,7 +529,7 @@ parse_number (pfile, number, leading_period)
cur++; cur++;
/* Check for slow-path cases. */ /* Check for slow-path cases. */
if (*cur == '?' || *cur == '\\' || *cur == '$') if (*cur == '$')
number->text = parse_slow (pfile, cur, 1 + leading_period, &number->len); number->text = parse_slow (pfile, cur, 1 + leading_period, &number->len);
else else
{ {
...@@ -648,18 +587,10 @@ parse_string (pfile, token, terminator) ...@@ -648,18 +587,10 @@ parse_string (pfile, token, terminator)
unsigned char *dest, *limit; unsigned char *dest, *limit;
cppchar_t c; cppchar_t c;
bool warned_nulls = false; bool warned_nulls = false;
#ifdef MULTIBYTE_CHARS
wchar_t wc;
int char_len;
#endif
dest = BUFF_FRONT (pfile->u_buff); dest = BUFF_FRONT (pfile->u_buff);
limit = BUFF_LIMIT (pfile->u_buff); limit = BUFF_LIMIT (pfile->u_buff);
#ifdef MULTIBYTE_CHARS
/* Reset multibyte conversion state. */
(void) local_mbtowc (NULL, NULL, 0);
#endif
for (;;) for (;;)
{ {
/* We need room for another char, possibly the terminating NUL. */ /* We need room for another char, possibly the terminating NUL. */
...@@ -671,41 +602,19 @@ parse_string (pfile, token, terminator) ...@@ -671,41 +602,19 @@ parse_string (pfile, token, terminator)
limit = BUFF_LIMIT (pfile->u_buff); limit = BUFF_LIMIT (pfile->u_buff);
} }
#ifdef MULTIBYTE_CHARS
char_len = local_mbtowc (&wc, (const char *) buffer->cur,
buffer->rlimit - buffer->cur);
if (char_len == -1)
{
cpp_error (pfile, DL_WARNING,
"ignoring invalid multibyte character");
char_len = 1;
c = *buffer->cur++;
}
else
{
buffer->cur += char_len;
c = wc;
}
#else
c = *buffer->cur++; c = *buffer->cur++;
#endif
/* Handle trigraphs, escaped newlines etc. */
if (c == '?' || c == '\\')
c = skip_escaped_newlines (pfile);
if (c == terminator) if (c == terminator)
{ {
if (unescaped_terminator_p (pfile, dest)) if (unescaped_terminator_p (pfile, dest))
break; break;
} }
else if (is_vspace (c)) else if (c == '\n')
{ {
/* No string literal may extend over multiple lines. In /* No string literal may extend over multiple lines. In
assembly language, suppress the error except for <> assembly language, suppress the error except for <>
includes. This is a kludge around not knowing where includes. This is a kludge around not knowing where
comments are. */ comments are. */
unterminated:
if (CPP_OPTION (pfile, lang) != CLK_ASM || terminator == '>') if (CPP_OPTION (pfile, lang) != CLK_ASM || terminator == '>')
cpp_error (pfile, DL_ERROR, "missing terminating %c character", cpp_error (pfile, DL_ERROR, "missing terminating %c character",
(int) terminator); (int) terminator);
...@@ -714,8 +623,6 @@ parse_string (pfile, token, terminator) ...@@ -714,8 +623,6 @@ parse_string (pfile, token, terminator)
} }
else if (c == '\0') else if (c == '\0')
{ {
if (buffer->cur - 1 == buffer->rlimit)
goto unterminated;
if (!warned_nulls) if (!warned_nulls)
{ {
warned_nulls = true; warned_nulls = true;
...@@ -723,14 +630,6 @@ parse_string (pfile, token, terminator) ...@@ -723,14 +630,6 @@ parse_string (pfile, token, terminator)
"null character(s) preserved in literal"); "null character(s) preserved in literal");
} }
} }
#ifdef MULTIBYTE_CHARS
if (char_len > 1)
{
for ( ; char_len > 0; --char_len)
*dest++ = (*buffer->cur - char_len);
}
else
#endif
*dest++ = c; *dest++ = c;
} }
...@@ -890,55 +789,55 @@ _cpp_lex_token (pfile) ...@@ -890,55 +789,55 @@ _cpp_lex_token (pfile)
return result; return result;
} }
/* A NUL terminates the current buffer. For ISO preprocessing this is /* Returns true if a fresh line has been loaded. */
EOF, but for traditional preprocessing it indicates we need a line bool
refill. Returns TRUE to continue preprocessing a new buffer, FALSE _cpp_get_fresh_line (pfile)
to return a CPP_EOF to the caller. */
static bool
continue_after_nul (pfile)
cpp_reader *pfile; cpp_reader *pfile;
{ {
cpp_buffer *buffer = pfile->buffer; /* We can't get a new line until we leave the current directive. */
bool more = false; if (pfile->state.in_directive)
return false;
buffer->saved_flags = BOL; for (;;)
if (CPP_OPTION (pfile, traditional))
{ {
if (pfile->state.in_directive) cpp_buffer *buffer = pfile->buffer;
return false;
_cpp_remove_overlay (pfile); if (!buffer->need_line)
more = _cpp_read_logical_line_trad (pfile); return true;
_cpp_overlay_buffer (pfile, pfile->out.base,
pfile->out.cur - pfile->out.base); if (buffer->next_line < buffer->rlimit)
pfile->line = pfile->out.first_line;
}
else
{
/* Stop parsing arguments with a CPP_EOF. When we finally come
back here, do the work of popping the buffer. */
if (!pfile->state.parsing_args)
{ {
if (buffer->cur != buffer->line_base) _cpp_clean_line (pfile);
{ return true;
/* Non-empty files should end in a newline. Don't warn }
for command line and _Pragma buffers. */
if (!buffer->from_stage3)
cpp_error (pfile, DL_PEDWARN, "no newline at end of file");
handle_newline (pfile);
}
/* Similarly, finish an in-progress directive with CPP_EOF /* First, get out of parsing arguments state. */
before popping the buffer. */ if (pfile->state.parsing_args)
if (!pfile->state.in_directive && buffer->prev) return false;
{
more = !buffer->return_at_eof; /* End of buffer. Non-empty files should end in a newline. */
_cpp_pop_buffer (pfile); if (buffer->buf != buffer->rlimit
} && buffer->next_line > buffer->rlimit
&& !buffer->from_stage3)
{
/* Only warn once. */
buffer->next_line = buffer->rlimit;
cpp_error_with_line (pfile, DL_PEDWARN, pfile->line - 1,
CPP_BUF_COLUMN (buffer, buffer->cur),
"no newline at end of file");
}
if (buffer->return_at_eof)
{
buffer->return_at_eof = false;
return false;
} }
}
return more; if (!buffer->prev)
return false;
_cpp_pop_buffer (pfile);
}
} }
#define IF_NEXT_IS(CHAR, THEN_TYPE, ELSE_TYPE) \ #define IF_NEXT_IS(CHAR, THEN_TYPE, ELSE_TYPE) \
...@@ -973,74 +872,49 @@ _cpp_lex_direct (pfile) ...@@ -973,74 +872,49 @@ _cpp_lex_direct (pfile)
cpp_token *result = pfile->cur_token++; cpp_token *result = pfile->cur_token++;
fresh_line: fresh_line:
result->flags = 0;
if (pfile->buffer->need_line)
{
if (!_cpp_get_fresh_line (pfile))
{
result->type = CPP_EOF;
return result;
}
if (!pfile->keep_tokens)
{
pfile->cur_run = &pfile->base_run;
result = pfile->base_run.base;
pfile->cur_token = result + 1;
}
result->flags = BOL;
if (pfile->state.parsing_args == 2)
result->flags |= PREV_WHITE;
}
buffer = pfile->buffer; buffer = pfile->buffer;
result->flags = buffer->saved_flags;
buffer->saved_flags = 0;
update_tokens_line: update_tokens_line:
result->line = pfile->line; result->line = pfile->line;
skipped_white: skipped_white:
if (buffer->cur >= buffer->notes[buffer->cur_note].pos
&& !pfile->overlaid_buffer)
{
_cpp_process_line_notes (pfile, false);
result->line = pfile->line;
}
c = *buffer->cur++; c = *buffer->cur++;
result->col = CPP_BUF_COLUMN (buffer, buffer->cur); result->col = CPP_BUF_COLUMN (buffer, buffer->cur);
trigraph:
switch (c) switch (c)
{ {
case ' ': case '\t': case '\f': case '\v': case '\0': case ' ': case '\t': case '\f': case '\v': case '\0':
result->flags |= PREV_WHITE; result->flags |= PREV_WHITE;
if (skip_whitespace (pfile, c)) skip_whitespace (pfile, c);
goto skipped_white; goto skipped_white;
/* End of buffer. */
buffer->cur--;
if (continue_after_nul (pfile))
goto fresh_line;
result->type = CPP_EOF;
break;
case '\n': case '\r': case '\n':
handle_newline (pfile); pfile->line++;
buffer->saved_flags = BOL; buffer->need_line = true;
if (! pfile->state.in_directive) goto fresh_line;
{
if (pfile->state.parsing_args == 2)
buffer->saved_flags |= PREV_WHITE;
if (!pfile->keep_tokens)
{
pfile->cur_run = &pfile->base_run;
result = pfile->base_run.base;
pfile->cur_token = result + 1;
}
goto fresh_line;
}
result->type = CPP_EOF;
break;
case '?':
case '\\':
/* These could start an escaped newline, or '?' a trigraph. Let
skip_escaped_newlines do all the work. */
{
unsigned int line = pfile->line;
c = skip_escaped_newlines (pfile);
if (line != pfile->line)
{
buffer->cur--;
/* We had at least one escaped newline of some sort.
Update the token's line and column. */
goto update_tokens_line;
}
}
/* We are either the original '?' or '\\', or a trigraph. */
if (c == '?')
result->type = CPP_QUERY;
else if (c == '\\')
goto random_char;
else
goto trigraph;
break;
case '0': case '1': case '2': case '3': case '4': case '0': case '1': case '2': case '3': case '4':
case '5': case '6': case '7': case '8': case '9': case '5': case '6': case '7': case '8': case '9':
...@@ -1100,7 +974,7 @@ _cpp_lex_direct (pfile) ...@@ -1100,7 +974,7 @@ _cpp_lex_direct (pfile)
if (c == '*') if (c == '*')
{ {
if (skip_block_comment (pfile)) if (_cpp_skip_block_comment (pfile))
cpp_error (pfile, DL_ERROR, "unterminated comment"); cpp_error (pfile, DL_ERROR, "unterminated comment");
} }
else if (c == '/' && (CPP_OPTION (pfile, cplusplus_comments) else if (c == '/' && (CPP_OPTION (pfile, cplusplus_comments)
...@@ -1331,6 +1205,7 @@ _cpp_lex_direct (pfile) ...@@ -1331,6 +1205,7 @@ _cpp_lex_direct (pfile)
case '^': IF_NEXT_IS ('=', CPP_XOR_EQ, CPP_XOR); break; case '^': IF_NEXT_IS ('=', CPP_XOR_EQ, CPP_XOR); break;
case '#': IF_NEXT_IS ('#', CPP_PASTE, CPP_HASH); break; case '#': IF_NEXT_IS ('#', CPP_PASTE, CPP_HASH); break;
case '?': result->type = CPP_QUERY; break;
case '~': result->type = CPP_COMPL; break; case '~': result->type = CPP_COMPL; break;
case ',': result->type = CPP_COMMA; break; case ',': result->type = CPP_COMMA; break;
case '(': result->type = CPP_OPEN_PAREN; break; case '(': result->type = CPP_OPEN_PAREN; break;
...@@ -1349,7 +1224,6 @@ _cpp_lex_direct (pfile) ...@@ -1349,7 +1224,6 @@ _cpp_lex_direct (pfile)
goto start_ident; goto start_ident;
/* Fall through... */ /* Fall through... */
random_char:
default: default:
result->type = CPP_OTHER; result->type = CPP_OTHER;
result->val.c = c; result->val.c = c;
...@@ -1927,10 +1801,6 @@ cpp_interpret_charconst (pfile, token, pchars_seen, unsignedp) ...@@ -1927,10 +1801,6 @@ cpp_interpret_charconst (pfile, token, pchars_seen, unsignedp)
cppchar_t c, mask, result = 0; cppchar_t c, mask, result = 0;
bool unsigned_p; bool unsigned_p;
#ifdef MULTIBYTE_CHARS
(void) local_mbtowc (NULL, NULL, 0);
#endif
/* Width in bits. */ /* Width in bits. */
if (token->type == CPP_CHAR) if (token->type == CPP_CHAR)
{ {
...@@ -1952,25 +1822,7 @@ cpp_interpret_charconst (pfile, token, pchars_seen, unsignedp) ...@@ -1952,25 +1822,7 @@ cpp_interpret_charconst (pfile, token, pchars_seen, unsignedp)
while (str < limit) while (str < limit)
{ {
#ifdef MULTIBYTE_CHARS
wchar_t wc;
int char_len;
char_len = local_mbtowc (&wc, (const char *)str, limit - str);
if (char_len == -1)
{
cpp_error (pfile, DL_WARNING,
"ignoring invalid multibyte character");
c = *str++;
}
else
{
str += char_len;
c = wc;
}
#else
c = *str++; c = *str++;
#endif
if (c == '\\') if (c == '\\')
c = cpp_parse_escape (pfile, &str, limit, token->type == CPP_WCHAR); c = cpp_parse_escape (pfile, &str, limit, token->type == CPP_WCHAR);
......
...@@ -295,7 +295,7 @@ prepare_directive_trad (pfile) ...@@ -295,7 +295,7 @@ prepare_directive_trad (pfile)
|| pfile->directive == &dtable[T_ELIF]); || pfile->directive == &dtable[T_ELIF]);
if (no_expand) if (no_expand)
pfile->state.prevent_expansion++; pfile->state.prevent_expansion++;
_cpp_read_logical_line_trad (pfile); scan_out_logical_line (pfile, NULL);
if (no_expand) if (no_expand)
pfile->state.prevent_expansion--; pfile->state.prevent_expansion--;
pfile->state.skipping = was_skipping; pfile->state.skipping = was_skipping;
...@@ -451,13 +451,12 @@ _cpp_handle_directive (pfile, indented) ...@@ -451,13 +451,12 @@ _cpp_handle_directive (pfile, indented)
/* Restore state when within macro args. */ /* Restore state when within macro args. */
pfile->state.parsing_args = 2; pfile->state.parsing_args = 2;
pfile->state.prevent_expansion = 1; pfile->state.prevent_expansion = 1;
pfile->buffer->saved_flags |= PREV_WHITE;
} }
return skip; return skip;
} }
/* Directive handler wrapper used by the command line option /* Directive handler wrapper used by the command line option
processor. */ processor. BUF is \n terminated. */
static void static void
run_directive (pfile, dir_no, buf, count) run_directive (pfile, dir_no, buf, count)
cpp_reader *pfile; cpp_reader *pfile;
...@@ -471,8 +470,11 @@ run_directive (pfile, dir_no, buf, count) ...@@ -471,8 +470,11 @@ run_directive (pfile, dir_no, buf, count)
if (dir_no == T_PRAGMA) if (dir_no == T_PRAGMA)
pfile->buffer->inc = pfile->buffer->prev->inc; pfile->buffer->inc = pfile->buffer->prev->inc;
start_directive (pfile); start_directive (pfile);
/* We don't want a leading # to be interpreted as a directive. */
pfile->buffer->saved_flags = 0; /* This is a short-term fix to prevent a leading '#' being
interpreted as a directive. */
_cpp_clean_line (pfile);
pfile->directive = &dtable[dir_no]; pfile->directive = &dtable[dir_no];
if (CPP_OPTION (pfile, traditional)) if (CPP_OPTION (pfile, traditional))
prepare_directive_trad (pfile); prepare_directive_trad (pfile);
...@@ -1378,7 +1380,7 @@ destringize_and_run (pfile, in) ...@@ -1378,7 +1380,7 @@ destringize_and_run (pfile, in)
src++; src++;
*dest++ = *src++; *dest++ = *src++;
} }
*dest = '\0'; *dest = '\n';
/* Ugh; an awful kludge. We are really not set up to be lexing /* Ugh; an awful kludge. We are really not set up to be lexing
tokens when in the middle of a macro expansion. Use a new tokens when in the middle of a macro expansion. Use a new
...@@ -1904,7 +1906,7 @@ cpp_define (pfile, str) ...@@ -1904,7 +1906,7 @@ cpp_define (pfile, str)
buf[count++] = ' '; buf[count++] = ' ';
buf[count++] = '1'; buf[count++] = '1';
} }
buf[count] = '\0'; buf[count] = '\n';
run_directive (pfile, T_DEFINE, buf, count); run_directive (pfile, T_DEFINE, buf, count);
} }
...@@ -1915,7 +1917,11 @@ _cpp_define_builtin (pfile, str) ...@@ -1915,7 +1917,11 @@ _cpp_define_builtin (pfile, str)
cpp_reader *pfile; cpp_reader *pfile;
const char *str; const char *str;
{ {
run_directive (pfile, T_DEFINE, str, strlen (str)); size_t len = strlen (str);
char *buf = alloca (len + 1);
memcpy (buf, str, len);
buf[len] = '\n';
run_directive (pfile, T_DEFINE, buf, len);
} }
/* Process MACRO as if it appeared as the body of an #undef. */ /* Process MACRO as if it appeared as the body of an #undef. */
...@@ -1924,7 +1930,11 @@ cpp_undef (pfile, macro) ...@@ -1924,7 +1930,11 @@ cpp_undef (pfile, macro)
cpp_reader *pfile; cpp_reader *pfile;
const char *macro; const char *macro;
{ {
run_directive (pfile, T_UNDEF, macro, strlen (macro)); size_t len = strlen (macro);
char *buf = alloca (len + 1);
memcpy (buf, macro, len);
buf[len] = '\n';
run_directive (pfile, T_UNDEF, buf, len);
} }
/* Process the string STR as if it appeared as the body of a #assert. */ /* Process the string STR as if it appeared as the body of a #assert. */
...@@ -1955,18 +1965,18 @@ handle_assertion (pfile, str, type) ...@@ -1955,18 +1965,18 @@ handle_assertion (pfile, str, type)
size_t count = strlen (str); size_t count = strlen (str);
const char *p = strchr (str, '='); const char *p = strchr (str, '=');
/* Copy the entire option so we can modify it. Change the first
"=" in the string to a '(', and tack a ')' on the end. */
char *buf = (char *) alloca (count + 2);
memcpy (buf, str, count);
if (p) if (p)
{ {
/* Copy the entire option so we can modify it. Change the first
"=" in the string to a '(', and tack a ')' on the end. */
char *buf = (char *) alloca (count + 2);
memcpy (buf, str, count);
buf[p - str] = '('; buf[p - str] = '(';
buf[count++] = ')'; buf[count++] = ')';
buf[count] = '\0';
str = buf;
} }
buf[count] = '\n';
str = buf;
run_directive (pfile, type, str, count); run_directive (pfile, type, str, count);
} }
...@@ -2028,15 +2038,14 @@ cpp_push_buffer (pfile, buffer, len, from_stage3, return_at_eof) ...@@ -2028,15 +2038,14 @@ cpp_push_buffer (pfile, buffer, len, from_stage3, return_at_eof)
/* Clears, amongst other things, if_stack and mi_cmacro. */ /* Clears, amongst other things, if_stack and mi_cmacro. */
memset (new, 0, sizeof (cpp_buffer)); memset (new, 0, sizeof (cpp_buffer));
new->line_base = new->buf = new->cur = buffer; new->next_line = new->buf = buffer;
new->rlimit = buffer + len; new->rlimit = buffer + len;
new->from_stage3 = from_stage3 || CPP_OPTION (pfile, traditional); new->from_stage3 = from_stage3;
new->prev = pfile->buffer; new->prev = pfile->buffer;
new->return_at_eof = return_at_eof; new->return_at_eof = return_at_eof;
new->saved_flags = BOL; new->need_line = true;
pfile->buffer = new; pfile->buffer = new;
return new; return new;
} }
...@@ -2062,6 +2071,8 @@ _cpp_pop_buffer (pfile) ...@@ -2062,6 +2071,8 @@ _cpp_pop_buffer (pfile)
/* _cpp_do_file_change expects pfile->buffer to be the new one. */ /* _cpp_do_file_change expects pfile->buffer to be the new one. */
pfile->buffer = buffer->prev; pfile->buffer = buffer->prev;
free (buffer->notes);
/* Free the buffer object now; we may want to push a new buffer /* Free the buffer object now; we may want to push a new buffer
in _cpp_push_next_include_file. */ in _cpp_push_next_include_file. */
obstack_free (&pfile->buffer_ob, buffer); obstack_free (&pfile->buffer_ob, buffer);
......
...@@ -265,6 +265,8 @@ builtin_macro (pfile, node) ...@@ -265,6 +265,8 @@ builtin_macro (pfile, node)
cpp_hashnode *node; cpp_hashnode *node;
{ {
const uchar *buf; const uchar *buf;
size_t len;
char *nbuf;
if (node->value.builtin == BT_PRAGMA) if (node->value.builtin == BT_PRAGMA)
{ {
...@@ -278,14 +280,13 @@ builtin_macro (pfile, node) ...@@ -278,14 +280,13 @@ builtin_macro (pfile, node)
} }
buf = _cpp_builtin_macro_text (pfile, node); buf = _cpp_builtin_macro_text (pfile, node);
len = ustrlen (buf);
nbuf = alloca (len + 1);
memcpy (nbuf, buf, len);
nbuf[len]='\n';
cpp_push_buffer (pfile, buf, ustrlen (buf), /* from_stage3 */ true, 1); cpp_push_buffer (pfile, (uchar *) nbuf, len, /* from_stage3 */ true, 1);
_cpp_clean_line (pfile);
/* Tweak the column number the lexer will report. */
pfile->buffer->col_adjust = pfile->cur_token[-1].col - 1;
/* We don't want a leading # to be interpreted as a directive. */
pfile->buffer->saved_flags = 0;
/* Set pfile->cur_token as required by _cpp_lex_direct. */ /* Set pfile->cur_token as required by _cpp_lex_direct. */
pfile->cur_token = _cpp_temp_token (pfile); pfile->cur_token = _cpp_temp_token (pfile);
...@@ -445,15 +446,10 @@ paste_tokens (pfile, plhs, rhs) ...@@ -445,15 +446,10 @@ paste_tokens (pfile, plhs, rhs)
if (lhs->type == CPP_DIV && rhs->type != CPP_EQ) if (lhs->type == CPP_DIV && rhs->type != CPP_EQ)
*end++ = ' '; *end++ = ' ';
end = cpp_spell_token (pfile, rhs, end); end = cpp_spell_token (pfile, rhs, end);
*end = '\0'; *end = '\n';
cpp_push_buffer (pfile, buf, end - buf, /* from_stage3 */ true, 1); cpp_push_buffer (pfile, buf, end - buf, /* from_stage3 */ true, 1);
_cpp_clean_line (pfile);
/* Tweak the column number the lexer will report. */
pfile->buffer->col_adjust = pfile->cur_token[-1].col - 1;
/* We don't want a leading # to be interpreted as a directive. */
pfile->buffer->saved_flags = 0;
/* Set pfile->cur_token as required by _cpp_lex_direct. */ /* Set pfile->cur_token as required by _cpp_lex_direct. */
pfile->cur_token = _cpp_temp_token (pfile); pfile->cur_token = _cpp_temp_token (pfile);
......
...@@ -708,7 +708,7 @@ cpp_read_state (r, name, f, data) ...@@ -708,7 +708,7 @@ cpp_read_state (r, name, f, data)
if (fread (defn, 1, m.definition_length, f) != m.definition_length) if (fread (defn, 1, m.definition_length, f) != m.definition_length)
goto error; goto error;
defn[m.definition_length] = '\0'; defn[m.definition_length] = '\n';
h = cpp_lookup (r, defn, m.name_length); h = cpp_lookup (r, defn, m.name_length);
...@@ -722,6 +722,7 @@ cpp_read_state (r, name, f, data) ...@@ -722,6 +722,7 @@ cpp_read_state (r, name, f, data)
m.definition_length - m.name_length, m.definition_length - m.name_length,
true, 1) != NULL) true, 1) != NULL)
{ {
_cpp_clean_line (r);
if (!_cpp_create_definition (r, h)) if (!_cpp_create_definition (r, h))
abort (); abort ();
_cpp_pop_buffer (r); _cpp_pop_buffer (r);
......
...@@ -81,14 +81,10 @@ enum ls {ls_none = 0, /* Normal state. */ ...@@ -81,14 +81,10 @@ enum ls {ls_none = 0, /* Normal state. */
/* Lexing TODO: Maybe handle space in escaped newlines. Stop cpplex.c /* Lexing TODO: Maybe handle space in escaped newlines. Stop cpplex.c
from recognizing comments and directives during its lexing pass. */ from recognizing comments and directives during its lexing pass. */
static const uchar *handle_newline PARAMS ((cpp_reader *, const uchar *));
static const uchar *skip_escaped_newlines PARAMS ((cpp_reader *,
const uchar *));
static const uchar *skip_whitespace PARAMS ((cpp_reader *, const uchar *, static const uchar *skip_whitespace PARAMS ((cpp_reader *, const uchar *,
int)); int));
static cpp_hashnode *lex_identifier PARAMS ((cpp_reader *, const uchar *)); static cpp_hashnode *lex_identifier PARAMS ((cpp_reader *, const uchar *));
static const uchar *copy_comment PARAMS ((cpp_reader *, const uchar *, int)); static const uchar *copy_comment PARAMS ((cpp_reader *, const uchar *, int));
static void scan_out_logical_line PARAMS ((cpp_reader *pfile, cpp_macro *));
static void check_output_buffer PARAMS ((cpp_reader *, size_t)); static void check_output_buffer PARAMS ((cpp_reader *, size_t));
static void push_replacement_text PARAMS ((cpp_reader *, cpp_hashnode *)); static void push_replacement_text PARAMS ((cpp_reader *, cpp_hashnode *));
static bool scan_parameters PARAMS ((cpp_reader *, cpp_macro *)); static bool scan_parameters PARAMS ((cpp_reader *, cpp_macro *));
...@@ -125,43 +121,6 @@ check_output_buffer (pfile, n) ...@@ -125,43 +121,6 @@ check_output_buffer (pfile, n)
} }
} }
/* To be called whenever a newline character is encountered in the
input file, at CUR. Handles DOS, Mac and Unix ends of line, and
increments pfile->line.
Returns a pointer the character after the newline sequence. */
static const uchar *
handle_newline (pfile, cur)
cpp_reader *pfile;
const uchar *cur;
{
pfile->line++;
if (cur[0] + cur[1] == '\r' + '\n')
cur++;
return cur + 1;
}
/* CUR points to any character in the current context, not necessarily
a backslash. Advances CUR until all escaped newlines are skipped,
and returns the new position without updating the context.
Warns if a file buffer ends in an escaped newline. */
static const uchar *
skip_escaped_newlines (pfile, cur)
cpp_reader *pfile;
const uchar *cur;
{
const uchar *orig_cur = cur;
while (*cur == '\\' && is_vspace (cur[1]))
cur = handle_newline (pfile, cur + 1);
if (cur != orig_cur && cur == RLIMIT (pfile->context) && pfile->buffer->inc)
cpp_error (pfile, DL_PEDWARN, "backslash-newline at end of file");
return cur;
}
/* CUR points to the asterisk introducing a comment in the current /* CUR points to the asterisk introducing a comment in the current
context. IN_DEFINE is true if we are in the replacement text of a context. IN_DEFINE is true if we are in the replacement text of a
macro. macro.
...@@ -180,43 +139,16 @@ copy_comment (pfile, cur, in_define) ...@@ -180,43 +139,16 @@ copy_comment (pfile, cur, in_define)
const uchar *cur; const uchar *cur;
int in_define; int in_define;
{ {
bool unterminated, copy = false;
unsigned int from_line = pfile->line; unsigned int from_line = pfile->line;
const uchar *limit = RLIMIT (pfile->context); cpp_buffer *buffer = pfile->buffer;
uchar *out = pfile->out.cur;
do
{
unsigned int c = *cur++;
*out++ = c;
if (c == '/')
{
/* An immediate slash does not terminate the comment. */
if (out[-2] == '*' && out - 2 > pfile->out.cur)
goto done;
if (*cur == '*' && cur[1] != '/'
&& CPP_OPTION (pfile, warn_comments))
cpp_error_with_line (pfile, DL_WARNING, pfile->line, 0,
"\"/*\" within comment");
}
else if (is_vspace (c))
{
cur = handle_newline (pfile, cur - 1);
/* Canonicalize newline sequences and skip escaped ones. */
if (out[-2] == '\\')
out -= 2;
else
out[-1] = '\n';
}
}
while (cur < limit);
cpp_error_with_line (pfile, DL_ERROR, from_line, 0, "unterminated comment"); buffer->cur = cur;
*out++ = '*'; unterminated = _cpp_skip_block_comment (pfile);
*out++ = '/'; if (unterminated)
cpp_error_with_line (pfile, DL_ERROR, from_line, 0,
"unterminated comment");
done:
/* Comments in directives become spaces so that tokens are properly /* Comments in directives become spaces so that tokens are properly
separated when the ISO preprocessor re-lexes the line. The separated when the ISO preprocessor re-lexes the line. The
exception is #define. */ exception is #define. */
...@@ -227,7 +159,7 @@ copy_comment (pfile, cur, in_define) ...@@ -227,7 +159,7 @@ copy_comment (pfile, cur, in_define)
if (CPP_OPTION (pfile, discard_comments_in_macro_exp)) if (CPP_OPTION (pfile, discard_comments_in_macro_exp))
pfile->out.cur--; pfile->out.cur--;
else else
pfile->out.cur = out; copy = true;
} }
else else
pfile->out.cur[-1] = ' '; pfile->out.cur[-1] = ' ';
...@@ -235,9 +167,21 @@ copy_comment (pfile, cur, in_define) ...@@ -235,9 +167,21 @@ copy_comment (pfile, cur, in_define)
else if (CPP_OPTION (pfile, discard_comments)) else if (CPP_OPTION (pfile, discard_comments))
pfile->out.cur--; pfile->out.cur--;
else else
pfile->out.cur = out; copy = true;
return cur; if (copy)
{
size_t len = (size_t) (buffer->cur - cur);
memcpy (pfile->out.cur, cur, len);
pfile->out.cur += len;
if (unterminated)
{
*pfile->out.cur++ = '*';
*pfile->out.cur++ = '/';
}
}
return buffer->cur;
} }
/* CUR points to any character in the input buffer. Skips over all /* CUR points to any character in the input buffer. Skips over all
...@@ -265,31 +209,18 @@ skip_whitespace (pfile, cur, skip_comments) ...@@ -265,31 +209,18 @@ skip_whitespace (pfile, cur, skip_comments)
unsigned int c = *cur++; unsigned int c = *cur++;
*out++ = c; *out++ = c;
if (is_nvspace (c) && c) if (is_nvspace (c))
continue; continue;
if (!c && cur - 1 != RLIMIT (pfile->context)) if (c == '/' && *cur == '*' && skip_comments)
continue;
if (c == '/' && skip_comments)
{
const uchar *tmp = skip_escaped_newlines (pfile, cur);
if (*tmp == '*')
{
pfile->out.cur = out;
cur = copy_comment (pfile, tmp, false /* in_define */);
out = pfile->out.cur;
continue;
}
}
out--;
if (c == '\\' && is_vspace (*cur))
{ {
cur = skip_escaped_newlines (pfile, cur - 1); pfile->out.cur = out;
cur = copy_comment (pfile, cur, false /* in_define */);
out = pfile->out.cur;
continue; continue;
} }
out--;
break; break;
} }
...@@ -310,12 +241,7 @@ lex_identifier (pfile, cur) ...@@ -310,12 +241,7 @@ lex_identifier (pfile, cur)
cpp_hashnode *result; cpp_hashnode *result;
do do
{ *out++ = *cur++;
do
*out++ = *cur++;
while (is_numchar (*cur));
cur = skip_escaped_newlines (pfile, cur);
}
while (is_numchar (*cur)); while (is_numchar (*cur));
CUR (pfile->context) = cur; CUR (pfile->context) = cur;
...@@ -340,11 +266,12 @@ _cpp_overlay_buffer (pfile, start, len) ...@@ -340,11 +266,12 @@ _cpp_overlay_buffer (pfile, start, len)
pfile->overlaid_buffer = buffer; pfile->overlaid_buffer = buffer;
buffer->saved_cur = buffer->cur; buffer->saved_cur = buffer->cur;
buffer->saved_rlimit = buffer->rlimit; buffer->saved_rlimit = buffer->rlimit;
/* Prevent the ISO lexer from scanning a fresh line. */
pfile->saved_line = pfile->line--;
buffer->need_line = false;
buffer->cur = start; buffer->cur = start;
buffer->rlimit = start + len; buffer->rlimit = start + len;
pfile->saved_line = pfile->line;
} }
/* Restores a buffer overlaid by _cpp_overlay_buffer(). */ /* Restores a buffer overlaid by _cpp_overlay_buffer(). */
...@@ -356,7 +283,9 @@ _cpp_remove_overlay (pfile) ...@@ -356,7 +283,9 @@ _cpp_remove_overlay (pfile)
buffer->cur = buffer->saved_cur; buffer->cur = buffer->saved_cur;
buffer->rlimit = buffer->saved_rlimit; buffer->rlimit = buffer->saved_rlimit;
buffer->need_line = true;
pfile->overlaid_buffer = NULL;
pfile->line = pfile->saved_line; pfile->line = pfile->saved_line;
} }
...@@ -368,24 +297,10 @@ _cpp_read_logical_line_trad (pfile) ...@@ -368,24 +297,10 @@ _cpp_read_logical_line_trad (pfile)
{ {
do do
{ {
if (pfile->buffer->cur == pfile->buffer->rlimit) if (pfile->buffer->need_line && !_cpp_get_fresh_line (pfile))
{ return false;
bool stop = true;
/* Don't pop the last buffer. */
if (pfile->buffer->prev)
{
stop = pfile->buffer->return_at_eof;
_cpp_pop_buffer (pfile);
}
if (stop)
return false;
}
scan_out_logical_line (pfile, NULL);
} }
while (pfile->state.skipping); while (!scan_out_logical_line (pfile, NULL) || pfile->state.skipping);
return true; return true;
} }
...@@ -428,11 +343,12 @@ save_argument (macro, offset) ...@@ -428,11 +343,12 @@ save_argument (macro, offset)
If MACRO is non-NULL, then we are scanning the replacement list of If MACRO is non-NULL, then we are scanning the replacement list of
MACRO, and we call save_replacement_text() every time we meet an MACRO, and we call save_replacement_text() every time we meet an
argument. */ argument. */
static void bool
scan_out_logical_line (pfile, macro) scan_out_logical_line (pfile, macro)
cpp_reader *pfile; cpp_reader *pfile;
cpp_macro *macro; cpp_macro *macro;
{ {
bool result = true;
cpp_context *context; cpp_context *context;
const uchar *cur; const uchar *cur;
uchar *out; uchar *out;
...@@ -443,7 +359,6 @@ scan_out_logical_line (pfile, macro) ...@@ -443,7 +359,6 @@ scan_out_logical_line (pfile, macro)
fmacro.buff = NULL; fmacro.buff = NULL;
start_logical_line:
quote = 0; quote = 0;
header_ok = pfile->state.angled_headers; header_ok = pfile->state.angled_headers;
CUR (pfile->context) = pfile->buffer->cur; CUR (pfile->context) = pfile->buffer->cur;
...@@ -458,6 +373,12 @@ scan_out_logical_line (pfile, macro) ...@@ -458,6 +373,12 @@ scan_out_logical_line (pfile, macro)
for (;;) for (;;)
{ {
if (!context->prev
&& cur >= pfile->buffer->notes[pfile->buffer->cur_note].pos)
{
pfile->buffer->cur = cur;
_cpp_process_line_notes (pfile, false);
}
c = *cur++; c = *cur++;
*out++ = c; *out++ = c;
...@@ -469,12 +390,10 @@ scan_out_logical_line (pfile, macro) ...@@ -469,12 +390,10 @@ scan_out_logical_line (pfile, macro)
case '\t': case '\t':
case '\f': case '\f':
case '\v': case '\v':
continue;
case '\0': case '\0':
if (cur - 1 != RLIMIT (context)) continue;
continue;
case '\n':
/* If this is a macro's expansion, pop it. */ /* If this is a macro's expansion, pop it. */
if (context->prev) if (context->prev)
{ {
...@@ -483,22 +402,21 @@ scan_out_logical_line (pfile, macro) ...@@ -483,22 +402,21 @@ scan_out_logical_line (pfile, macro)
goto new_context; goto new_context;
} }
/* Premature end of file. Fake a new line. */ /* Omit the newline from the output buffer. */
cur--; pfile->out.cur = out - 1;
if (!pfile->buffer->from_stage3) pfile->buffer->cur = cur;
cpp_error (pfile, DL_PEDWARN, "no newline at end of file"); pfile->buffer->need_line = true;
pfile->line++; pfile->line++;
goto done;
case '\r': case '\n':
cur = handle_newline (pfile, cur - 1);
if ((lex_state == ls_fun_open || lex_state == ls_fun_close) if ((lex_state == ls_fun_open || lex_state == ls_fun_close)
&& !pfile->state.in_directive) && !pfile->state.in_directive
&& _cpp_get_fresh_line (pfile))
{ {
/* Newlines in arguments become a space, but we don't /* Newlines in arguments become a space, but we don't
clear any in-progress quote. */ clear any in-progress quote. */
if (lex_state == ls_fun_close) if (lex_state == ls_fun_close)
out[-1] = ' '; out[-1] = ' ';
cur = pfile->buffer->cur;
continue; continue;
} }
goto done; goto done;
...@@ -521,35 +439,20 @@ scan_out_logical_line (pfile, macro) ...@@ -521,35 +439,20 @@ scan_out_logical_line (pfile, macro)
break; break;
case '\\': case '\\':
if (is_vspace (*cur)) /* Skip escaped quotes here, it's easier than above. */
{ if (*cur == '\\' || *cur == '"' || *cur == '\'')
out--; *out++ = *cur++;
cur = skip_escaped_newlines (pfile, cur - 1);
continue;
}
else
{
/* Skip escaped quotes here, it's easier than above, but
take care to first skip escaped newlines. */
cur = skip_escaped_newlines (pfile, cur);
if (*cur == '\\' || *cur == '"' || *cur == '\'')
*out++ = *cur++;
}
break; break;
case '/': case '/':
/* Traditional CPP does not recognize comments within /* Traditional CPP does not recognize comments within
literals. */ literals. */
if (!quote) if (!quote && *cur == '*')
{ {
cur = skip_escaped_newlines (pfile, cur); pfile->out.cur = out;
if (*cur == '*') cur = copy_comment (pfile, cur, macro != 0);
{ out = pfile->out.cur;
pfile->out.cur = out; continue;
cur = copy_comment (pfile, cur, macro != 0);
out = pfile->out.cur;
continue;
}
} }
break; break;
...@@ -699,12 +602,14 @@ scan_out_logical_line (pfile, macro) ...@@ -699,12 +602,14 @@ scan_out_logical_line (pfile, macro)
cur = skip_whitespace (pfile, cur, true /* skip_comments */); cur = skip_whitespace (pfile, cur, true /* skip_comments */);
out = pfile->out.cur; out = pfile->out.cur;
if (is_vspace (*cur)) if (*cur == '\n')
{ {
/* Null directive. Ignore it and don't invalidate /* Null directive. Ignore it and don't invalidate
the MI optimization. */ the MI optimization. */
out = pfile->out.base; pfile->buffer->need_line = true;
continue; pfile->line++;
result = false;
goto done;
} }
else else
{ {
...@@ -724,9 +629,8 @@ scan_out_logical_line (pfile, macro) ...@@ -724,9 +629,8 @@ scan_out_logical_line (pfile, macro)
preprocessor lex the next token. */ preprocessor lex the next token. */
pfile->buffer->cur = cur; pfile->buffer->cur = cur;
_cpp_handle_directive (pfile, false /* indented */); _cpp_handle_directive (pfile, false /* indented */);
/* #include changes pfile->buffer so we need to result = false;
update the limits of the current context. */ goto done;
goto start_logical_line;
} }
} }
} }
...@@ -765,9 +669,6 @@ scan_out_logical_line (pfile, macro) ...@@ -765,9 +669,6 @@ scan_out_logical_line (pfile, macro)
} }
done: done:
out[-1] = '\0';
pfile->buffer->cur = cur;
pfile->out.cur = out - 1;
if (fmacro.buff) if (fmacro.buff)
_cpp_release_buff (pfile, fmacro.buff); _cpp_release_buff (pfile, fmacro.buff);
...@@ -775,6 +676,7 @@ scan_out_logical_line (pfile, macro) ...@@ -775,6 +676,7 @@ scan_out_logical_line (pfile, macro)
cpp_error_with_line (pfile, DL_ERROR, fmacro.line, 0, cpp_error_with_line (pfile, DL_ERROR, fmacro.line, 0,
"unterminated argument list invoking macro \"%s\"", "unterminated argument list invoking macro \"%s\"",
NODE_NAME (fmacro.node)); NODE_NAME (fmacro.node));
return result;
} }
/* Push a context holding the replacement text of the macro NODE on /* Push a context holding the replacement text of the macro NODE on
...@@ -787,11 +689,16 @@ push_replacement_text (pfile, node) ...@@ -787,11 +689,16 @@ push_replacement_text (pfile, node)
{ {
size_t len; size_t len;
const uchar *text; const uchar *text;
uchar *buf;
if (node->flags & NODE_BUILTIN) if (node->flags & NODE_BUILTIN)
{ {
text = _cpp_builtin_macro_text (pfile, node); text = _cpp_builtin_macro_text (pfile, node);
len = ustrlen (text); len = ustrlen (text);
buf = _cpp_unaligned_alloc (pfile, len + 1);
memcpy (buf, text, len);
buf[len]='\n';
text = buf;
} }
else else
{ {
...@@ -944,7 +851,7 @@ replace_args_and_push (pfile, fmacro) ...@@ -944,7 +851,7 @@ replace_args_and_push (pfile, fmacro)
exp += BLOCK_LEN (b->text_len); exp += BLOCK_LEN (b->text_len);
} }
/* Allocate room for the expansion plus NUL. */ /* Allocate room for the expansion plus \n. */
buff = _cpp_get_buff (pfile, len + 1); buff = _cpp_get_buff (pfile, len + 1);
/* Copy the expansion and replace arguments. */ /* Copy the expansion and replace arguments. */
...@@ -966,8 +873,8 @@ replace_args_and_push (pfile, fmacro) ...@@ -966,8 +873,8 @@ replace_args_and_push (pfile, fmacro)
exp += BLOCK_LEN (b->text_len); exp += BLOCK_LEN (b->text_len);
} }
/* NUL-terminate. */ /* \n-terminate. */
*p = '\0'; *p = '\n';
_cpp_push_text_context (pfile, fmacro->node, BUFF_FRONT (buff), len); _cpp_push_text_context (pfile, fmacro->node, BUFF_FRONT (buff), len);
/* So we free buffer allocation when macro is left. */ /* So we free buffer allocation when macro is left. */
...@@ -1034,10 +941,10 @@ save_replacement_text (pfile, macro, arg_index) ...@@ -1034,10 +941,10 @@ save_replacement_text (pfile, macro, arg_index)
if (macro->paramc == 0) if (macro->paramc == 0)
{ {
/* Object-like and function-like macros without parameters /* Object-like and function-like macros without parameters
simply store their NUL-terminated replacement text. */ simply store their \n-terminated replacement text. */
exp = _cpp_unaligned_alloc (pfile, len + 1); exp = _cpp_unaligned_alloc (pfile, len + 1);
memcpy (exp, pfile->out.base, len); memcpy (exp, pfile->out.base, len);
exp[len] = '\0'; exp[len] = '\n';
macro->exp.text = exp; macro->exp.text = exp;
macro->count = len; macro->count = len;
} }
......
...@@ -692,7 +692,7 @@ read_scan_file (in_fname, argc, argv) ...@@ -692,7 +692,7 @@ read_scan_file (in_fname, argc, argv)
if (special_file_handling == stdio_h if (special_file_handling == stdio_h
&& (fn = lookup_std_proto ("_filbuf", 7)) != NULL) && (fn = lookup_std_proto ("_filbuf", 7)) != NULL)
{ {
static const unsigned char getchar_call[] = "getchar();"; unsigned char getchar_call[] = "getchar();\n";
int seen_filbuf = 0; int seen_filbuf = 0;
/* Scan the macro expansion of "getchar();". */ /* Scan the macro expansion of "getchar();". */
......
2003-04-19 Neil Booth <neil@daikokuya.co.uk>
* gcc.dg/cpp/_Pragma4.c: Remove stray space.
* gcc.dg/cpp/trad/escaped-eof.c: Correct line number.
2003-04-18 Eric Botcazou <ebotcazou@libertysurf.fr> 2003-04-18 Eric Botcazou <ebotcazou@libertysurf.fr>
* gcc.c-torture/compile/20030418-1.c: New test. * gcc.c-torture/compile/20030418-1.c: New test.
......
...@@ -7,6 +7,6 @@ a b c ...@@ -7,6 +7,6 @@ a b c
/* /*
{ dg-final { if ![file exists _Pragma4.i] { return } } } { dg-final { if ![file exists _Pragma4.i] { return } } }
{ dg-final { if { [grep _Pragma4.i "#pragma bar "] != "" } { return } } } { dg-final { if { [grep _Pragma4.i "#pragma bar"] != "" } { return } } }
{ dg-final { fail "_Pragma4.c: #pragma appearing on its own line" } } { dg-final { fail "_Pragma4.c: #pragma appearing on its own line" } }
*/ */
...@@ -2,5 +2,5 @@ ...@@ -2,5 +2,5 @@
/* { dg-do preprocess } */ /* { dg-do preprocess } */
/* { dg-warning "backslash-new" "escaped EOF warning" { target *-*-* } 7 } */ /* { dg-warning "backslash-new" "escaped EOF warning" { target *-*-* } 6 } */
\ \
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment