Linemap infrastructure for virtual locations

This is the first instalment of a set which goal is to track locations
of tokens across macro expansions.  Tom Tromey did the original work
and attached the patch to PR preprocessor/7263.  This opus is a
derivative of that original work.

This patch modifies the linemap module of libcpp to add virtual
locations support.

A virtual location is a mapped location that can resolve to several
different physical locations.  It can always resolve to the spelling
location of a token.  For tokens resulting from macro expansion it can
resolve to:
  - either the location of the expansion point of the macro.
  - or the location of the token in the definition of the
  macro
  - or, if the token is an argument of a function-like macro,
  the location of the use of the matching macro parameter in
  the definition of the macro

The patch creates a new type of line map called a macro map.  For every
single macro expansion, there is a macro map that generates a virtual
location for every single resulting token of the expansion.

The good old type of line map we all know is now called an ordinary
map.  That one still encodes spelling locations as it has always had.

As a result linemap_lookup as been extended to return a macro map when
given a virtual location resulting from a macro expansion.  The layout
of structs line_map has changed to support this new type of map.  So
did the layout of struct line_maps.  Accessor macros have been
introduced to avoid messing with the implementation details of these
datastructures directly.  This helped already as we have been testing
different ways of arranging these datastructure.  Having to constantly
adjust client code that is too tied with the internals of line_map and
line_maps would have been even more painful.

Of course, many new public functions have been added to the linemap
module to handle the resolution of virtual locations.

This patch introduces the infrastructure but no part of the compiler
uses virtual locations yet.

However the client code of the linemap data structures has been
adjusted as per the changes.  E.g, it's not anymore reliable for a
client code to manipulate struct line_map directly if it just wants to
deal with spelling locations, because struct line_map can now
represent a macro map as well.  In that case, it's better to use the
convenient API to resolve the initial (possibly virtual) location to a
spelling location (or to an ordinary map) and use that.

This is the reason why the patch adjusts the Java, Ada and Fortran
front ends.

Also, note that virtual locations are not supposed to be ordered for
relations '<' and '>' anymore.  To test if a virtual location appears
"before" another one, one has to use a new operator exposed by the
line map interface.  The patch updates the only spot (in the
diagnostics module) I have found that was making the assumption that
locations were ordered for these relations.  This is the only change
that introduces a use of the new line map API in this patch, so I am
adding a regression test for it only.

From-SVN: r180081

gcc/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* input.h (struct expanded_location): Move to libcpp/line-map.h.
+	(LOCATION_COLUMN): New accessor
+	(in_system_header_at): Use linemap_location_in_system_header_p.
+	* diagnostic.c (diagnostic_report_current_module): Adjust to avoid
+	touching the internals of struct line_map.  Use the public API.
+	instead.
+	(diagnostic_report_diagnostic): Don't use relational operator '<'
+	on virtual locations.  Use linemap_location_before_p instead.
+	* input.c (expand_location): Adjust to expand to the tokens'
+	spelling location when macro location tracking is on.
+
+
 2011-10-08  Andi Kleen  <ak@linux.intel.com>
 
 	* ggc-page.c (GGC_QUIRE_SIZE): Increase to 512

gcc/ada/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat.com> 
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* gcc-interface/trans.c (gigi, Sloc_to_locus): Adjust to use the
+	new public ordinary map interface.
+
 2011-10-16  Tristan Gingold  <gingold@adacore.com>
 
 	* link.c (_AIX): Add support for GNU ld.        

gcc/ada/gcc-interface/trans.c

@@ -314,7 +314,7 @@ gigi (Node_Id gnat_root, int max_gnat_node, int number_name ATTRIBUTE_UNUSED,
 	     (Get_Name_String (file_info_ptr[i].File_Name))));
 
       /* We rely on the order isomorphism between files and line maps.  */
-      gcc_assert ((int) line_table->used == i);
+      gcc_assert ((int) LINEMAPS_ORDINARY_USED (line_table) == i);
 
       /* We create the line map for a source file at once, with a fixed number
 	 of columns chosen to avoid jumping over the next power of 2.  */
@@ -8391,12 +8391,10 @@ Sloc_to_locus (Source_Ptr Sloc, location_t *locus)
       Source_File_Index file = Get_Source_File_Index (Sloc);
       Logical_Line_Number line = Get_Logical_Line_Number (Sloc);
       Column_Number column = Get_Column_Number (Sloc);
-      struct line_map *map = &line_table->maps[file - 1];
+      struct line_map *map = LINEMAPS_ORDINARY_MAP_AT (line_table, file - 1);
 
-      /* Translate the location according to the line-map.h formula.  */
-      *locus = map->start_location
-		+ ((line - map->to_line) << map->column_bits)
-		+ (column & ((1 << map->column_bits) - 1));
+      /* Translate the location.  */
+      *locus = linemap_position_for_line_and_column (map, line, column);
     }
 
   ref_filename

gcc/c-family/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* c-ppoutput.c (scan_translation_unit, maybe_print_line)
+	(print_line, cb_define, do_line_change): Adjust to avoid touching
+	the internals of struct line_map.  Use the public API instead.
+	* c-pch.c (c_common_read_pch): Likewise.
+	* c-lex.c (fe_file_change): Likewise.
+
 2011-10-11  Michael Meissner  <meissner@linux.vnet.ibm.com>
 
 	* c-common.c (def_builtin_1): Delete old interface with two

gcc/c-family/c-lex.c

@@ -207,7 +207,7 @@ fe_file_change (const struct line_map *new_map)
 	    line = SOURCE_LINE (new_map - 1, included_at);
 
 	  input_location = new_map->start_location;
-	  (*debug_hooks->start_source_file) (line, new_map->to_file);
+	  (*debug_hooks->start_source_file) (line, LINEMAP_FILE (new_map));
 #ifndef NO_IMPLICIT_EXTERN_C
 	  if (c_header_level)
 	    ++c_header_level;
@@ -231,10 +231,10 @@ fe_file_change (const struct line_map *new_map)
 #endif
       input_location = new_map->start_location;
 
-      (*debug_hooks->end_source_file) (new_map->to_line);
+      (*debug_hooks->end_source_file) (LINEMAP_LINE (new_map));
     }
 
-  update_header_times (new_map->to_file);
+  update_header_times (LINEMAP_FILE (new_map));
   input_location = new_map->start_location;
 }
 

gcc/c-family/c-ppoutput.c

@@ -190,9 +190,7 @@ scan_translation_unit (cpp_reader *pfile)
       /* Subtle logic to output a space if and only if necessary.  */
       if (avoid_paste)
 	{
-	  const struct line_map *map
-	    = linemap_lookup (line_table, loc);
-	  int src_line = SOURCE_LINE (map, loc);
+	  int src_line = LOCATION_LINE (loc);
 
 	  if (print.source == NULL)
 	    print.source = token;
@@ -212,9 +210,7 @@ scan_translation_unit (cpp_reader *pfile)
 	}
       else if (token->flags & PREV_WHITE)
 	{
-	  const struct line_map *map
-	    = linemap_lookup (line_table, loc);
-	  int src_line = SOURCE_LINE (map, loc);
+	  int src_line = LOCATION_LINE (loc);
 
 	  if (src_line != print.src_line
 	      && do_line_adjustments
@@ -304,8 +300,9 @@ scan_translation_unit_trad (cpp_reader *pfile)
 static void
 maybe_print_line (source_location src_loc)
 {
-  const struct line_map *map = linemap_lookup (line_table, src_loc);
-  int src_line = SOURCE_LINE (map, src_loc);
+  int src_line = LOCATION_LINE (src_loc);
+  const char *src_file = LOCATION_FILE (src_loc);
+
   /* End the previous line of text.  */
   if (print.printed)
     {
@@ -317,7 +314,7 @@ maybe_print_line (source_location src_loc)
   if (!flag_no_line_commands
       && src_line >= print.src_line
       && src_line < print.src_line + 8
-      && strcmp (map->to_file, print.src_file) == 0)
+      && strcmp (src_file, print.src_file) == 0)
     {
       while (src_line > print.src_line)
 	{
@@ -341,28 +338,30 @@ print_line (source_location src_loc, const char *special_flags)
 
   if (!flag_no_line_commands)
     {
-      const struct line_map *map = linemap_lookup (line_table, src_loc);
-
-      size_t to_file_len = strlen (map->to_file);
+      const char *file_path = LOCATION_FILE (src_loc);
+      int sysp;
+      size_t to_file_len = strlen (file_path);
       unsigned char *to_file_quoted =
          (unsigned char *) alloca (to_file_len * 4 + 1);
       unsigned char *p;
 
-      print.src_line = SOURCE_LINE (map, src_loc);
-      print.src_file = map->to_file;
+      print.src_line = LOCATION_LINE (src_loc);
+      print.src_file = file_path;
 
       /* cpp_quote_string does not nul-terminate, so we have to do it
 	 ourselves.  */
       p = cpp_quote_string (to_file_quoted,
-			    (const unsigned char *) map->to_file, to_file_len);
+			    (const unsigned char *) file_path,
+			    to_file_len);
       *p = '\0';
       fprintf (print.outf, "# %u \"%s\"%s",
 	       print.src_line == 0 ? 1 : print.src_line,
 	       to_file_quoted, special_flags);
 
-      if (map->sysp == 2)
+      sysp = in_system_header_at (src_loc);
+      if (sysp == 2)
 	fputs (" 3 4", print.outf);
-      else if (map->sysp == 1)
+      else if (sysp == 1)
 	fputs (" 3", print.outf);
 
       putc ('\n', print.outf);
@@ -391,8 +390,7 @@ do_line_change (cpp_reader *pfile, const cpp_token *token,
      ought to care.  Some things do care; the fault lies with them.  */
   if (!CPP_OPTION (pfile, traditional))
     {
-      const struct line_map *map = linemap_lookup (line_table, src_loc);
-      int spaces = SOURCE_COLUMN (map, src_loc) - 2;
+      int spaces = LOCATION_COLUMN (src_loc) - 2;
       print.printed = 1;
 
       while (-- spaces >= 0)
@@ -421,6 +419,8 @@ cb_ident (cpp_reader *pfile ATTRIBUTE_UNUSED, source_location line,
 static void
 cb_define (cpp_reader *pfile, source_location line, cpp_hashnode *node)
 {
+  const struct line_map *map;
+
   maybe_print_line (line);
   fputs ("#define ", print.outf);
 
@@ -432,7 +432,10 @@ cb_define (cpp_reader *pfile, source_location line, cpp_hashnode *node)
     fputs ((const char *) NODE_NAME (node), print.outf);
 
   putc ('\n', print.outf);
-  if (linemap_lookup (line_table, line)->to_line != 0)
+  linemap_resolve_location (line_table, line,
+			    LRK_MACRO_DEFINITION_LOCATION,
+			    &map);
+  if (LINEMAP_LINE (map) != 0)
     print.src_line++;
 }
 

gcc/diagnostic.c

@@ -278,18 +278,18 @@ diagnostic_report_current_module (diagnostic_context *context)
 	  if (context->show_column)
 	    pp_verbatim (context->printer,
 			 "In file included from %s:%d:%d",
-			 map->to_file,
+			 LINEMAP_FILE (map),
 			 LAST_SOURCE_LINE (map), LAST_SOURCE_COLUMN (map));
 	  else
 	    pp_verbatim (context->printer,
 			 "In file included from %s:%d",
-			 map->to_file, LAST_SOURCE_LINE (map));
+			 LINEMAP_FILE (map), LAST_SOURCE_LINE (map));
 	  while (! MAIN_FILE_P (map))
 	    {
 	      map = INCLUDED_FROM (line_table, map);
 	      pp_verbatim (context->printer,
 			   ",\n                 from %s:%d",
-			   map->to_file, LAST_SOURCE_LINE (map));
+			   LINEMAP_FILE (map), LAST_SOURCE_LINE (map));
 	    }
 	  pp_verbatim (context->printer, ":");
 	  pp_newline (context->printer);
@@ -459,7 +459,10 @@ diagnostic_report_diagnostic (diagnostic_context *context,
 	  /* FIXME: Stupid search.  Optimize later. */
 	  for (i = context->n_classification_history - 1; i >= 0; i --)
 	    {
-	      if (context->classification_history[i].location <= location)
+	      if (linemap_location_before_p
+		  (line_table,
+		   context->classification_history[i].location,
+		   location))
 		{
 		  if (context->classification_history[i].kind == (int) DK_POP)
 		    {

gcc/fortran/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* cpp.c (print_line, cb_define): Adjust to avoid using internals
+	of struct line_map.  Use the public API instead.
+
 2011-10-17  Janus Weil  <janus@gcc.gnu.org>
 
 	PR fortran/47023

gcc/fortran/cpp.c

@@ -818,27 +818,29 @@ print_line (source_location src_loc, const char *special_flags)
 
   if (!gfc_cpp_option.no_line_commands)
     {
-      const struct line_map *map = linemap_lookup (line_table, src_loc);
-
-      size_t to_file_len = strlen (map->to_file);
-      unsigned char *to_file_quoted =
-         (unsigned char *) alloca (to_file_len * 4 + 1);
+      expanded_location loc;
+      size_t to_file_len;
+      unsigned char *to_file_quoted;
       unsigned char *p;
 
-      print.src_line = SOURCE_LINE (map, src_loc);
+      loc = expand_location (src_loc);
+      to_file_len = strlen (loc.file);
+      to_file_quoted = (unsigned char *) alloca (to_file_len * 4 + 1);
+
+      print.src_line = loc.line;
 
       /* cpp_quote_string does not nul-terminate, so we have to do it
 	 ourselves.  */
       p = cpp_quote_string (to_file_quoted,
-			    (const unsigned char *) map->to_file, to_file_len);
+			    (const unsigned char *) loc.file, to_file_len);
       *p = '\0';
       fprintf (print.outf, "# %u \"%s\"%s",
 	       print.src_line == 0 ? 1 : print.src_line,
 	       to_file_quoted, special_flags);
 
-      if (map->sysp == 2)
+      if (loc.sysp == 2)
 	fputs (" 3 4", print.outf);
-      else if (map->sysp == 1)
+      else if (loc.sysp == 1)
 	fputs (" 3", print.outf);
 
       putc ('\n', print.outf);
@@ -935,7 +937,7 @@ cb_define (cpp_reader *pfile ATTRIBUTE_UNUSED, source_location line,
     fputs ((const char *) NODE_NAME (node), print.outf);
 
   putc ('\n', print.outf);
-  if (linemap_lookup (line_table, line)->to_line != 0)
+  if (LOCATION_LINE (line) != 0)
     print.src_line++;
 }
 

gcc/input.c

@@ -42,12 +42,7 @@ expand_location (source_location loc)
       xloc.sysp = 0;
     }
   else
-    {
-      const struct line_map *map = linemap_lookup (line_table, loc);
-      xloc.file = map->to_file;
-      xloc.line = SOURCE_LINE (map, loc);
-      xloc.column = SOURCE_COLUMN (map, loc);
-      xloc.sysp = map->sysp != 0;
-    };
+    xloc = linemap_expand_location_full (line_table, loc,
+					 LRK_SPELLING_LOCATION);
   return xloc;
 }

gcc/input.h

@@ -37,20 +37,6 @@ extern GTY(()) struct line_maps *line_table;
 extern char builtins_location_check[(BUILTINS_LOCATION
 				     < RESERVED_LOCATION_COUNT) ? 1 : -1];
 
-typedef struct
-{
-  /* The name of the source file involved.  */
-  const char *file;
-
-  /* The line-location in the source file.  */
-  int line;
-
-  int column;
-
-  /* In a system header?. */
-  bool sysp;
-} expanded_location;
-
 extern expanded_location expand_location (source_location);
 
 /* Historically GCC used location_t, while cpp used source_location.
@@ -61,10 +47,12 @@ extern location_t input_location;
 
 #define LOCATION_FILE(LOC) ((expand_location (LOC)).file)
 #define LOCATION_LINE(LOC) ((expand_location (LOC)).line)
+#define LOCATION_COLUMN(LOC)((expand_location (LOC)).column)
 
 #define input_line LOCATION_LINE (input_location)
 #define input_filename LOCATION_FILE (input_location)
-#define in_system_header_at(LOC) ((expand_location (LOC)).sysp != 0)
+#define in_system_header_at(LOC) \
+  ((linemap_location_in_system_header_p (line_table, LOC)))
 #define in_system_header (in_system_header_at (input_location))
 
 #endif

gcc/java/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* jcf-parse.c (set_source_filename): Adjust to the new map API.
+
 2011-10-11  Michael Meissner  <meissner@linux.vnet.ibm.com>
 
 	* class.c (build_static_field_ref): Delete old interface with two

gcc/java/jcf-parse.c

@@ -355,7 +355,7 @@ set_source_filename (JCF *jcf, int index)
     }
       
   sfname = find_sourcefile (sfname);
-  line_table->maps[line_table->used-1].to_file = sfname;
+  ORDINARY_MAP_FILE_NAME (LINEMAPS_LAST_ORDINARY_MAP (line_table)) = sfname;
   if (current_class == main_class) main_input_filename = sfname;
 }
 

gcc/testsuite/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat.com>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* gcc.dg/cpp/pragma-diagnostic-1.c: New test.
+
 2011-10-17  Paolo Carlini  <paolo.carlini@oracle.com>
 
 	PR c++/48489

gcc/testsuite/gcc.dg/cpp/pragma-diagnostic-1.c

+/*
+  { dg-options "-Wuninitialized" }
+  { dg-do compile }
+*/
+
+void f (unsigned);
+
+#define CODE_WITH_WARNING \
+  int a;		  \
+  f (a)
+
+#pragma GCC diagnostic ignored "-Wuninitialized"
+
+void
+g (void)
+{
+  CODE_WITH_WARNING;
+}
+
+#pragma GCC diagnostic push
+
+#pragma GCC diagnostic error "-Wuninitialized"
+
+void
+h (void)
+{
+  CODE_WITH_WARNING;		/* { dg-error "uninitialized" } */
+}
+
+/*
+  { dg-message "some warnings being treated as errors" "" {target *-*-*} 0 }
+*/

libcpp/ChangeLog

+2011-10-15  Tom Tromey  <tromey@redhat>
+	    Dodji Seketeli  <dodji@redhat.com>
+
+	* include/line-map.h (enum lc_reason)<LC_ENTER_MACRO>: New enum
+	member.
+	(MAX_SOURCE_LOCATION): New constant.
+	(struct line_map_ordinary, struct line_map_macro): New structs.
+    	(struct line_map): Turn this into a union of the two above.  Add
+	comments.
+	(struct maps_info): New struct.
+	(struct line_maps)<info_ordinary, info_macro>: Two new fields.
+	These now carry the map information that was previously scattered
+	in struct line_maps.
+	(struct map_info::allocated): Fix comment.
+	(MAP_START_LOCATION, ORDINARY_MAP_FILE_NAME)
+	(ORDINARY_MAP_STARTING_LINE_NUMBER)
+	(ORDINARY_MAP_INCLUDER_FILE_INDEX)
+	(ORDINARY_MAP_IN_SYSTEM_HEADER_P)
+	(ORDINARY_MAP_NUMBER_OF_COLUMN_BITS, MACRO_MAP_MACRO)
+	(MACRO_MAP_NUM_MACRO_TOKENS MACRO_MAP_LOCATIONS)
+	(MACRO_MAP_EXPANSION_POINT_LOCATION)
+	(LOCATION_POSSIBLY_IN_MACRO_MAP_P, LINEMAPS_MAP_INFO)
+	(LINEMAPS_MAPS, LINEMAPS_ALLOCATE, LINEMAPS_USED, LINEMAPS_CACHE)
+	(LINEMAPS_LAST_MAP, LINEMAPS_LAST_ALLOCATED_MAP)
+	(LINEMAPS_ORDINARY_MAPS, LINEMAPS_ORDINARY_ALLOCATED)
+	(LINEMAPS_ORDINARY_USED, LINEMAPS_ORDINARY_CACHE)
+	(LINEMAPS_LAST_ORDINARY_MAP, LINEMAPS_LAST_ALLOCATED_ORDINARY_MAP)
+	(LINEMAPS_MACRO_MAPS, LINEMAPS_MACRO_ALLOCATED)
+	(LINEMAPS_MACRO_USED, LINEMAPS_MACRO_CACHE)
+	(LINEMAPS_LAST_MACRO_MAP, LINEMAPS_LAST_ALLOCATED_MACRO_MAP)
+	(LINEMAPS_MAP_AT, LINEMAPS_ORDINARY_MAP_AT)
+	(LINEMAPS_MACRO_MAP_AT): New accessors for ordinary and macro map
+	information.
+	(linemap_check_ordinary, linemap_assert)
+	(linemap_location_before_p): New macros.
+	(linemap_position_for_line_and_column)
+	(linemap_tracks_macro_expansion_locs_p, linemap_add_macro_token)
+	(linemap_macro_expansion_map_p)
+	(linemap_macro_map_loc_to_def_point)
+	(linemap_macro_map_loc_unwind_once)
+	(linemap_macro_map_loc_to_exp_point, linemap_step_out_once)
+	(linemap_get_source_line linemap_get_source_column)
+	(linemap_map_get_macro_name, linemap_get_file_path)
+	(linemap_location_in_system_header_p)
+	(linemap_location_from_macro_expansion_p): Declare new functions.
+	(SOURCE_LINE, SOURCE_COLUMN, LAST_SOURCE_LINE_LOCATION)
+	(LINEMAP_FILE, LINEMAP_LINE, LINEMAP_SYSP): Assert that this
+	accessors act on ordinary maps only.
+	(INCLUDED_FROM): Return NULL for main files; use the new
+	accessors.
+	(LINEMAP_POSITION_FOR_COLUMN): Use the new accessors.
+	(struct expanded_location): Move here from gcc/input.h
+	(linemap_resolve_location, linemap_expand_location)
+	(linemap_expand_location_full): Declare new functions.
+	* line-map.c: Include cpplib.h, internal.h
+	(linemap_enter_macro, linemap_add_macro_token)
+	(linemap_get_expansion_line, linemap_get_expansion_filename): New
+	functions that are private to libcpp.
+	(linemap_assert): New macro.
+	(linemap_macro_loc_to_exp_point, linemap_macro_loc_to_exp_point)
+	(linemap_macro_loc_unwind, linemap_macro_map_loc_to_def_point)
+	(linemap_macro_map_loc_unwind_toward_spelling)
+	(linemap_macro_map_loc_to_exp_point)
+	(first_map_in_common_1, first_map_in_common): New static
+	functions.
+	(new_linemap): Define new static functions.  Extracted and
+	enhanced from ...
+	(linemap_add): ... here.  Use linemap_assert in lieu of abort
+	previously.
+	(linemap_tracks_macro_expansion_locs_p)
+	(linemap_add_macro_token, linemap_macro_expansion_map_p)
+	(linemap_check_ordinary, linemap_macro_map_loc_to_exp_point)
+	(linemap_macro_map_loc_to_def_point)
+	(linemap_macro_map_loc_unwind_once)
+	(linemap_step_out_once, linemap_map_get_index)
+	(linemap_get_source_line,linemap_get_source_column)
+	(linemap_get_file_path, linemap_map_get_macro_name)
+	(linemap_location_in_system_header_p)
+	(linemap_location_originated_from_system_header_p)
+	(linemap_location_from_macro_expansion_p)
+	(linemap_tracks_macro_expansion_locs_p)
+	(linemap_resolve_location, linemap_expand_location)
+	(linemap_expand_location_full)
+	(linemap_tracks_macro_expansion_locs_p)
+	(linemap_position_for_line_and_column, linemap_compare_locations):
+	Define new public functions.
+	(linemap_init): Initialize ordinary and macro maps information in
+	the map set.
+	(linemap_check_files_exited): Use the new accessors.
+	(linemap_free): Remove this dead code.
+	(linemap_line_start): Assert this uses an ordinary map.  Adjust to
+	use the new ordinary map accessors and data structures.  Don't
+	overflow past the lowest possible macro token's location.
+	(linemap_position_for_column): Assert the ordinary maps of the map
+	set are really ordinary.  Use ordinary map accessors.
+	(linemap_lookup): Keep the same logic but generalize to allow
+	lookup of both ordinary and macro maps.  Do not crash when called
+	with an empty line table.
+	* directives-only.c (_cpp_preprocess_dir_only): Adjust to use the
+	new API of line-map.h.
+	* directives.c (start_directive, do_line, do_linemarker)
+	(do_linemarker): Likewise.
+	* files.c (_cpp_find_file, _cpp_stack_include, open_file_failed)
+	(make_cpp_dir, cpp_make_system_header): Likewise.
+	* init.c (cpp_read_main_file): Likewise.
+	* internal.h (CPP_INCREMENT_LINE): Likewise.
+	(linemap_enter_macro, linemap_add_macro_token)
+	(linemap_get_expansion_line, linemap_get_expansion_filename): New
+	functions private to libcpp.
+	* lex.c (_cpp_process_line_notes, _cpp_skip_block_comment)
+	(skip_line_comment, skip_whitespace, lex_raw_string)
+	(_cpp_lex_direct): Likewise.
+	* macro.c (_cpp_builtin_macro_text): Likewise.
+	(_cpp_aligned_alloc): Initialize the new name member of the macro.
+	* traditional.c (copy_comment, _cpp_scan_out_logical_line):
+	Likewise.
+	* errors.c (cpp_diagnostic): Adjust to new linemap API.
+
 2011-08-28  Dodji Seketeli  <dodji@redhat.com>
 
 	* line-map.c (linemap_add): Assert that reason must not be

libcpp/directives.c

@@ -884,14 +884,14 @@ static void
 do_line (cpp_reader *pfile)
 {
   const struct line_maps *line_table = pfile->line_table;
-  const struct line_map *map = &line_table->maps[line_table->used - 1];
+  const struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (line_table);
 
   /* skip_rest_of_line() may cause line table to be realloc()ed so note down
      sysp right now.  */
 
-  unsigned char map_sysp = map->sysp;
+  unsigned char map_sysp = ORDINARY_MAP_IN_SYSTEM_HEADER_P (map);
   const cpp_token *token;
-  const char *new_file = map->to_file;
+  const char *new_file = ORDINARY_MAP_FILE_NAME (map);
   linenum_type new_lineno;
 
   /* C99 raised the minimum limit on #line numbers.  */
@@ -946,11 +946,11 @@ static void
 do_linemarker (cpp_reader *pfile)
 {
   const struct line_maps *line_table = pfile->line_table;
-  const struct line_map *map = &line_table->maps[line_table->used - 1];
+  const struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (line_table);
   const cpp_token *token;
-  const char *new_file = map->to_file;
+  const char *new_file = ORDINARY_MAP_FILE_NAME (map);
   linenum_type new_lineno;
-  unsigned int new_sysp = map->sysp;
+  unsigned int new_sysp = ORDINARY_MAP_IN_SYSTEM_HEADER_P (map);
   enum lc_reason reason = LC_RENAME_VERBATIM;
   int flag;
   bool wrapped;
@@ -1038,7 +1038,9 @@ _cpp_do_file_change (cpp_reader *pfile, enum lc_reason reason,
   const struct line_map *map = linemap_add (pfile->line_table, reason, sysp,
 					    to_file, file_line);
   if (map != NULL)
-    linemap_line_start (pfile->line_table, map->to_line, 127);
+    linemap_line_start (pfile->line_table,
+			ORDINARY_MAP_STARTING_LINE_NUMBER (map),
+			127);
 
   if (pfile->cb.file_change)
     pfile->cb.file_change (pfile, map);

libcpp/files.c

@@ -1220,13 +1220,12 @@ cpp_make_system_header (cpp_reader *pfile, int syshdr, int externc)
 {
   int flags = 0;
   const struct line_maps *line_table = pfile->line_table;
-  const struct line_map *map = &line_table->maps[line_table->used-1];
-
+  const struct line_map *map = LINEMAPS_LAST_ORDINARY_MAP (line_table);
   /* 1 = system header, 2 = system header to be treated as C.  */
   if (syshdr)
     flags = 1 + (externc != 0);
   pfile->buffer->sysp = flags;
-  _cpp_do_file_change (pfile, LC_RENAME, map->to_file,
+  _cpp_do_file_change (pfile, LC_RENAME, ORDINARY_MAP_FILE_NAME (map),
 		       SOURCE_LINE (map, pfile->line_table->highest_line), flags);
 }
 

libcpp/init.c

@@ -586,7 +586,9 @@ cpp_read_main_file (cpp_reader *pfile, const char *fname)
   if (CPP_OPTION (pfile, preprocessed))
     {
       read_original_filename (pfile);
-      fname = pfile->line_table->maps[pfile->line_table->used-1].to_file;
+      fname =
+	ORDINARY_MAP_FILE_NAME
+	((LINEMAPS_LAST_ORDINARY_MAP (pfile->line_table)));
     }
   return fname;
 }

libcpp/internal.h

@@ -67,7 +67,8 @@ struct cset_converter
 
 #define CPP_INCREMENT_LINE(PFILE, COLS_HINT) do { \
     const struct line_maps *line_table = PFILE->line_table; \
-    const struct line_map *map = &line_table->maps[line_table->used-1]; \
+    const struct line_map *map = \
+      LINEMAPS_LAST_ORDINARY_MAP (line_table); \
     linenum_type line = SOURCE_LINE (map, line_table->highest_line); \
     linemap_line_start (PFILE->line_table, line + 1, COLS_HINT); \
   } while (0)
@@ -739,6 +740,76 @@ ufputs (const unsigned char *s, FILE *f)
   return fputs ((const char *)s, f);
 }
 
+  /* In line-map.c.  */
+
+/* Create a macro map.  A macro map encodes source locations of tokens
+   that are part of a macro replacement-list, at a macro expansion
+   point. See the extensive comments of struct line_map and struct
+   line_map_macro, in line-map.h.
+
+   This map shall be created when the macro is expanded. The map
+   encodes the source location of the expansion point of the macro as
+   well as the "original" source location of each token that is part
+   of the macro replacement-list. If a macro is defined but never
+   expanded, it has no macro map.  SET is the set of maps the macro
+   map should be part of.  MACRO_NODE is the macro which the new macro
+   map should encode source locations for.  EXPANSION is the location
+   of the expansion point of MACRO. For function-like macros
+   invocations, it's best to make it point to the closing parenthesis
+   of the macro, rather than the the location of the first character
+   of the macro.  NUM_TOKENS is the number of tokens that are part of
+   the replacement-list of MACRO.  */
+const struct line_map *linemap_enter_macro (struct line_maps *,
+					    struct cpp_hashnode*,
+					    source_location,
+					    unsigned int);
+
+/* Create and return a virtual location for a token that is part of a
+   macro expansion-list at a macro expansion point.  See the comment
+   inside struct line_map_macro to see what an expansion-list exactly
+   is.
+
+   A call to this function must come after a call to
+   linemap_enter_macro.
+
+   MAP is the map into which the source location is created.  TOKEN_NO
+   is the index of the token in the macro replacement-list, starting
+   at number 0.
+
+   ORIG_LOC is the location of the token outside of this macro
+   expansion.  If the token comes originally from the macro
+   definition, it is the locus in the macro definition; otherwise it
+   is a location in the context of the caller of this macro expansion
+   (which is a virtual location or a source location if the caller is
+   itself a macro expansion or not).
+
+   MACRO_DEFINITION_LOC is the location in the macro definition,
+   either of the token itself or of a macro parameter that it
+   replaces.  */
+source_location linemap_add_macro_token (const struct line_map *,
+					 unsigned int,
+					 source_location,
+					 source_location);
+
+/* Return the source line number corresponding to source location
+   LOCATION.  SET is the line map set LOCATION comes from.  If
+   LOCATION is the location of token that is part of the
+   expansion-list of a macro expansion return the line number of the
+   macro expansion point.  */
+int linemap_get_expansion_line (struct line_maps *,
+				source_location);
+
+/* Return the path of the file corresponding to source code location
+   LOCATION.
+
+   If LOCATION is the location of a token that is part of the
+   replacement-list of a macro expansion return the file path of the
+   macro expansion point.
+
+   SET is the line map set LOCATION comes from.  */
+const char* linemap_get_expansion_filename (struct line_maps *,
+					    source_location);
+
 #ifdef __cplusplus
 }
 #endif

libcpp/macro.c

@@ -171,13 +171,17 @@ _cpp_builtin_macro_text (cpp_reader *pfile, cpp_hashnode *node)
 	unsigned int len;
 	const char *name;
 	uchar *buf;
-	map = linemap_lookup (pfile->line_table, pfile->line_table->highest_line);
-
-	if (node->value.builtin == BT_BASE_FILE)
-	  while (! MAIN_FILE_P (map))
-	    map = INCLUDED_FROM (pfile->line_table, map);
-
-	name = map->to_file;
+	
+	if (node->value.builtin == BT_FILE)
+	  name = linemap_get_expansion_filename (pfile->line_table,
+						 pfile->line_table->highest_line);
+	else
+	  {
+	    map = linemap_lookup (pfile->line_table, pfile->line_table->highest_line);
+	    while (! MAIN_FILE_P (map))
+	      map = INCLUDED_FROM (pfile->line_table, map);
+	    name = ORDINARY_MAP_FILE_NAME (map);
+	  }
 	len = strlen (name);
 	buf = _cpp_unaligned_alloc (pfile, len * 2 + 3);
 	result = buf;
@@ -196,14 +200,14 @@ _cpp_builtin_macro_text (cpp_reader *pfile, cpp_hashnode *node)
       break;
 
     case BT_SPECLINE:
-      map = &pfile->line_table->maps[pfile->line_table->used-1];
+      map = LINEMAPS_LAST_ORDINARY_MAP (pfile->line_table);
       /* If __LINE__ is embedded in a macro, it must expand to the
 	 line of the macro's invocation, not its definition.
 	 Otherwise things like assert() will not work properly.  */
-      number = SOURCE_LINE (map, 
-			    CPP_OPTION (pfile, traditional) 
-			    ? pfile->line_table->highest_line
-			    : pfile->cur_token[-1].src_loc);
+      number = linemap_get_expansion_line (pfile->line_table,
+					   CPP_OPTION (pfile, traditional)
+					   ? pfile->line_table->highest_line
+					   : pfile->cur_token[-1].src_loc);
       break;
 
       /* __STDC__ has the value 1 under normal circumstances.