Support embedding the driver in-process within libgccjit

gcc/ChangeLog:
	* gcc-main.c (main): Add params to driver ctor.
	* gcc.c (class env_manager): New.
	(env): New global.
	(env_manager::init): New.
	(env_manager::get): New.
	(env_manager::xput): New.
	(env_manager::restore): New.
	Poison getenv and putenv.
	(DEFAULT_TARGET_SYSTEM_ROOT): New.
	(target_system_root): Update initialization to use
	DEFAULT_TARGET_SYSTEM_ROOT.
	(struct spec_list): Add field "default_ptr".
	(INIT_STATIC_SPEC): Initialize new field "default_ptr".
	(init_spec): Likewise.
	(set_spec): Clear field "default_ptr".
	(read_specs): Free "spec" and "buffer".
	(xputenv): Reimplement in terms of env_manager.
	(process_command): Replace ::getenv calls with calls to the
	env_manager singleton.
	(process_brace_body): Free string in three places.
	(driver::driver): New.
	(driver::~driver): New.
	(used_arg): Convert from a function to...
	(class used_arg_t): ...this class, and...
	(used_arg): ...this new global instance.
	(used_arg_t::finalize): New function.
	(getenv_spec_function): Add "const" to local "value".  Replace
	::getenv call with call to the env_manager singleton.
	(path_prefix_reset): New function.
	(driver::finalize): New function.
	* gcc.h (driver::driver): New.
	(driver::~driver): New.
	(driver::finalize): New.

gcc/jit/ChangeLog:
	* docs/cp/topics/contexts.rst
	(gccjit::context::set_bool_use_external_driver): New.
	* docs/internals/test-hello-world.exe.log.txt: Update.
	* docs/topics/compatibility.rst (LIBGCCJIT_ABI_5): New.
	* docs/topics/contexts.rst
	(gcc_jit_context_set_bool_use_external_driver): New.
	* jit-common.h (enum inner_bool_option): Add
	INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER.
	* jit-playback.c (gcc_driver_name): New global.
	(gcc:jit::playback::context::invoke_driver): Split out second
	half into...
	(gcc::jit::playback::context::invoke_embedded_driver): ...this new
	function, and...
	(gcc::jit::playback::context::invoke_external_driver): ...this new
	function.
	* jit-playback.h
	(gcc::jit::playback::context::get_inner_bool_option): New.
	(gcc::jit::playback::context::invoke_embedded_driver): New.
	(gcc::jit::playback::context::invoke_external_driver): New.
	* jit-recording.c (inner_bool_option_reproducer_strings):
	Add entry for INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER.
	* libgccjit++.h
	(gccjit::context::set_bool_use_external_driver): New.
	* libgccjit.c (gcc_jit_context_set_bool_use_external_driver): New.
	* libgccjit.h (gcc_jit_context_set_bool_use_external_driver): New.
	(LIBGCCJIT_HAVE_gcc_jit_context_set_bool_use_external_driver):
	New.
	* libgccjit.map (LIBGCCJIT_ABI_5): New.
	* notes.txt: Show invocation of embedded copy of driver.
	* docs/internals/test-hello-world.exe.log.txt: Update

gcc/testsuite/ChangeLog:
	* jit.dg/test-error-pr63969-missing-driver.c: Add call to
	gcc_jit_context_set_bool_use_external_driver.

From-SVN: r227188

gcc/ChangeLog

+2015-08-25  David Malcolm  <dmalcolm@redhat.com>
+
+	* gcc-main.c (main): Add params to driver ctor.
+	* gcc.c (class env_manager): New.
+	(env): New global.
+	(env_manager::init): New.
+	(env_manager::get): New.
+	(env_manager::xput): New.
+	(env_manager::restore): New.
+	Poison getenv and putenv.
+	(DEFAULT_TARGET_SYSTEM_ROOT): New.
+	(target_system_root): Update initialization to use
+	DEFAULT_TARGET_SYSTEM_ROOT.
+	(struct spec_list): Add field "default_ptr".
+	(INIT_STATIC_SPEC): Initialize new field "default_ptr".
+	(init_spec): Likewise.
+	(set_spec): Clear field "default_ptr".
+	(read_specs): Free "spec" and "buffer".
+	(xputenv): Reimplement in terms of env_manager.
+	(process_command): Replace ::getenv calls with calls to the
+	env_manager singleton.
+	(process_brace_body): Free string in three places.
+	(driver::driver): New.
+	(driver::~driver): New.
+	(used_arg): Convert from a function to...
+	(class used_arg_t): ...this class, and...
+	(used_arg): ...this new global instance.
+	(used_arg_t::finalize): New function.
+	(getenv_spec_function): Add "const" to local "value".  Replace
+	::getenv call with call to the env_manager singleton.
+	(path_prefix_reset): New function.
+	(driver::finalize): New function.
+	* gcc.h (driver::driver): New.
+	(driver::~driver): New.
+	(driver::finalize): New.
+
 2015-08-25  Nathan Sidwell  <nathan@acm.org>
 
 	* optabs.c (emit_indirect_jump): Don't try an emit a jump if the

gcc/gcc-main.c

@@ -40,7 +40,8 @@ extern int main (int, char **);
 int
 main (int argc, char **argv)
 {
-  driver d;
+  driver d (false, /* can_finalize */
+	    false); /* debug */
 
   return d.main (argc, argv);
 }

gcc/gcc.h

@@ -30,7 +30,10 @@ along with GCC; see the file COPYING3.  If not see
 class driver
 {
  public:
+  driver (bool can_finalize, bool debug);
+  ~driver ();
   int main (int argc, char **argv);
+  void finalize ();
 
  private:
   void set_progname (const char *argv0) const;

gcc/jit/ChangeLog

+2015-08-25  David Malcolm  <dmalcolm@redhat.com>
+
+	* docs/cp/topics/contexts.rst
+	(gccjit::context::set_bool_use_external_driver): New.
+	* docs/internals/test-hello-world.exe.log.txt: Update.
+	* docs/topics/compatibility.rst (LIBGCCJIT_ABI_5): New.
+	* docs/topics/contexts.rst
+	(gcc_jit_context_set_bool_use_external_driver): New.
+	* jit-common.h (enum inner_bool_option): Add
+	INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER.
+	* jit-playback.c (gcc_driver_name): New global.
+	(gcc:jit::playback::context::invoke_driver): Split out second
+	half into...
+	(gcc::jit::playback::context::invoke_embedded_driver): ...this new
+	function, and...
+	(gcc::jit::playback::context::invoke_external_driver): ...this new
+	function.
+	* jit-playback.h
+	(gcc::jit::playback::context::get_inner_bool_option): New.
+	(gcc::jit::playback::context::invoke_embedded_driver): New.
+	(gcc::jit::playback::context::invoke_external_driver): New.
+	* jit-recording.c (inner_bool_option_reproducer_strings):
+	Add entry for INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER.
+	* libgccjit++.h
+	(gccjit::context::set_bool_use_external_driver): New.
+	* libgccjit.c (gcc_jit_context_set_bool_use_external_driver): New.
+	* libgccjit.h (gcc_jit_context_set_bool_use_external_driver): New.
+	(LIBGCCJIT_HAVE_gcc_jit_context_set_bool_use_external_driver):
+	New.
+	* libgccjit.map (LIBGCCJIT_ABI_5): New.
+	* notes.txt: Show invocation of embedded copy of driver.
+	* docs/internals/test-hello-world.exe.log.txt: Update
+
 2015-08-13  David Malcolm  <dmalcolm@redhat.com>
 
 	* jit-playback.c (invoke_driver): On OS X, add

gcc/jit/docs/cp/topics/contexts.rst

@@ -201,6 +201,26 @@ Boolean options
 
       #ifdef LIBGCCJIT_HAVE_gcc_jit_context_set_bool_allow_unreachable_blocks
 
+.. function:: void \
+              gccjit::context::set_bool_use_external_driver (int bool_value)
+
+   libgccjit internally generates assembler, and uses "driver" code
+   for converting it to other formats (e.g. shared libraries).
+
+   By default, libgccjit will use an embedded copy of the driver
+   code.
+
+   This option can be used to instead invoke an external driver executable
+   as a subprocess; it is a thin wrapper around the C API
+   :c:func:`gcc_jit_context_set_bool_use_external_driver`.
+
+   This entrypoint was added in :ref:`LIBGCCJIT_ABI_5`; you can test for
+   its presence using
+
+   .. code-block:: c
+
+      #ifdef LIBGCCJIT_HAVE_gcc_jit_context_set_bool_use_external_driver
+
 Integer options
 ***************
 

gcc/jit/docs/internals/test-hello-world.exe.log.txt

-JIT: libgccjit (GCC) version 6.0.0 20150723 (experimental) (x86_64-unknown-linux-gnu)
+JIT: libgccjit (GCC) version 6.0.0 20150803 (experimental) (x86_64-pc-linux-gnu)
 JIT:	compiled by GNU C version 4.8.3 20140911 (Red Hat 4.8.3-7), GMP version 5.1.2, MPFR version 3.1.2, MPC version 1.0.1
 JIT: entering: gcc_jit_context_set_str_option
 JIT:  GCC_JIT_STR_OPTION_PROGNAME: "./test-hello-world.c.exe"
@@ -65,6 +65,7 @@ JIT:   GCC_JIT_BOOL_OPTION_DUMP_EVERYTHING: false
 JIT:   GCC_JIT_BOOL_OPTION_SELFCHECK_GC: true
 JIT:   GCC_JIT_BOOL_OPTION_KEEP_INTERMEDIATES: false
 JIT:   gcc_jit_context_set_bool_allow_unreachable_blocks: false
+JIT:   gcc_jit_context_set_bool_use_external_driver: false
 JIT:   entering: void gcc::jit::recording::context::validate()
 JIT:   exiting: void gcc::jit::recording::context::validate()
 JIT:   entering: gcc::jit::playback::context::context(gcc::jit::recording::context*)
@@ -133,7 +134,8 @@ JIT:       argv[3]: /tmp/libgccjit-CKq1M9/fake.s
 JIT:       argv[4]: -o
 JIT:       argv[5]: /tmp/libgccjit-CKq1M9/fake.so
 JIT:       argv[6]: -fno-use-linker-plugin
-JIT:       argv[7]: (null)
+JIT:       entering: void gcc::jit::playback::context::invoke_embedded_driver(const vec<char*>*)
+JIT:       exiting: void gcc::jit::playback::context::invoke_embedded_driver(const vec<char*>*)
 JIT:      exiting: void gcc::jit::playback::context::invoke_driver(const char*, const char*, const char*, timevar_id_t, bool, bool)
 JIT:     exiting: void gcc::jit::playback::context::convert_to_dso(const char*)
 JIT:     entering: gcc::jit::result* gcc::jit::playback::context::dlopen_built_dso()

gcc/jit/docs/topics/compatibility.rst

@@ -128,3 +128,10 @@ entrypoints:
   * :func:`gcc_jit_timer_pop`
 
   * :func:`gcc_jit_timer_print`
+
+.. _LIBGCCJIT_ABI_5:
+
+``LIBGCCJIT_ABI_5``
+-------------------
+``LIBGCCJIT_ABI_5`` covers the addition of
+:func:`gcc_jit_context_set_bool_use_external_driver`

gcc/jit/docs/topics/contexts.rst

@@ -469,6 +469,26 @@ Boolean options
 
       #ifdef LIBGCCJIT_HAVE_gcc_jit_context_set_bool_allow_unreachable_blocks
 
+.. function:: void \
+              gcc_jit_context_set_bool_use_external_driver (gcc_jit_context *ctxt, \
+                                                            int bool_value)
+
+   libgccjit internally generates assembler, and uses "driver" code
+   for converting it to other formats (e.g. shared libraries).
+
+   By default, libgccjit will use an embedded copy of the driver
+   code.
+
+   This option can be used to instead invoke an external driver executable
+   as a subprocess.
+
+   This entrypoint was added in :ref:`LIBGCCJIT_ABI_5`; you can test for
+   its presence using
+
+   .. code-block:: c
+
+      #ifdef LIBGCCJIT_HAVE_gcc_jit_context_set_bool_use_external_driver
+
 Integer options
 ***************
 

gcc/jit/jit-common.h

@@ -191,6 +191,7 @@ private:
 enum inner_bool_option
 {
   INNER_BOOL_OPTION_ALLOW_UNREACHABLE_BLOCKS,
+  INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER,
 
   NUM_INNER_BOOL_OPTIONS
 };

gcc/jit/jit-playback.c

@@ -2373,6 +2373,8 @@ convert_to_dso (const char *ctxt_progname)
 		 true);/* bool run_linker */
 }
 
+static const char * const gcc_driver_name = GCC_DRIVER_NAME;
+
 void
 playback::context::
 invoke_driver (const char *ctxt_progname,
@@ -2383,15 +2385,15 @@ invoke_driver (const char *ctxt_progname,
 	       bool run_linker)
 {
   JIT_LOG_SCOPE (get_logger ());
+
+  bool embedded_driver
+    = !get_inner_bool_option (INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER);
+
   /* Currently this lumps together both assembling and linking into
      TV_ASSEMBLE.  */
   auto_timevar assemble_timevar (get_timer (), tv_id);
-  const char *errmsg;
   auto_argvec argvec;
 #define ADD_ARG(arg) argvec.safe_push (xstrdup (arg))
-  int exit_status = 0;
-  int err = 0;
-  const char *gcc_driver_name = GCC_DRIVER_NAME;
 
   ADD_ARG (gcc_driver_name);
 
@@ -2425,8 +2427,10 @@ invoke_driver (const char *ctxt_progname,
   ADD_ARG ("-Wl,-undefined,dynamic_lookup");
 #endif
 
-  /* pex argv arrays are NULL-terminated.  */
-  argvec.safe_push (NULL);
+  if (0)
+    ADD_ARG ("-v");
+
+#undef ADD_ARG
 
   /* pex_one's error-handling requires pname to be non-NULL.  */
   gcc_assert (ctxt_progname);
@@ -2435,9 +2439,42 @@ invoke_driver (const char *ctxt_progname,
     for (unsigned i = 0; i < argvec.length (); i++)
       get_logger ()->log ("argv[%i]: %s", i, argvec[i]);
 
+  if (embedded_driver)
+    invoke_embedded_driver (&argvec);
+  else
+    invoke_external_driver (ctxt_progname, &argvec);
+}
+
+void
+playback::context::
+invoke_embedded_driver (const vec <char *> *argvec)
+{
+  JIT_LOG_SCOPE (get_logger ());
+  driver d (true, /* can_finalize */
+	    false); /* debug */
+  int result = d.main (argvec->length (),
+		       const_cast <char **> (argvec->address ()));
+  d.finalize ();
+  if (result)
+    add_error (NULL, "error invoking gcc driver");
+}
+
+void
+playback::context::
+invoke_external_driver (const char *ctxt_progname,
+			vec <char *> *argvec)
+{
+  JIT_LOG_SCOPE (get_logger ());
+  const char *errmsg;
+  int exit_status = 0;
+  int err = 0;
+
+  /* pex argv arrays are NULL-terminated.  */
+  argvec->safe_push (NULL);
+
   errmsg = pex_one (PEX_SEARCH, /* int flags, */
 		    gcc_driver_name,
-		    const_cast <char *const *> (argvec.address ()),
+		    const_cast <char *const *> (argvec->address ()),
 		    ctxt_progname, /* const char *pname */
 		    NULL, /* const char *outname */
 		    NULL, /* const char *errname */
@@ -2464,7 +2501,6 @@ invoke_driver (const char *ctxt_progname,
 		 getenv ("PATH"));
       return;
     }
-#undef ADD_ARG
 }
 
 /* Extract the target-specific MULTILIB_DEFAULTS to

gcc/jit/jit-playback.h

@@ -177,6 +177,12 @@ public:
     return m_recording_ctxt->get_bool_option (opt);
   }
 
+  int
+  get_inner_bool_option (enum inner_bool_option opt) const
+  {
+    return m_recording_ctxt->get_inner_bool_option (opt);
+  }
+
   builtins_manager *get_builtins_manager () const
   {
     return m_recording_ctxt->get_builtins_manager ();
@@ -280,6 +286,14 @@ protected:
   result *
   dlopen_built_dso ();
 
+ private:
+  void
+  invoke_embedded_driver (const vec <char *> *argvec);
+
+  void
+  invoke_external_driver (const char *ctxt_progname,
+			  vec <char *> *argvec);
+
 private:
   ::gcc::jit::recording::context *m_recording_ctxt;
 

gcc/jit/jit-recording.c

@@ -1452,7 +1452,8 @@ static const char * const
 
 static const char * const
  inner_bool_option_reproducer_strings[NUM_INNER_BOOL_OPTIONS] = {
-  "gcc_jit_context_set_bool_allow_unreachable_blocks"
+  "gcc_jit_context_set_bool_allow_unreachable_blocks",
+  "gcc_jit_context_set_bool_use_external_driver"
 };
 
 /* Write the current value of all options to the log file (if any).  */

gcc/jit/libgccjit++.h

@@ -124,6 +124,7 @@ namespace gccjit
 			  int value);
 
     void set_bool_allow_unreachable_blocks (int bool_value);
+    void set_bool_use_external_driver (int bool_value);
 
     void add_command_line_option (const char *optname);
 
@@ -666,6 +667,13 @@ context::set_bool_allow_unreachable_blocks (int bool_value)
 }
 
 inline void
+context::set_bool_use_external_driver (int bool_value)
+{
+  gcc_jit_context_set_bool_use_external_driver (m_inner_ctxt,
+						bool_value);
+}
+
+inline void
 context::add_command_line_option (const char *optname)
 {
   gcc_jit_context_add_command_line_option (m_inner_ctxt, optname);

gcc/jit/libgccjit.c

@@ -2611,6 +2611,23 @@ gcc_jit_context_set_bool_allow_unreachable_blocks (gcc_jit_context *ctxt,
 /* Public entrypoint.  See description in libgccjit.h.
 
    After error-checking, the real work is done by the
+   gcc::jit::recording::context::set_inner_bool_option method in
+   jit-recording.c.  */
+
+extern void
+gcc_jit_context_set_bool_use_external_driver (gcc_jit_context *ctxt,
+					      int bool_value)
+{
+  RETURN_IF_FAIL (ctxt, NULL, NULL, "NULL context");
+  JIT_LOG_FUNC (ctxt->get_logger ());
+  ctxt->set_inner_bool_option (
+    gcc::jit::INNER_BOOL_OPTION_USE_EXTERNAL_DRIVER,
+    bool_value);
+}
+
+/* Public entrypoint.  See description in libgccjit.h.
+
+   After error-checking, the real work is done by the
    gcc::jit::recording::context::add_command_line_option method in
    jit-recording.c.  */
 

gcc/jit/libgccjit.h

@@ -278,6 +278,30 @@ gcc_jit_context_set_bool_allow_unreachable_blocks (gcc_jit_context *ctxt,
    tested for with #ifdef.  */
 #define LIBGCCJIT_HAVE_gcc_jit_context_set_bool_allow_unreachable_blocks
 
+/* Implementation detail:
+   libgccjit internally generates assembler, and uses "driver" code
+   for converting it to other formats (e.g. shared libraries).
+
+   By default, libgccjit will use an embedded copy of the driver
+   code.
+
+   This option can be used to instead invoke an external driver executable
+   as a subprocess.
+
+   This entrypoint was added in LIBGCCJIT_ABI_5; you can test for
+   its presence using
+     #ifdef LIBGCCJIT_HAVE_gcc_jit_context_set_bool_use_external_driver
+*/
+
+extern void
+gcc_jit_context_set_bool_use_external_driver (gcc_jit_context *ctxt,
+					      int bool_value);
+
+/* Pre-canned feature macro to indicate the presence of
+   gcc_jit_context_set_bool_use_external_driver.  This can be
+   tested for with #ifdef.  */
+#define LIBGCCJIT_HAVE_gcc_jit_context_set_bool_use_external_driver
+
 /* Add an arbitrary gcc command-line option to the context.
    The context takes a copy of the string, so the
    (const char *) optname is not needed anymore after the call

gcc/jit/libgccjit.map

@@ -140,3 +140,8 @@ LIBGCCJIT_ABI_4 {
     gcc_jit_timer_pop;
     gcc_jit_timer_print;
 };
+
+LIBGCCJIT_ABI_5 {
+  global:
+    gcc_jit_context_set_bool_use_external_driver;
+} LIBGCCJIT_ABI_4;

gcc/jit/notes.txt

@@ -78,7 +78,13 @@ Client Code   . Generated .            libgccjit.so
               .           .      │   .               .
               .           .      │   (assuming an in-memory compile):
               .           .      │   .               .
-              .           .      │   . Convert assembler to DSO ("fake.so")
+              .           .      --> Convert assembler to DSO, via embedded
+              .           .          copy of driver:
+              .           .           driver::main ()
+              .           .             invocation of "as"
+              .           .             invocation of "ld"
+              .           .           driver::finalize ()
+              .           .      <----
               .           .      │   .               .
               .           .      │   . Load DSO (dlopen "fake.so")
               .           .      │   .               .

gcc/testsuite/ChangeLog

+2015-08-25  David Malcolm  <dmalcolm@redhat.com>
+
+	* jit.dg/test-error-pr63969-missing-driver.c: Add call to
+	gcc_jit_context_set_bool_use_external_driver.
+
 2015-08-25  Nathan Sidwell  <nathan@acm.org>
 
 	* gcc.c-torture/execute/builtins/20010124-1.x: New.

gcc/testsuite/jit.dg/test-error-pr63969-missing-driver.c

@@ -21,6 +21,10 @@ create_code (gcc_jit_context *ctxt, void *user_data)
      Unset it.  */
   gcc_jit_context_set_str_option (ctxt, GCC_JIT_STR_OPTION_PROGNAME, NULL);
 
+  /* By default, we use an embedded copy of the driver.
+     Opt-in to using an external copy of the driver.  */
+  gcc_jit_context_set_bool_use_external_driver (ctxt, 1);
+
   /* Break PATH, so that the driver can't be found
      by gcc::jit::playback::context::compile ()
      within gcc_jit_context_compile.  */