Skip to content

R
riscv-gcc-1
Commit 277d7ee0 by Andi Kleen Committed by Andi Kleen
Options

Add make autoprofiledbootstrap 

Add support for profiledbootstrap with autofdo. Will be useful
to get better testing coverage of autofdo.

This requires Linux perf and autofdo to be installed, only
really for x86_64 linux on Intel so far.

Profile the whole build process with perf, then convert the file,
and pass it back to the compiler in the feedback stage.

The conversion has to be done per language, as only that knows
the name of the binary. Currently we only do it for C and C++,
as the other languages don't have enough coverage during a normal
bootstrap.

For lto1 it is also disabled, because it would only be useful during a LTO
bootstrap, but right now autofdo and LTO are not working together due to
PR66229

For common backend files always the profile output of the C++ compiler
is used. In theory multiple inputs could be merged here, but so far
that is not implemented.

The method is not friendly to partial rebuilds, as only the profile
information from the current rebuild is used. So if an error
occurs it is best to clean and restart, otherwise the code
quality may be worse.

This patch is fairly large, but most of it is auto generated from
autogen in Makefile.in. for the new stage targets.

Passes profiledbootstrap and normal bootstrap on x86_64-linux.
autoprofiledbootstrap is currently not working due to
PR70427 (but it finishes with that worked around)

The autofdo'ed compiler is ~7% faster on insn-recog.i (vs ~11% for
profiledfeedback), and ~4% faster for tramp3d-v4 (vs 10% for
profiledfeedback) on a Sandy Bridge system.

gcc/lto/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Make-lang.in: Add support for autofdo (disabled for now)

gcc/cp/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Make-lang.in: Add support for autofdo.

gcc/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Makefile.in: Regenerate.
	* doc/install.texi: Document autoprofiledbootstrap.

/:
2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Makefile.def: Add autoprofiledbootstrap.
	* Makefile.tpl: Dito.
	* Makefile.in: Regenerate.

gcc/c/:

2016-06-23  Andi Kleen  <ak@linux.intel.com>

	* Make-lang.in: Add support for autofdo.

From-SVN: r237733
parent 7f586d90
Showing with 103 additions and 5 deletions
ChangeLog
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Makefile.def: Add autoprofiledbootstrap.
* Makefile.tpl: Dito.
* Makefile.in: Regenerate.
2016-06-23 Jakub Sejdak <jakub.sejdak@phoesys.com>
* configure.ac: Disable libgcj and libgloss for Phoenix-RTOS targets.
......
Makefile.def
......@@ -639,3 +639,10 @@ bootstrap_stage = {
bootstrap_stage = {
id=feedback ; prev=profile ;
bootstrap_target=profiledbootstrap ; };
bootstrap_stage = {
id=autoprofile ; prev=1 ;
autoprofile="$$s/gcc/config/i386/$(AUTO_PROFILE)" ; };
bootstrap_stage = {
id=autofeedback ; prev=autoprofile ;
bootstrap_target=autoprofiledbootstrap ;
profile_data="PERF_DATA=perf.data" ; };
Makefile.in
This source diff could not be displayed because it is too large. You can view the blob instead.
Makefile.tpl
......@@ -382,6 +382,8 @@ MAKEINFO = @MAKEINFO@
EXPECT = @EXPECT@
RUNTEST = @RUNTEST@
AUTO_PROFILE = gcc-auto-profile -c 1000000
# This just becomes part of the MAKEINFO definition passed down to
# sub-makes. It lets flags be given on the command line while still
# using the makeinfo from the object tree.
......@@ -418,6 +420,8 @@ CXXFLAGS = @CXXFLAGS@
LIBCXXFLAGS = $(CXXFLAGS) -fno-implicit-templates
GOCFLAGS = $(CFLAGS)
CREATE_GCOV = create_gcov
TFLAGS =
# Defaults for all stages; some are overridden below.
......@@ -462,6 +466,12 @@ STAGEprofile_TFLAGS = $(STAGE2_TFLAGS)
STAGEfeedback_CFLAGS = $(STAGE3_CFLAGS) -fprofile-use
STAGEfeedback_TFLAGS = $(STAGE3_TFLAGS)
STAGEautoprofile_CFLAGS = $(STAGE2_CFLAGS) -g
STAGEautoprofile_TFLAGS = $(STAGE2_TFLAGS)
STAGEautofeedback_CFLAGS = $(STAGE3_CFLAGS)
STAGEautofeedback_TFLAGS = $(STAGE3_TFLAGS)
do-compare = @do_compare@
do-compare3 = $(do-compare)
......@@ -617,7 +627,8 @@ EXTRA_HOST_FLAGS = \
'READELF=$(READELF)' \
'STRIP=$(STRIP)' \
'WINDRES=$(WINDRES)' \
'WINDMC=$(WINDMC)'
'WINDMC=$(WINDMC)' \
'CREATE_GCOV=$(CREATE_GCOV)'
FLAGS_TO_PASS = $(BASE_FLAGS_TO_PASS) $(EXTRA_HOST_FLAGS)
......@@ -1147,6 +1158,7 @@ all-stage[+id+]-[+prefix+][+module+]: configure-stage[+id+]-[+prefix+][+module+]
[+exports+][+ IF prev +] \
[+poststage1_exports+][+ ENDIF prev +] [+extra_exports+] \
cd [+subdir+]/[+module+] && \
[+autoprofile+] \
$(MAKE) $(BASE_FLAGS_TO_PASS)[+ IF prefix +] \
CFLAGS="$(CFLAGS_FOR_TARGET)" \
CXXFLAGS="$(CXXFLAGS_FOR_TARGET)" \
......@@ -1160,7 +1172,7 @@ all-stage[+id+]-[+prefix+][+module+]: configure-stage[+id+]-[+prefix+][+module+]
LIBCFLAGS_FOR_TARGET="$(LIBCFLAGS_FOR_TARGET)" \
[+args+] [+IF prev +][+poststage1_args+][+ ELSE prev +] \
[+stage1_args+][+ ENDIF prev +] [+extra_make_flags+] \
TFLAGS="$(STAGE[+id+]_TFLAGS)" \
TFLAGS="$(STAGE[+id+]_TFLAGS)" [+profile_data+] \
$(TARGET-stage[+id+]-[+prefix+][+module+])
maybe-clean-stage[+id+]-[+prefix+][+module+]: clean-stage[+id+]-[+prefix+][+module+]
......
gcc/ChangeLog
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Makefile.in: Regenerate.
* doc/install.texi: Document autoprofiledbootstrap.
2016-06-23 Andi Kleen <ak@linux.intel.com>
* config/i386/gcc-auto-profile: New file.
2016-06-23 Martin Liska <mliska@suse.cz>
......
gcc/Makefile.in
......@@ -1567,6 +1567,13 @@ ALL_HOST_BACKEND_OBJS = $(GCC_OBJS) $(OBJS) $(OBJS-libcommon) \
$(GCOV_TOOL_OBJS) $(GENGTYPE_OBJS) gcc-ar.o gcc-nm.o gcc-ranlib.o \
lto-wrapper.o collect-utils.o
# for anything that is shared use the cc1plus profile data, as that
# is likely the most exercised during the build
ifeq ($(shell cat ../stage_current),stageautofeedback)
$(ALL_HOST_BACKEND_OBJS): ALL_COMPILERFLAGS += -fauto-profile=cc1plus.fda
$(ALL_HOST_BACKEND_OBJS): cc1plus.fda
endif
# This lists all host object files, whether they are included in this
# compilation or not.
ALL_HOST_OBJS = $(ALL_HOST_FRONTEND_OBJS) $(ALL_HOST_BACKEND_OBJS)
......@@ -1596,7 +1603,7 @@ MOSTLYCLEANFILES = insn-flags.h insn-config.h insn-codes.h \
gcov-iov$(build_exeext) gcov$(exeext) gcov-dump$(exeext) \
gcov-tool$(exeect) \
gengtype$(exeext) *.[0-9][0-9].* *.[si] *-checksum.c libbackend.a \
libcommon-target.a libcommon.a libgcc.mk
libcommon-target.a libcommon.a libgcc.mk perf.data
# This symlink makes the full installation name of the driver be available
# from within the *build* directory, for use when running the JIT library
......
gcc/c/ChangeLog
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Make-lang.in: Add support for autofdo.
2016-06-22 David Malcolm <dmalcolm@redhat.com>
PR c/70339
......
gcc/c/Make-lang.in
......@@ -60,6 +60,11 @@ c_OBJS = $(C_OBJS) cc1-checksum.o c/gccspec.o
# Use strict warnings for this front end.
c-warn = $(STRICT_WARN)
ifeq ($(shell cat ../stage_current),stageautofeedback)
$(C_OBJS): ALL_COMPILERFLAGS += -fauto-profile=cc1.fda
$(C_OBJS): cc1.fda
endif
# compute checksum over all object files and the options
# re-use the checksum from the prev-final stage so it passes
# the bootstrap comparison and allows comparing of the cc1 binary
......@@ -77,6 +82,10 @@ cc1-checksum.c : build/genchecksum$(build_exeext) checksum-options \
cc1$(exeext): $(C_OBJS) cc1-checksum.o $(BACKEND) $(LIBDEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ $(C_OBJS) \
cc1-checksum.o $(BACKEND) $(LIBS) $(BACKENDLIBS)
cc1.fda: ../stage1-gcc/cc1$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../stage1-gcc/cc1$(exeext) -gcov cc1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
#
# Build hooks:
......@@ -126,6 +135,7 @@ c.mostlyclean:
-rm -f cc1$(exeext)
-rm -f c/*$(objext)
-rm -f c/*$(coverageexts)
-rm -f cc1.fda
c.clean:
c.distclean:
-rm -f c/config.status c/Makefile
......@@ -146,3 +156,7 @@ c.stageprofile: stageprofile-start
-mv c/*$(objext) stageprofile/c
c.stagefeedback: stagefeedback-start
-mv c/*$(objext) stagefeedback/c
c.autostageprofile: autostageprofile-start
-mv c/*$(objext) autostageprofile/c
c.autostagefeedback: autostagefeedback-start
-mv c/*$(objext) autostagefeedback/c
gcc/cp/ChangeLog
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Make-lang.in: Add support for autofdo.
2016-06-21 Jason Merrill <jason@redhat.com>
* constraint.cc (constraints_satisfied_p): Keep as many levels of
......
gcc/cp/Make-lang.in
......@@ -81,6 +81,11 @@ CXX_AND_OBJCXX_OBJS = cp/call.o cp/decl.o cp/expr.o cp/pt.o cp/typeck2.o \
cp/vtable-class-hierarchy.o cp/constexpr.o cp/cp-ubsan.o \
cp/constraint.o cp/logic.o $(CXX_C_OBJS)
ifeq ($(shell cat ../stage_current),stageautofeedback)
$(CXX_AND_OBJCXX_OBJS): CFLAGS += -fauto-profile=cc1plus.fda
$(CXX_AND_OBJCXX_OBJS): cc1plus.fda
endif
# Language-specific object files for C++.
CXX_OBJS = cp/cp-lang.o c-family/stub-objc.o $(CXX_AND_OBJCXX_OBJS)
......@@ -122,6 +127,9 @@ endif
gperf -o -C -E -k '1-6,$$' -j1 -D -N 'libc_name_p' -L C++ \
$(srcdir)/cp/cfns.gperf --output-file $(srcdir)/cp/cfns.h
cc1plus.fda: ../stage1-gcc/cc1plus$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../stage1-gcc/cc1plus$(exeext) -gcov cc1plus.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
#
# Build hooks:
......@@ -235,7 +243,7 @@ c++.mostlyclean:
-rm -f doc/g++.1
-rm -f cp/*$(objext)
-rm -f cp/*$(coverageexts)
-rm -f xg++$(exeext) g++-cross$(exeext) cc1plus$(exeext)
-rm -f xg++$(exeext) g++-cross$(exeext) cc1plus$(exeext) cc1plus.fda
c++.clean:
c++.distclean:
-rm -f cp/config.status cp/Makefile
......@@ -257,3 +265,7 @@ c++.stageprofile: stageprofile-start
-mv cp/*$(objext) stageprofile/cp
c++.stagefeedback: stagefeedback-start
-mv cp/*$(objext) stagefeedback/cp
c++.autostageprofile: stageprofile-start
-mv cp/*$(objext) autostageprofile/cp
c++.autostagefeedback: stagefeedback-start
-mv cp/*$(objext) autostagefeedback/cp
gcc/doc/install.texi
......@@ -2776,6 +2776,17 @@ Unlike standard bootstrap, several additional restrictions apply. The
compiler used to build @code{stage1} needs to support a 64-bit integral type.
It is recommended to only use GCC for this.
On Linux/x86_64 hosts with some restrictions (no virtualization) it is
also possible to do autofdo build with @samp{make
autoprofiledback}. This uses Linux perf to sample branches in the
binary and then rebuild it with feedback derived from the profile.
Linux perf and the @code{autofdo} toolkit needs to be installed for
this.
Only the profile from the current build is used, so when an error
occurs it is recommended to clean before restarting. Otherwise
the code quality may be much worse.
@html
<hr />
<p>
......
gcc/lto/ChangeLog
2016-06-23 Andi Kleen <ak@linux.intel.com>
* Make-lang.in: Add support for autofdo (disabled for now)
2016-06-10 Martin Sebor <msebor@redhat.com>
PR c/71392
......
gcc/lto/Make-lang.in
......@@ -25,6 +25,15 @@ LTO_EXE = lto1$(exeext)
LTO_OBJS = lto/lto-lang.o lto/lto.o lto/lto-object.o attribs.o lto/lto-partition.o lto/lto-symtab.o
lto_OBJS = $(LTO_OBJS)
# this is only useful in a LTO bootstrap, but this does not work right
# now. Should reenable after this is fixed, but only when LTO bootstrap
# is enabled.
#ifeq ($(shell cat ../stage_current),stageautofeedback)
#$(LTO_OBJS): CFLAGS += -fauto-profile=lto1.fda
#$(LTO_OBJS): lto1.fda
#endif
# Rules
# These hooks are used by the main GCC Makefile. Consult that
......@@ -50,7 +59,7 @@ lto.srcinfo:
lto.install-plugin:
lto.mostlyclean:
rm -f $(LTO_OBJS) $(LTO_EXE)
rm -f $(LTO_OBJS) $(LTO_EXE) lto1.fda
lto.clean:
lto.distclean:
......@@ -71,5 +80,8 @@ $(LTO_EXE): $(LTO_OBJS) $(BACKEND) $(LIBDEPS)
+$(LLINKER) $(ALL_LINKERFLAGS) $(LDFLAGS) -o $@ \
$(LTO_OBJS) $(BACKEND) $(BACKENDLIBS) $(LIBS)
lto1.fda: ../prev-gcc/lto1$(exeext) ../prev-gcc/$(PERF_DATA)
$(CREATE_GCOV) -binary ../prev-gcc/lto1$(exeext) -gcov lto1.fda -profile ../prev-gcc/$(PERF_DATA) -gcov_version 1
# LTO testing is done as part of C/C++/Fortran etc. testing.
check-lto:
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment