03d90a20a1
This is new incarantion of patch to identify unmergeable tree at streaming out
time rather than streaming in and to avoid pickling them to sccs with with hash
codes.
Building cc1 plus this patch reduces:
[WPA] read 4452927 SCCs of average size 1.986030
[WPA] 8843646 tree bodies read in total
[WPA] tree SCC table: size 524287, 205158 elements, collision ratio: 0.505204
[WPA] tree SCC max chain length 43 (size 1)
[WPA] Compared 947551 SCCs, 780270 collisions (0.823460)
[WPA] Merged 944038 SCCs
[WPA] Merged 5253521 tree bodies
[WPA] Merged 590027 types
...
[WPA] Size of mmap'd section decls: 99229066 bytes
[WPA] Size of mmap'd section function_body: 18398837 bytes
[WPA] Size of mmap'd section refs: 733678 bytes
[WPA] Size of mmap'd section jmpfuncs: 2965981 bytes
[WPA] Size of mmap'd section pureconst: 170248 bytes
[WPA] Size of mmap'd section profile: 17985 bytes
[WPA] Size of mmap'd section symbol_nodes: 3392736 bytes
[WPA] Size of mmap'd section inline: 2693920 bytes
[WPA] Size of mmap'd section icf: 435557 bytes
[WPA] Size of mmap'd section offload_table: 0 bytes
[WPA] Size of mmap'd section lto: 4320 bytes
[WPA] Size of mmap'd section ipa_sra: 651660 bytes
... to ...
[WPA] read 3312246 unshared trees
[WPA] read 1144381 mergeable SCCs of average size 4.833785
[WPA] 8843938 tree bodies read in total
[WPA] tree SCC table: size 524287, 197767 elements, collision ratio: 0.506446
[WPA] tree SCC max chain length 43 (size 1)
[WPA] Compared 946614 SCCs, 775077 collisions (0.818789)
[WPA] Merged 943798 SCCs
[WPA] Merged 5253336 tree bodies
[WPA] Merged 590105 types
....
[WPA] Size of mmap'd section decls: 81262144 bytes
[WPA] Size of mmap'd section function_body: 14702611 bytes
[WPA] Size of mmap'd section ext_symtab: 0 bytes
[WPA] Size of mmap'd section refs: 733695 bytes
[WPA] Size of mmap'd section jmpfuncs: 2332150 bytes
[WPA] Size of mmap'd section pureconst: 170292 bytes
[WPA] Size of mmap'd section profile: 17986 bytes
[WPA] Size of mmap'd section symbol_nodes: 3393358 bytes
[WPA] Size of mmap'd section inline: 2567939 bytes
[WPA] Size of mmap'd section icf: 435633 bytes
[WPA] Size of mmap'd section lto: 4320 bytes
[WPA] Size of mmap'd section ipa_sra: 651824 bytes
so results in about 22% reduction in global decl stream and 24% reduction on
function bodies stream (which is read mostly by ICF)
Martin, the zstd compression breaks the compression statistics (it works when
GCC is configured for zlib)
At first ltrans I get:
[LTRANS] Size of mmap'd section decls: 3734248 bytes
[LTRANS] Size of mmap'd section function_body: 4895962 bytes
... to ...
[LTRANS] Size of mmap'd section decls: 3479850 bytes
[LTRANS] Size of mmap'd section function_body: 3722935 bytes
So 7% reduction of global stream and 31% reduction of function bodies.
Stream in seems to get about 3% faster and stream out about 5% but it is
close to noise factor of my experiment. I expect bigger speedups on
Firefox but I did not test it today since my Firefox setup broke again.
GCC is not very good example on the problem with anonymous namespace
types since we do not have so many of them.
Sice of object files in gcc directory is reduced by 11% (because hash
numbers do not compress well I guess).
The patch makes DFS walk to recognize trees that are not merged (anonymous
namespace, local function/variable decls, anonymous types etc). As discussed
on IRC this is now done during the SCC walk rather than during the hash
computation. When local tree is discovered we know that SCC components of everything that is on
the stack reffers to it and thus is also local. Moreover we mark trees into hash set in output block
so if we get a cross edge referring to local tree it gets marked too.
Patch also takes care of avoiding SCC wrappers around some trees. In particular
1) singleton unmergeable SCCs are now streamed inline in global decl stream
This includes INTEGER_CSTs and IDENTIFIER_NODEs that are shared by different
code than rest of tree merging.
2) We use LTO_trees instead of LTO_tree_scc to wrap unmergeable SCC components.
It is still necessary to mark them because of forward references. LTO_trees
has simple header with number of trees and then things are streamed same way
as for LTO_tree_scc. That is tree headers first followed by pickled references
so things may point to future.
Of course it is not necessary for LTO_tree_scc to be single component and
streamer out may group more components together, but I decided to not snowball
the patch even more
3) In local streams when lto_output_tree is called and the topmost SCC components
turns out to be singleton we stream the tree directly
instead of LTO_tree_scc, hash code, pickled tree, reference to just stremaed tree.
LTO_trees is used to wrap all trees needed to represent tree being streamed.
It would make sense again to use only one LTO_trees rather than one per SCC
but I think this can be done incrementally.
In general local trees are now recognized by new predicate local_tree_p
Bit subtle is handing of TRANLSATION_UNIT_DECL, INTEGER_CST and
IDENTIFIER_NODE.
TRANSLATION_UNIT_DECL a local tree but references to it does not make
other trees local (because we also understand local decls now).
So I check for it later after localness propagation is done.
INTEGER_CST and IDENTIFIER_NODEs are merged but not via the tree merging
machinery. So it makes sense to stream them as unmergeable trees but we
still need to compute their hashes so SCCs referring them do not get too
large collision chains. For this reason they are checked just prior
stream out.
lto-bootstrapped/regteted x86_64-linux, OK?
gcc/ChangeLog:
2020-05-19 Jan Hubicka <hubicka@ucw.cz>
* lto-streamer-in.c (lto_input_scc): Add SHARED_SCC parameter.
(lto_input_tree_1): Strenghten sanity check.
(lto_input_tree): Update call of lto_input_scc.
* lto-streamer-out.c: Include ipa-utils.h
(create_output_block): Initialize local_trees if merigng is going
to happen.
(destroy_output_block): Destroy local_trees.
(DFS): Add max_local_entry.
(local_tree_p): New function.
(DFS::DFS): Initialize and maintain it.
(DFS::DFS_write_tree): Decide on streaming format.
(lto_output_tree): Stream inline singleton SCCs
* lto-streamer.h (enum LTO_tags): Add LTO_trees.
(struct output_block): Add local_trees.
(lto_input_scc): Update prototype.
gcc/lto/ChangeLog:
2020-05-19 Jan Hubicka <hubicka@ucw.cz>
* lto-common.c (compare_tree_sccs_1): Sanity check that we never
read TRANSLATION_UNIT_DECL.
(process_dref): Break out from ...
(unify_scc): ... here.
(process_new_tree): Break out from ...
(lto_read_decls): ... here; handle streaming of singleton trees.
(print_lto_report_1): Update statistics.
|
||
|---|---|---|
| config | ||
| contrib | ||
| fixincludes | ||
| gcc | ||
| gnattools | ||
| gotools | ||
| include | ||
| INSTALL | ||
| intl | ||
| libada | ||
| libatomic | ||
| libbacktrace | ||
| libcc1 | ||
| libcpp | ||
| libdecnumber | ||
| libffi | ||
| libgcc | ||
| libgfortran | ||
| libgo | ||
| libgomp | ||
| libhsail-rt | ||
| libiberty | ||
| libitm | ||
| libobjc | ||
| liboffloadmic | ||
| libphobos | ||
| libquadmath | ||
| libsanitizer | ||
| libssp | ||
| libstdc++-v3 | ||
| libvtv | ||
| lto-plugin | ||
| maintainer-scripts | ||
| zlib | ||
| .dir-locals.el | ||
| .gitattributes | ||
| .gitignore | ||
| ABOUT-NLS | ||
| ar-lib | ||
| ChangeLog | ||
| ChangeLog.jit | ||
| ChangeLog.tree-ssa | ||
| compile | ||
| config-ml.in | ||
| config.guess | ||
| config.rpath | ||
| config.sub | ||
| configure | ||
| configure.ac | ||
| COPYING | ||
| COPYING3 | ||
| COPYING3.LIB | ||
| COPYING.LIB | ||
| COPYING.RUNTIME | ||
| depcomp | ||
| install-sh | ||
| libtool-ldflags | ||
| libtool.m4 | ||
| lt~obsolete.m4 | ||
| ltgcc.m4 | ||
| ltmain.sh | ||
| ltoptions.m4 | ||
| ltsugar.m4 | ||
| ltversion.m4 | ||
| MAINTAINERS | ||
| Makefile.def | ||
| Makefile.in | ||
| Makefile.tpl | ||
| missing | ||
| mkdep | ||
| mkinstalldirs | ||
| move-if-change | ||
| multilib.am | ||
| README | ||
| symlink-tree | ||
| test-driver | ||
| ylwrap | ||
This directory contains the GNU Compiler Collection (GCC). The GNU Compiler Collection is free software. See the files whose names start with COPYING for copying permission. The manuals, and some of the runtime libraries, are under different terms; see the individual source files for details. The directory INSTALL contains copies of the installation information as HTML and plain text. The source of this information is gcc/doc/install.texi. The installation information includes details of what is included in the GCC sources and what files GCC installs. See the file gcc/doc/gcc.texi (together with other files that it includes) for usage and porting information. An online readable version of the manual is in the files gcc/doc/gcc.info*. See http://gcc.gnu.org/bugs/ for how to report bugs usefully. Copyright years on GCC source files may be listed using range notation, e.g., 1987-2012, indicating that every year in the range, inclusive, is a copyrightable year that could otherwise be listed individually.