I've just tested the script and I'm going to install the patch
to all active branches.
contrib/ChangeLog:
* git-backport.py: The script did 'git co HEAD~' when
there was no modified ChangeLog file in a successful
git cherry pick.
Run cherry-pick --continue without editor.
The patch is about to handle situations like seen
in 3ea6977d0f.
contrib/ChangeLog:
* gcc-changelog/git_commit.py: Properly
handle duplicite authors.
* gcc-changelog/test_email.py: New test.
* gcc-changelog/test_patches.txt: New patch.
So far, we expect from a commit that renames a file to contain a
changelog entry only for the new name. For example, after the following
commit:
$ git move foo bar
$ git commit
We expect the following changelog:
* bar: Renamed from foo.
Git does not keep track of renamings, only file deletions and additions.
The display of patches then uses heuristics (with config-dependent
parameters) to try to match deleted and added files in the same commit.
It is thus brittle to rely on this information.
This commit modifies changelog processing so that renames are considered
as a deletion of a file plus an addition of another file. The following
changelog is now expected for the above example:
* foo: Move...
* bar: Here.
contrib/
* gcc-changelog/git_email.py (GitEmail.__init__): Interpret file
renamings as a file deletion plus a file addition.
* gcc-changelog/git_repository.py (parse_git_revisions):
Likewise.
* gcc-changelog/test_email.py: New testcase.
* gcc-changelog/test_patches.txt: New testcase.
This extends the ChangeLog entries parsing machinery to handle entries
that cover multiple files spanning over multiple lines. For instance:
* first_file_patched.c, second_file_patched.c, third_file_patched.c,
fourth_file_patched.c: Do things.
contrib/
* gcc-changelog/git_commit.py (ChangeLogEntry): Handle entries
with multi-line file lists.
* gcc-changelog/test_email.py: New testcase.
* gcc-changelog/test_patches.txt: Likewise.
Currently, running gcc-changelog's unit tests may clutter the output
with tons of warnings such as:
.../contrib/gcc-changelog/git_email.py:40: ResourceWarning: unclosed
file <_io.TextIOWrapper name='/tmp/tmpt5okd4qp.patch' mode='r'
encoding='UTF-8'>
lines = open(self.filename).read().splitlines()
ResourceWarning: Enable tracemalloc to get the object allocation
traceback
This commit fixes these leaks, which restores a clean testsuite output.
contrib/
* gcc-changelog/git_update_version.py: Close file objects after
use.
* gcc-changelog/git_email.py: Likewise.
* gcc-changelog/test_email.py: Likewise.
The patch improves the script based on Jakub's needs,
I'm going to install the patch.
contrib/ChangeLog:
* gcc-changelog/git_update_version.py: Add 2 new options.
I've long introduced ChangeLog entries as "for dir/ChangeLog", a
format adopted by GNU CVS-Utilities some 20 years ago. My commits
have been formatted like this forever.
This patch makes it acceptable for git gcc-verify.
contrib/ChangeLog:
* gcc-changelog/git_commit.py (changelog_regex): Accept optional
'for' prefix.
This patch simplifies (!!!) the logic governing the naming of dump
files and auxiliary output files in the driver, in the compiler, and
in the LTO wrapper. No changes are made to the naming of primary
outputs, there are often ways to restore past behavior, and a number
of inconsistencies are fixed. Some internal options are removed
(-auxbase and -auxbase-strip), sensible existing uses of -dumpdir and
-dumpbase options remain unchanged, additional useful cases are added,
making for what is still admittedly quite complex. Extensive
documentation and testcases provide numerous examples, from normal to
corner cases.
The most visible changes are:
- aux and dump files now always go in the same directory, that
defaults to the directory of the primary output, but that can be
overridden with -dumpdir, -save-temps=*, or, preserving past behavior,
with a -dumpbase with a directory component.
- driver and compiler now have the same notion of naming of auxiliary
outputs, e.g. .dwo files will no longer be in one location while the
debug info suggests they are elsewhere, and -save-temps and .dwo
auxiliary outputs now go in the same location as .su, .ci and
coverage data, with consistent naming.
- explicitly-specified primary output names guide not only the
location of aux and dump outputs: the output base name is also used in
their base name, as a prefix when also linking (e.g. foo.c bar.c -o
foobar creates foobar-foo.dwo and foobar-bar.dwo with -gsplit-dwarf),
or as the base name instead of the input name (foo.c -c -o whatever.o
creates whatever.su rather than foo.su with -fstack-usage). The
preference for the input file base name, quite useful for our
testsuite, can be restored with -dumpbase "". When compiling and
linking tests in the testsuite with additional inputs, we now use this
flag. Files named in dejagnu board ldflags, libs, and ldscripts are
now quoted in the gcc testsuite with -Wl, so that they are not counted
as additional inputs by the compiler driver.
- naming a -dumpbase when compiling multiple sources used to cause
dumps from later compiles to overwrite those of earlier ones; it is
now used as a prefix when compiling multiple sources, like an
executable name above.
- the dumpbase, explicitly specified or computed from output or input
names, now also governs the naming of aux outputs; since aux outputs
usually replaced the suffix from the input name, while dump outputs
append their own additional suffixes, a -dumpbase-ext option is
introduced to enable a chosen suffix to be dropped from dumpbase to
form aux output names.
- LTO dump and aux outputs were quite a mess, sometimes leaking
temporary output names into -save-temps output names, sometimes
conversely generating desirable aux outputs in temporary locations.
They now obey the same logic of compiler aux and dump outputs, landing
in the expected location and taking the linker output name or an
explicit dumpbase overrider into account.
- Naming of -fdump-final-insns outputs now follows the dump file
naming logic for the .gkd files, and the .gk dump files generated in
the second -fcompare-debug compilation get the .gk inserted before the
suffix that -dumpbase-ext drops in aux outputs.
gcc/ChangeLog:
* common.opt (aux_base_name): Define.
(dumpbase, dumpdir): Mark as Driver options.
(-dumpbase, -dumpdir): Likewise.
(dumpbase-ext, -dumpbase-ext): New.
(auxbase, auxbase-strip): Drop.
* doc/invoke.texi (-dumpbase, -dumpbase-ext, -dumpdir):
Document.
(-o): Introduce the notion of primary output, mention it
influences auxiliary and dump output names as well, add
examples.
(-save-temps): Adjust, move examples into -dump*.
(-save-temps=cwd, -save-temps=obj): Likewise.
(-fdump-final-insns): Adjust.
* dwarf2out.c (gen_producer_string): Drop auxbase and
auxbase_strip; add dumpbase_ext.
* gcc.c (enum save_temps): Add SAVE_TEMPS_DUMP.
(save_temps_prefix, save_temps_length): Drop.
(save_temps_overrides_dumpdir): New.
(dumpdir, dumpbase, dumpbase_ext): New.
(dumpdir_length, dumpdir_trailing_dash_added): New.
(outbase, outbase_length): New.
(The Specs Language): Introduce %". Adjust %b and %B.
(ASM_FINAL_SPEC): Use %b.dwo for an aux output name always.
Precede object file with %w when it's the primary output.
(cpp_debug_options): Do not pass on incoming -dumpdir,
-dumpbase and -dumpbase-ext options; recompute them with
%:dumps.
(cc1_options): Drop auxbase with and without compare-debug;
use cpp_debug_options instead of dumpbase. Mark asm output
with %w when it's the primary output.
(static_spec_functions): Drop %:compare-debug-auxbase-opt and
%:replace-exception. Add %:dumps.
(driver_handle_option): Implement -save-temps=*/-dumpdir
mutual overriding logic. Save dumpdir, dumpbase and
dumpbase-ext options. Do not save output_file in
save_temps_prefix.
(adds_single_suffix_p): New.
(single_input_file_index): New.
(process_command): Combine output dir, output base name, and
dumpbase into dumpdir and outbase.
(set_collect_gcc_options): Pass a possibly-adjusted -dumpdir.
(do_spec_1): Optionally dumpdir instead of save_temps_prefix,
and outbase instead of input_basename in %b, %B and in
-save-temps aux files. Handle empty argument %".
(driver::maybe_run_linker): Adjust dumpdir and auxbase.
(compare_debug_dump_opt_spec_function): Adjust gkd dump file
naming. Spec-quote the computed -fdump-final-insns file name.
(debug_auxbase_opt): Drop.
(compare_debug_self_opt_spec_function): Drop auxbase-strip
computation.
(compare_debug_auxbase_opt_spec_function): Drop.
(not_actual_file_p): New.
(replace_extension_spec_func): Drop.
(dumps_spec_func): New.
(convert_white_space): Split-out parts into...
(quote_string, whitespace_to_convert_p): ... these. New.
(quote_spec_char_p, quote_spec, quote_spec_arg): New.
(driver::finalize): Release and reset new variables; drop
removed ones.
* lto-wrapper.c (HAVE_TARGET_EXECUTABLE_SUFFIX): Define if...
(TARGET_EXECUTABLE_SUFFIX): ... is defined; define this to the
empty string otherwise.
(DUMPBASE_SUFFIX): Drop leading period.
(debug_objcopy): Use concat.
(run_gcc): Recognize -save-temps=* as -save-temps too. Obey
-dumpdir. Pass on empty dumpdir and dumpbase with a directory
component. Simplify temp file names.
* opts.c (finish_options): Drop aux base name handling.
(common_handle_option): Drop auxbase-strip handling.
* toplev.c (print_switch_values): Drop auxbase, add
dumpbase-ext.
(process_options): Derive aux_base_name from dump_base_name
and dump_base_ext.
(lang_dependent_init): Compute dump_base_ext along with
dump_base_name. Disable stack usage and callgraph-info during
lto generation and compare-debug recompilation.
gcc/fortran/ChangeLog:
* options.c (gfc_get_option_string): Drop auxbase, add
dumpbase_ext.
gcc/ada/ChangeLog:
* gcc-interface/lang-specs.h: Drop auxbase and auxbase-strip.
Use %:dumps instead of -dumpbase. Add %w for implicit .s
primary output.
* switch.adb (Is_Internal_GCC_Switch): Recognize dumpdir and
dumpbase-ext. Drop auxbase and auxbase-strip.
lto-plugin/ChangeLog:
* lto-plugin.c (skip_in_suffix): New.
(exec_lto_wrapper): Use skip_in_suffix and concat to build
non-temporary output names.
(onload): Look for -dumpdir in COLLECT_GCC_OPTIONS, and
override link_output_name with it.
contrib/ChangeLog:
* compare-debug: Adjust for .gkd files named as dump files,
with the source suffix rather than the object suffix.
gcc/testsuite/ChangeLog:
* gcc.misc-tests/outputs.exp: New.
* gcc.misc-tests/outputs-0.c: New.
* gcc.misc-tests/outputs-1.c: New.
* gcc.misc-tests/outputs-2.c: New.
* lib/gcc-defs.exp (gcc_adjusted_linker_flags): New.
(gcc_adjust_linker_flags): New.
(dg-additional-files-options): Call it. Pass -dumpbase ""
when there are additional sources.
* lib/profopt.exp (profopt-execute): Pass the executable
suffix with -dumpbase-ext.
* lib/scandump.exp (dump-base): Mention -dumpbase "" use.
* lib/scanltranstree.exp: Adjust dump suffix expectation.
* lib/scanwpaipa.exp: Likewise.
We should be able to generate ChangeLog entries for
commits like b3d566f570.
I'm going to install the patch.
contrib/ChangeLog:
* gcc-changelog/git_commit.py: Parse changelog entries for
ignored locations.
* gcc-changelog/test_email.py: Add new test for it.
* gcc-changelog/test_patches.txt: Likewise.
This patch introduces a prepare-commit-msg hook that appends a ChangeLog
skeleton to a commit message when the GCC_FORCE_MKLOG environment variable
is set, and a 'git commit-mklog' command set that variable while running
'git commit'.
contrib/ChangeLog:
* prepare-commit-msg: New file.
* gcc-git-customization.sh: Install it. Add commit-mklog alias.
* mklog.py: Add new option -c which appends
to a ChangeLog file.
* gcc-changelog/git_commit.py: Add trailing '/'
for libdruntime. Allow empty changelog for
only ignored files.
* gcc-changelog/test_email.py: New test for go
patch in ignored location.
* gcc-changelog/test_patches.txt: Add test.
2020-05-22 Jakub Jelinek <jakub@redhat.com>
* gcc-changelog/git_commit.py: Add trailing / to
gcc/testsuite/go.test/test and replace gcc/go/frontend/
with gcc/go/gofrontend/ in ignored locations.
* gcc-changelog/git_commit.py: Add author_tuple
only if not present in author_lines.
* gcc-changelog/test_email.py: New test.
* gcc-changelog/test_patches.txt: Add new patch.
* gcc-changelog/git_commit.py: Refactor to make flake8 happy.
* gcc-changelog/test_email.py: Add new test.
* gcc-changelog/test_patches.txt: Add new patch.
git-check-commit.py does not know about "Reviewed-by",
"Reviewed-on", and "Signed-off-by" lines and hence it
expects those lines which follow the ChangeLog entries
to be indented by a tab.
This commit makes the script skip those lines. No further
processing is attempted because the review information
is not part of the ChangeLogs.
contrib/
2020-05-20 Frederik Harwath <frederik@codesourcery.com>
* gcc-changelog/git_commit.py: Skip over lines starting
with "Reviewed-by: ", "Reviewed-on: ", or "Signed-off-by: "
* gcc-changelog/git_commit.py: Support cherry pick
prefix.
* gcc-changelog/test_email.py: Test it.
* gcc-changelog/test_patches.txt: Add new patch.
* gcc-git-customization.sh: Add gcc-backport.
contrib/ChangeLog:
2020-05-15 Martin Liska <mliska@suse.cz>
* gcc-git-customization.sh: Add
alias.gcc-mklog new hook.
* mklog.py: New file.
* test_mklog.py: New file.
This commit is for the benefit of GDB, but as the binutils-gdb
repository shares the contrib/ directory with gcc, this commit must
first be applied to gcc then copied back to binutils-gdb.
This commit extends the two scripts contrib/dg-extract-results.{py,sh}
to handle some new, GDB specific test result types. These test
results types should never appear in GCC, or any other tool that
shares the contrib/ directly, so this change should be harmless.
In this patch series:
https://sourceware.org/pipermail/gdb-patches/2020-April/167847.html
changes were made in GDB's use of Dejagnu so that two additional
conditions could be detected, these are:
1. Test names that contain either the build or source paths. Such
test names make it difficult to compare the results of two test runs
of GDB from two different directories, and
2. Duplicate test names. Duplicates make it difficult to track down
exactly which test has failed.
When running Dejagnu on GDB we can now (sometimes) see two additional
test result types matching the above conditions, these are '# of paths
in test names' and '# of duplicate test names'.
If the test is run in parallel mode (make -j...) then these extra test
results will appear in the individual test summary files, but are not
merged into the final summary file.
Additionally, within the summary file there are now two new types of
test summary line, these are 'PATH: ...' and 'DUPLICATE: ...', these
allow users to quickly search the test summary to track down where the
offending test names are. These lines are similarly not merged into
the unified gdb.sum file after a parallel test run.
This commit extends the dg-extract-results.* scripts to calculate the
totals for the two new result types, and to copy the new test summary
lines into the unified summary file.
contrib/ChangeLog:
* dg-extract-results.py: Handle GDB specific test types.
* dg-extract-results.sh: Likewise.
* gcc-changelog/git_check_commit.py: New file.
* gcc-changelog/git_commit.py: New file.
* gcc-changelog/git_email.py: New file.
* gcc-changelog/git_repository.py: New file.
* gcc-changelog/git_update_version.py: New file.
As Iain reported, my change broke the case when one has bison >= 3,
but make decides there is no reason to regenerate plural.c, unfortunately
that seems to be a scenario I haven't tested. The problem is that
the pregenerated plural.c has been generated with bison 1.35, but when
config.h says HAVE_BISON3, the code assumes it is the bison3 variant.
What used to work fine is when one has bison >= 3 and plural.c has been
regenerated (e.g. do touch intl/plural.y and it will work), or when
one doesn't have any bison (then nothing is regenerated, but HAVE_BISON3
isn't defined either), or when one has bison < 3 and doesn't need to
regenerate, or when one has bison < 3 and it is regenerated.
The following patch fixes this, by killing the HAVE_BISON3 macro from
config.h, and instead remembering the fact whether plural.c has been created
with bison < 3 or bison >= 3 in a separate new plural-config.h header.
The way this works:
- user doesn't have bison
- user has bison >= 3, but intl/{plural-config.h,plural.c} aren't older than intl/plural.y
- user has bison < 3, but intl/{plural-config.h,plural.c} aren't older than intl/plural.y
pregenerated !USE_BISON3 plural.c and plural-config.h from source
dir is used, nothing in the objdir
- user has bison >= 3 and intl/plural.y is newer
Makefile generates plural.c and USE_BISON3 plural-config.h in the
objdir, which is then used in preference to srcdir copies
- user has bison < 3 and intl/plural.y is newer
Makefile generates plural.c and !USE_BISON3 plural-config.h in the
objdir, which is then used in preference to srcdir copies
I have tested all these cases and make all-yes worked in all the cases.
If one uses the unsupported ./configure where srcdir == objdir, I guess
(though haven't tested) that it should still work, just it would be nice
if such people didn't try to check in the plural{.c,-config.h} they have
regenerated.
What doesn't work, but didn't work before either (just tested gcc-9 branch
too) is when one doesn't have bison and plural.y is newer than plural.c.
Don't do that ;)
2020-04-16 Jakub Jelinek <jakub@redhat.com>
PR bootstrap/92008
intl/
* configure.ac: Remove HAVE_BISON3 AC_DEFINE.
* Makefile.in (HEADERS): Add plural-config.h.
(.y.c): Also create plural-config.h.
(dcigettext.o loadmsgcat.o plural.o plural-exp.o): Also depend
on plural-config.h.
(plural-config.h): Depend on plural.c.
* plural-exp.h: Include plural-config.h. Use USE_BISON3 instead
of HAVE_BISON3.
* plural.y: Use USE_BISON3 instead of HAVE_BISON3.
* configure: Regenerated.
* plural.c: Regenerated.
* config.h.in: Regenerated.
* plural-config.h: Generated.
contrib/
* gcc_update: Add intl/plural.y dependency for intl/plural-config.h.
Whilst trying to convert the add vendor branch script to work with
personal branches I encountered a minor issue where git would report
ambiguous refs when checking out the new branch.
It turns out that this is because git considers <me>/<topic> to be
ambiguous if both
refs/heads/<me>/<topic>
and
refs/remotes/<me>/<topic>
exist in the list of known branches.
Having thought about this a bit, I think the best solution is to adopt
something more like the vendors space and call the remote users/<me>
(this also works better if you want to set up remotes to track other
users branches as well).
So this patch has two parts.
1) It updates gcc-git-customization.sh to set up the new 'remote' and
converts any existing remote and branches tracking that to the new
format
2) It adds a new script to set up a personal branch on the gcc git repository.
* gcc-git-customization.sh: Use users/<pfx> for the personal remote
rather than just <pfx>. Convert any existing personal branches to the
new remote.
* git-add-user-branch.sh: New file.
