Commit Graph

21 Commits

Author SHA1 Message Date
Andrew Burgess
c9a41202b2 contrib: Handle GDB specific test result types
This commit is for the benefit of GDB, but as the binutils-gdb
repository shares the contrib/ directory with gcc, this commit must
first be applied to gcc then copied back to binutils-gdb.

This commit extends the two scripts contrib/dg-extract-results.{py,sh}
to handle some new, GDB specific test result types.  These test
results types should never appear in GCC, or any other tool that
shares the contrib/ directly, so this change should be harmless.

In this patch series:
  https://sourceware.org/pipermail/gdb-patches/2020-April/167847.html
changes were made in GDB's use of Dejagnu so that two additional
conditions could be detected, these are:

  1. Test names that contain either the build or source paths.  Such
  test names make it difficult to compare the results of two test runs
  of GDB from two different directories, and

  2. Duplicate test names.  Duplicates make it difficult to track down
  exactly which test has failed.

When running Dejagnu on GDB we can now (sometimes) see two additional
test result types matching the above conditions, these are '# of paths
in test names' and '# of duplicate test names'.

If the test is run in parallel mode (make -j...) then these extra test
results will appear in the individual test summary files, but are not
merged into the final summary file.

Additionally, within the summary file there are now two new types of
test summary line, these are 'PATH: ...' and 'DUPLICATE: ...', these
allow users to quickly search the test summary to track down where the
offending test names are.  These lines are similarly not merged into
the unified gdb.sum file after a parallel test run.

This commit extends the dg-extract-results.* scripts to calculate the
totals for the two new result types, and to copy the new test summary
lines into the unified summary file.

contrib/ChangeLog:

	* dg-extract-results.py: Handle GDB specific test types.
	* dg-extract-results.sh: Likewise.
2020-05-15 11:19:15 +01:00
Andrew Burgess
aa7710cb6c contrib: Add KPASS support to dg-extract-results.{sh,py}
Extend dg-extract-results.sh and dg-extract-results.py to support the
KPASS test result status.  This is required by GDB which uses a copy
of the dg-extract-results.{sh,py} scripts that it tries to keep in
sync with GCC.

ChangeLog:

        * contrib/dg-extract-results.sh: Add support for KPASS.
        * contrib/dg-extract-results.py: Likewise.

From-SVN: r277260
2019-10-21 13:41:29 +01:00
Jakub Jelinek
ec60715d39 dg-extract-results.sh: Only handle WARNING: program timed out lines specially in "$MODE" == "sum".
* dg-extract-results.sh: Only handle WARNING: program timed out
	lines specially in "$MODE" == "sum".  Restore previous behavior
	for "$MODE" != "sum".  Clear has_timeout and timeout_cnt if in
	a different variant or curfile is empty.
	* dg-extract-results.py: Fix a typo.

From-SVN: r270415
2019-04-17 19:57:02 +02:00
Christophe Lyon
e8d18fd8f6 dg-extract-results.sh: Fix order of WARNING and following test result.
2019-03-05  Christophe Lyon  <christophe.lyon@linaro.org>

	contrib/
	* dg-extract-results.sh: Fix order of WARNING and following test
	result.

From-SVN: r269393
2019-03-05 17:28:11 +01:00
Christophe Lyon
f874089d31 contrib/dg-extract-results: Handle timeout warnings
2019-02-04  Christophe Lyon  <christophe.lyon@linaro.org>

	contrib/
	* dg-extract-results.py: Keep timeout warnings next to their
	matching test.
	* dg-extract-results.sh: Likewise.

From-SVN: r268511
2019-02-04 09:55:00 +01:00
Rainer Orth
643e08545f Revert bogus dg-extract-results.sh change
* dg-extract-results.sh: Revert bogus change.

From-SVN: r240429
2016-09-23 11:35:34 +00:00
Christophe Lyon
aa9baacfc9 Report DejaGnu ERROR messages in dg-extract-results.
2016-09-20  Christophe Lyon  <christophe.lyon@linaro.org>

	* dg-extract-results.py: Report DejaGnu error in the final
	summary.
	* dg-extract-results.sh: Likewise.

From-SVN: r240289
2016-09-20 22:15:30 +02:00
Bernd Edlinger
74d3531523 dg-extract-results.sh: Check that the necessary python modules exist.
2015-03-06  Bernd Edlinger  <bernd.edlinger@hotmail.de>

        * dg-extract-results.sh: Check that the necessary python modules exist.

From-SVN: r221247
2015-03-06 18:35:15 +00:00
Sergio Durigan Junior
64e6d5c442 dg-extract-results.sh: Use --text with grep to avoid issues with binary files.
2014-12-17  Sergio Durigan Junior  <sergiodj@redhat.com>

	* dg-extract-results.sh: Use --text with grep to avoid issues with
	  binary files.  Fall back to cat -v, if that doesn't work.

From-SVN: r218843
2014-12-17 22:35:04 +00:00
Richard Sandiford
0af7b53a8b dg-extract-results.py: New file.
contrib/
	* dg-extract-results.py: New file.
	* dg-extract-results.sh: Use it if the environment seems suitable.

From-SVN: r210243
2014-05-08 20:24:33 +00:00
David Blaikie
db06526856 dg-extract-results.sh: Fix order of summary counts.
2013-01-15  David Blaikie <dblaikie@gmail.com>

	* dg-extract-results.sh: Fix order of summary counts.

From-SVN: r195224
2013-01-16 03:42:31 +00:00
David Blaikie
9fa633fc14 dg-extract-results.sh: Constrain the start-of-log pattern.
2013-01-15  David Blaikie <dblaikie@gmail.com>

	* dg-extract-results.sh: Constrain the start-of-log pattern.

From-SVN: r195216
2013-01-15 20:19:33 +00:00
David Blaikie
16dd2667ee dg-extract-results.sh: Handle KPASSes.
2013-01-15  David Blaikie <dblaikie@gmail.com>

	* dg-extract-results.sh: Handle KPASSes.

From-SVN: r195215
2013-01-15 20:18:20 +00:00
Doug Evans
a0806f33ad * dg-extract-results.sh: Handle KFAILs.
From-SVN: r186200
2012-04-06 19:39:52 +00:00
Rainer Orth
bc492e47c8 configure.ac: Redirect grep stdout, stderr to /dev/null instead of grep -q.
libjava:
	* configure.ac: Redirect grep stdout, stderr to /dev/null instead
	of grep -q.
	Use -- instead of grep -e.
	* configure: Regenerate.

	gcc:
	* configure.ac (gcc_cv_as_ld_jalr_reloc): Redirect grep stdout,
	stderr to /dev/null instead of grep -q.
	* configure: Regenerate.

	contrib:
	* dg-extract-results.sh: Redirect grep output to /dev/null instead
	of grep -q.

From-SVN: r159840
2010-05-25 18:50:01 +00:00
Jan Kratochvil
fccff34135 Fix *.log tests output containing "===".
contrib/
	Fix *.log tests output containing "===".
	* dg-extract-results.sh (/\===/): Rename to ...
	(/^\t\t=== .* ===$/): ... this pattern.

From-SVN: r157645
2010-03-22 17:36:49 +00:00
Rainer Orth
54bceaf397 * dg-extract-results.sh: Fix result order to match sequential case.
From-SVN: r157175
2010-03-02 20:28:55 +00:00
Rainer Orth
5392f6be70 * dg-extract-results.sh: Don't use tail -n.
From-SVN: r155655
2010-01-05 18:29:48 +00:00
Jakub Jelinek
060717fcc5 re PR testsuite/39807 (Reporting of testsuite failures are messed up when using -j)
PR testsuite/39807
	* dg-extract-results.sh: Close open files and use >> instead of >
	to decrease number of concurrently open files from awk.  Avoid
	= at the beginning of a regexp and redirect to a file determined
	by curfile variable rather than concatenated strings to workaround
	Solaris nawk bug.

From-SVN: r146850
2009-04-27 21:11:23 +02:00
Jakub Jelinek
ba4f8572c1 Mark dg-extract-results.sh executable.
From-SVN: r141338
2008-10-24 11:00:54 +02:00
Jakub Jelinek
7134e6056c dg-extract-results.sh: New file.
contrib/
	* dg-extract-results.sh: New file.
gcc/
	* Makefile.in (lang_checks_parallelized, check_gcc_parallelize,
	check_p_tool, check_p_vars, check_p_subno, check_p_comma,
	check_p_subwork, check_p_numbers, check_p_subdir, check_p_subdirs):
	New variables.
	(check-subtargets, check-%-subtargets, check-parallel-%): New
	targets.
	(check-%): For test targets listed in lang_checks_parallelized
	if -j is used and RUNTESTFLAGS doesn't specify tests to execute,
	run the testing in multiple make goals, possibly parallel, and
	afterwards run dg-extract-results.sh to merge the sum and log files.
gcc/cp/
	* Make-lang.in (check-c++-subtargets): New alias for
	check-g++-subtargets.
	(lang_checks_parallelized): Add check-g++.
	(check_g++_parallelize): New variable.
gcc/fortran/
	* Make-lang.in (check-f95-subtargets, check-fortran-subtargets): New
	aliases for check-gfortran-subtargets.
	(lang_checks_parallelized): Add check-gfortran.
	(check_gfortran_parallelize): New variable.
gcc/ada/
	* gcc-interface/Make-lang.in (check-ada-subtargets): Depend on
	check-acats-subtargets and check-gnat-subtargets.
	(check_acats_targets): New variable.
	(check-acats-subtargets, check-acats%): New targets.
	(check-acats): If -j is used and CHAPTERS is empty, run the testing
	in multiple make goals, possibly parallel, and afterwards run
	dg-extract-results.sh to merge the sum and log files.
gcc/java/
	* Make-lang.in (check-java-subtargets): New target.
libstdc++-v3/
	* testsuite/Makefile.am (AUTOMAKE_OPTIONS): Remove dejagnu.
	(RUNTESTDEFAULTFLAGS, EXPECT, check_DEJAGNU_normal_targets): New
	variables.
	(%/site.exp, check-DEJAGNU%): New targets.
	(check-am): Run $(MAKE) check-DEJAGNU.
	* testsuite/Makefile.in: Regenerated.

From-SVN: r141337
2008-10-24 10:58:44 +02:00