gcc/contrib
Lawrence Crowl bff0e5295a Add a contrib script for comparing the performance of two sets of
compiler runs.

Usage documentation is in the script.

The script produces output of the form:

$ compare_two_ftime_report_sets "Log0/*perf" "Log3/*perf" 

Arithmetic sample for timevar log files
"Log0/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is 
trial count is 4, mean is 443.022 (95% confidence in 440.234 to 445.811),
std.deviation is 1.75264, std.error is 0.876322

Arithmetic sample for timevar log files
"Log3/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is 
trial count is 4, mean is 441.302 (95% confidence in 436.671 to 445.934),
std.deviation is 2.91098, std.error is 1.45549

The first sample appears to be 0.39% larger,
with 60% confidence of being larger.
To reach 95% confidence, you need roughly 14 trials,
assuming the standard deviation is stable, which is iffy.

Tested on x86_64 builds.


Index: contrib/ChangeLog

2012-11-05  Lawrence Crowl  <crowl@google.com>

	* compare_two_ftime_report_sets: New.

From-SVN: r193277
2012-11-07 00:36:04 +00:00
..
reghunt
regression * btest-gcc.sh (TESTLOGS): Make gfortran.sum optional. 2012-08-26 19:30:44 +00:00
testsuite-management Add a new option --clean_build to validate_failures.py 2012-11-02 15:43:25 -04:00
ChangeLog Add a contrib script for comparing the performance of two sets of 2012-11-07 00:36:04 +00:00
ChangeLog.tree-ssa
analyze_brprob
check_GNU_style.sh check_GNU_style.sh: Remove temporay file upon exit. 2012-10-02 09:41:28 +02:00
check_makefile_deps.sh check_makefile_deps.sh: Look for c-common.o in c-family/. 2012-06-19 20:01:23 +00:00
check_warning_flags.sh
compare-all-tests MAINTAINERS (crx port, [...]): Remove. 2011-03-22 19:58:18 +00:00
compare-debug Remove any .comment sections if the first cmp failed 2012-05-25 04:47:05 -07:00
compareSumTests3
compare_tests compare_tests: Append '/' to make 'find' traverse symlinked directories. 2012-04-24 07:22:41 +00:00
compare_two_ftime_report_sets Add a contrib script for comparing the performance of two sets of 2012-11-07 00:36:04 +00:00
config-list.mk contrib-list.mk (LIST): Remove arm-freebsd6, arm-linux, arm-ecos-elf, arm-rtems, arm-elf and arm-wince-pe. 2012-09-26 23:03:44 +01:00
dg-cmp-results.sh
dg-extract-results.sh * dg-extract-results.sh: Handle KFAILs. 2012-04-06 19:39:52 +00:00
dglib.pm
download_ecj
download_prerequisites
filter_gcc_for_doxygen
filter_knr2ansi.pl
filter_params.pl
gcc.doxy
gcc_build
gcc_update * gcc_update: Use $GCC_SVN to retrieve branch and revision. 2012-05-09 11:35:00 -04:00
gennews * gennews (files): Add files for GCC 4.7. 2012-03-13 23:21:38 +00:00
gthr_supp_vxw_5x.c
index-prop
make_sunver.pl * make_sunver.pl: Add missing newline at the end of extern "C++" block. 2012-09-04 13:56:06 +00:00
mklog The new mklog script prints three spaces after the date part, which is wrong. 2012-06-06 08:50:55 -04:00
paranoia.cc ansidecl.h (ENUM_BITFIELD): New, from gcc/system.h. 2011-04-25 18:05:37 +00:00
patch_tester.sh patch_tester.sh (nopristinecache): New shell var, set according to presence or absence of new -nopristinecache option. 2011-01-26 03:46:07 +00:00
prepare_patch.sh
repro_fail This patch adds two common shortcuts to the failure reproducer 2012-02-10 10:52:03 -05:00
test_installed
test_recheck
test_summary
texi2pod.pl
uninclude
warn_summary