compiler runs.
Usage documentation is in the script.
The script produces output of the form:
$ compare_two_ftime_report_sets "Log0/*perf" "Log3/*perf"
Arithmetic sample for timevar log files
"Log0/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is
trial count is 4, mean is 443.022 (95% confidence in 440.234 to 445.811),
std.deviation is 1.75264, std.error is 0.876322
Arithmetic sample for timevar log files
"Log3/*perf"
and selecting lines containing "TOTAL" with desired confidence 95 is
trial count is 4, mean is 441.302 (95% confidence in 436.671 to 445.934),
std.deviation is 2.91098, std.error is 1.45549
The first sample appears to be 0.39% larger,
with 60% confidence of being larger.
To reach 95% confidence, you need roughly 14 trials,
assuming the standard deviation is stable, which is iffy.
Tested on x86_64 builds.
Index: contrib/ChangeLog
2012-11-05 Lawrence Crowl <crowl@google.com>
* compare_two_ftime_report_sets: New.
From-SVN: r193277
Lawrence Crowl