single-step.exp takes a while to run, and while that's not necessarily
bad, here it's because the default value of SINGLE_STEP_COUNT is 10,000.
We're not going to gain any more insight into perf issues
single-stepping (stepi) 10,000 times over 1,000 times,
so this patch changes the default to 1,000.
gdb/testsuite/ChangeLog:
* gdb.perf/single-step.exp (SINGLE_STEP_COUNT): Change to 1000 from
10000.
This patch does two things.
1) Add support for multiple data points.
2) Move the "report" output from perftest.log to perftest.sum.
I want to record the raw data somewhere, and a bit of statistical analysis
(standard deviation left for another day), but I also don't want
it to clutter up the basic report.
This patch takes a cue from gdb.{sum,log} and does the same thing
with perftest.{sum,log}.
Ultimately, we'll probably want to emit raw data to csv files or some
such and then do post-processing passes on that.
gdb/testsuite/ChangeLog:
* lib/perftest/reporter.py (SUM_FILE_NAME): New global.
(LOG_FILE_NAME): New global.
(TextReporter.__init__): Initialize self.txt_sum.
(TextReporter.report): Add support for multiple data-points.
Move report to perftest.sum, put raw data in perftest.log.
(TextReporter.start): Open sum and log files.
(TextReporter.end): Close sum and log files.
* lib/perftest/testresult.py (SingleStatisticTestResult.record): Handle
multiple data-points.
I think I lost a patch along the way, because I remember needing
something like this, but the reverted patch isn't the right way to
do this. Removing ...
gdb/testsuite/ChangeLog:
* gdb.perf/lib/perftest/measure.py (MeasurementCpuTime::stop): Print
result.
(MeasurementWallTime::stop): Ditto.
(MeasurementVmSizeTime::stop): Ditto.
These testcases are mocks of real programs.
GDB doesn't care what the programs do, they just have to look
and/or behave like the real program.
These testcases exercise gdb when debugging really large programs.
E.g., gmonster-1 has 10,000 CUs, and gmonster-2 has 1000 shared libs
(which is actually a little small, 5000 would be more accurate).
gdb/testsuite/ChangeLog:
* gdb.perf/lib/perftest/utils.py: New file.
* gdb.perf/gm-hello.cc: New file.
* gdb.perf/gm-pervasive-typedef.cc: New file.
* gdb.perf/gm-pervasive-typedef.h: New file.
* gdb.perf/gm-std.cc: New file.
* gdb.perf/gm-std.h: New file.
* gdb.perf/gm-use-cerr.cc: New file.
* gdb.perf/gm-utils.h: New file.
* gdb.perf/gmonster-null-lookup.py: New file.
* gdb.perf/gmonster-pervasive-typedef.py: New file.
* gdb.perf/gmonster-print-cerr.py: New file.
* gdb.perf/gmonster-ptype-string.py: New file.
* gdb.perf/gmonster-runto-main.py: New file.
* gdb.perf/gmonster-select-file.py: New file.
* gdb.perf/gmonster1-null-lookup.exp: New file.
* gdb.perf/gmonster1-pervasive-typedef.exp: New file.
* gdb.perf/gmonster1-print-cerr.exp: New file.
* gdb.perf/gmonster1-ptype-string.exp: New file.
* gdb.perf/gmonster1-runto-main.exp: New file.
* gdb.perf/gmonster1-select-file.exp: New file.
* gdb.perf/gmonster1.cc: New file.
* gdb.perf/gmonster1.exp: New file.
* gdb.perf/gmonster2-null-lookup.exp: New file.
* gdb.perf/gmonster2-pervasive-typedef.exp: New file.
* gdb.perf/gmonster2-print-cerr.exp: New file.
* gdb.perf/gmonster2-ptype-string.exp: New file.
* gdb.perf/gmonster2-runto-main.exp: New file.
* gdb.perf/gmonster2-select-file.exp: New file.
* gdb.perf/gmonster2.cc: New file.
* gdb.perf/gmonster2.exp: New file.
gdb/testsuite/ChangeLog:
* gdb.perf/README: New file.
* lib/perftest.exp (tcl_string_list_to_python_list): New function.
* lib/gen-perf-test.exp: New file.
I had occasion to use with_gdb_prompt in a test for the patch for PR 17314
and was passing the plain text prompt as the value, "(top-gdb)",
instead of a regexp, "\(top-gdb\)" (expressed as "\\(top-gdb\\)" in TCL).
I then discovered that in order to restore the prompt gdb passes the
original value of $gdb_prompt to "set prompt", which works because
"set prompt \(gdb\) " is equivalent to "set prompt (gdb) ".
Perhaps I'm being overly cautious but this feels a bit subtle,
but at any rate as an API choice I'd much rather pass the plain text
form to with_gdb_prompt.
I also discovered that the initial value of gdb_prompt is set in
two places to two different values.
At the global level gdb.exp sets it to "\[(\]gdb\[)\]"
and default_gdb_init sets it to "\\(gdb\\)".
The former form is undesirable as an argument to "set prompt",
but it's not clear to me that just deleting this code won't break
anything. Thus I just changed the value to be consistent and added
a comment.
gdb/testsuite/ChangeLog:
* lib/gdb.exp (gdb_prompt): Add comment and change initial value to
be consistent with what default_gdb_init uses.
(with_gdb_prompt): Change form of PROMPT argument from a regexp to
the plain text of the prompt. Add some logging printfs.
* gdb.perf/disassemble.exp: Update call to with_gdb_prompt.
This patch add a perf test case on skip-prologue by inserting
breakpoints on two functions many times, in order to exercise
skip-prologue.
gdb/testsuite:
2013-12-15 Yao Qi <yao@codesourcery.com>
* gdb.perf/skip-prologue.c: New.
* gdb.perf/skip-prologue.exp: New.
* gdb.perf/skip-prologue.py: New.
This patch adds a test case to test the performance of GDB doing
disassembly.
gdb/testsuite/
2013-11-28 Yao Qi <yao@codesourcery.com>
* lib/gdb.exp (with_gdb_prompt): New proc.
* gdb.perf/disassemble.exp: New.
* gdb.perf/disassemble.py: New.
This patch is to add a test case to on the performance of GDB handling
load and unload of shared library.
In V4:
- Handle malloc and dlopen failure,
- Document test parameters.
In V3, there are some changes,
- Adapt to perf test framework changes.
- Measure load and unload separately.
In V2, there are some changes,
- A new proc gdb_produce_source to produce source files. I tried to
move all source file generation code out of solib.exp, but
compilation step still needs to know the generated file names. I
have to hard-code the file names in compilation step, which is not
good to me, so I give up on this moving.
- SOLIB_NUMBER -> SOLIB_COUNT
- New variable SOLIB_DLCLOSE_REVERSED_ORDER to control the order of
iterating a list of shared libs to dlclose them.
- New variable GDB_PERFORMANCE to enable these perf test cases.
- Remove dlsym call in solib.c.
- Update solib.py for the updated framework.
gdb/testsuite/
* lib/gdb.exp (gdb_produce_source): New procedure.
* gdb.perf/solib.c: New.
* gdb.perf/solib.exp: New.
* gdb.perf/solib.py: New.
This patch adds a basic framework to do performance testing for GDB.
perftest.py is about the test case, testresult.py is about test
results, and how are they saved. reporter.py is about how results
are reported (in what format). measure.py is about measuring the
execution of tests by a collection of measurements.
In V5:
- Simplify perftest.exp.
In V4:
- Rename MeasurementCPUTime to MeasurementCpuTime,
- Add 'pass' in empty method,
- Simplify string comparison in perftest.exp.
- Rename GDB_PERFORMANCE to GDB_PERFTEST_MODE and rename
GDB_PERFORMANCE_TIMEOUT to GDB_PERFTEST_TIMEOUT.
In V3, there are some changes,
- Add wall time measurement, cpu time measurement and vmsize
measurement.
- Rename SingleStatisticTestCase to TestCaseWithBasicMeasurements,
which measures cpu time, wall time, and memory (vmsize).
- GDB_PERFORMANCE=run|compile|both to control the mode of perf
testing.
- New GDB_PERFORMANCE_TIMEOUT to specify the timeout.
- Split proc prepare to proc compile and startup.
- Disable GC while doing measurements.
In V2, there are several changes to address Doug and Sanimir's
comments.
- Add copyright header and docstring in perftest/__init__.py
- Remove config.py.
- Fix docstring format.
- Rename classes "SingleVariable" to "SingleStatistic".
- Don't extend gdb.Function in class TestCase. Add a new method run
to run the test case so that we can pass parameters to test.
- Allow to customize whether to warm up and to append test log.
- Move time measurement into test harness. Add a new class
Measurement for a specific measurement and a new class Measure to
measure them for a given test case.
- A new class ResultFactory to create instances of TestResult.
- New file lib/perftest.exp, which is to do some preparations and
cleanups to simplify each *.exp file.
- Skip compilation step if GDB_PERFORMANCE_SKIP_COMPILE is set.
gdb/testsuite/
2013-11-06 Yao Qi <yao@codesourcery.com>
* lib/perftest.exp: New.
* gdb.perf/lib/perftest/__init__.py: New.
* gdb.perf/lib/perftest/measure.py: New.
* gdb.perf/lib/perftest/perftest.py: New.
* gdb.perf/lib/perftest/reporter.py: New.
* gdb.perf/lib/perftest/testresult.py: New.
We add a new dir gdb.perf in testsuite for all performance tests.
However, current 'make check' logic will either run dejagnu in
directory testsuite or iterate all gdb.* directories which has *.exp
files. Both of them will run tests in gdb.perf. We want to achieve:
1) typical 'make check' should not run performance tests. In each perf
test case, GDB_PERFTEST_MODE is checked. If it doesn't exist, return.
2) run perf tests easily. We add a new makefile target 'check-perf'.
gdb:
2013-11-06 Yao Qi <yao@codesourcery.com>
* Makefile.in (check-perf): New target.
gdb/testsuite:
2013-11-06 Yao Qi <yao@codesourcery.com>
* Makefile.in (check-perf): New target.
* configure.ac (AC_OUTPUT): Output Makefile in gdb.perf.
* configure: Re-generated.
* gdb.perf/Makefile.in: New.
Doug Evans
Doug Evans
Joel Brobecker
Yao Qi