Add selftests run filtering

With the growing number of selftests, I think it would be useful to be
able to run only a subset of the tests.  This patch associates a name to
each registered selftest.  It then allows doing something like:

  (gdb) maintenance selftest aarch64
  Running self-tests.
  Running selftest aarch64-analyze-prologue.
  Running selftest aarch64-process-record.
  Ran 2 unit tests, 0 failed

or with gdbserver:

  ./gdbserver --selftest=aarch64

In both cases, only the tests that contain "aarch64" in their name are
ran.  To help validate that the tests you want to run were actually ran,
it also prints a message with the test name before running each test.

Right now, all the arch-dependent tests are registered as a single test
of the selftests.  To be able to filter those too, I made them
"first-class citizen" selftests.  The selftest type is an interface,
with different implementations for "simple selftests" and "arch
selftests".  The run_tests function simply iterates on that an invokes
operator() on each test.

I changed the tests data structure from a vector to a map, because

  - it allows iterating in a stable (alphabetical) order
  - it allows to easily verify if a test with a given name has been
    registered, to avoid duplicates

There's also a new command "maintenance info selftests" that lists the
registered selftests.

gdb/ChangeLog:

	* common/selftest.h (selftest): New struct/interface.
	(register_test): Add name parameter, add new overload.
	(run_tests): Add filter parameter.
	(for_each_selftest_ftype): New typedef.
	(for_each_selftest): New declaration.
	* common/selftest.c (tests): Change type to
	map<string, unique_ptr<selftest>>.
	(simple_selftest): New struct.
	(register_test): New function.
	(register_test): Add name parameter and use it.
	(run_tests): Add filter parameter and use it.  Add prints.
	Adjust to vector -> map change.
	* aarch64-tdep.c (_initialize_aarch64_tdep): Add names when
	registering selftests.
	* arm-tdep.c (_initialize_arm_tdep): Likewise.
	* disasm-selftests.c (_initialize_disasm_selftests): Likewise.
	* dwarf2-frame.c (_initialize_dwarf2_frame): Likewise.
	* dwarf2loc.c (_initialize_dwarf2loc): Likewise.
	* findvar.c (_initialize_findvar): Likewise.
	* gdbarch-selftests.c (_initialize_gdbarch_selftests): Likewise.
	* maint.c (maintenance_selftest): Update call to run_tests.
	(maintenance_info_selftests): New function.
	(_initialize_maint_cmds): Register "maintenance info selftests"
	command.  Update "maintenance selftest" doc.
	* regcache.c (_initialize_regcache): Add names when registering
	selftests.
	* rust-exp.y (_initialize_rust_exp): Likewise.
	* selftest-arch.c (gdbarch_selftest): New struct.
	(gdbarch_tests): Remove.
	(register_test_foreach_arch): Add name parameter.  Call
	register_test.
	(tests_with_arch): Remove, move most content to
	gdbarch_selftest::operator().
	(_initialize_selftests_foreach_arch): Remove.
	* selftest-arch.h (register_test_foreach_arch): Add name
	parameter.
	(run_tests_with_arch): New declaration.
	* utils-selftests.c (_initialize_utils_selftests): Add names
	when registering selftests.
	* utils.c (_initialize_utils): Likewise.
	* unittests/array-view-selftests.c
	(_initialize_array_view_selftests): Likewise.
	* unittests/environ-selftests.c (_initialize_environ_selftests):
	Likewise.
	* unittests/function-view-selftests.c
	(_initialize_function_view_selftests): Likewise.
	* unittests/offset-type-selftests.c
	(_initialize_offset_type_selftests): Likewise.
	* unittests/optional-selftests.c
	(_initialize_optional_selftests): Likewise.
	* unittests/scoped_restore-selftests.c
	(_initialize_scoped_restore_selftests): Likewise.
	* NEWS: Document "maintenance selftest" and "maint info
	selftests".

gdb/gdbserver/ChangeLog:

	* server.c (captured_main): Accept argument for --selftest.
	Update run_tests call.
	* linux-x86-tdesc-selftest.c (initialize_low_tdesc): Add names
	when registering selftests.

gdb/doc/ChangeLog:

	* gdb.texinfo (Maintenance Commands): Document filter parameter
	of "maint selftest".  Document "maint info selftests" command.
This commit is contained in:
Simon Marchi 2017-09-16 14:06:03 +02:00
parent 5846367a24
commit 1526853e34
29 changed files with 287 additions and 108 deletions

View File

@ -1,3 +1,60 @@
2017-09-16 Simon Marchi <simon.marchi@ericsson.com>
* common/selftest.h (selftest): New struct/interface.
(register_test): Add name parameter, add new overload.
(run_tests): Add filter parameter.
(for_each_selftest_ftype): New typedef.
(for_each_selftest): New declaration.
* common/selftest.c (tests): Change type to
map<string, unique_ptr<selftest>>.
(simple_selftest): New struct.
(register_test): New function.
(register_test): Add name parameter and use it.
(run_tests): Add filter parameter and use it. Add prints.
Adjust to vector -> map change.
* aarch64-tdep.c (_initialize_aarch64_tdep): Add names when
registering selftests.
* arm-tdep.c (_initialize_arm_tdep): Likewise.
* disasm-selftests.c (_initialize_disasm_selftests): Likewise.
* dwarf2-frame.c (_initialize_dwarf2_frame): Likewise.
* dwarf2loc.c (_initialize_dwarf2loc): Likewise.
* findvar.c (_initialize_findvar): Likewise.
* gdbarch-selftests.c (_initialize_gdbarch_selftests): Likewise.
* maint.c (maintenance_selftest): Update call to run_tests.
(maintenance_info_selftests): New function.
(_initialize_maint_cmds): Register "maintenance info selftests"
command. Update "maintenance selftest" doc.
* regcache.c (_initialize_regcache): Add names when registering
selftests.
* rust-exp.y (_initialize_rust_exp): Likewise.
* selftest-arch.c (gdbarch_selftest): New struct.
(gdbarch_tests): Remove.
(register_test_foreach_arch): Add name parameter. Call
register_test.
(tests_with_arch): Remove, move most content to
gdbarch_selftest::operator().
(_initialize_selftests_foreach_arch): Remove.
* selftest-arch.h (register_test_foreach_arch): Add name
parameter.
(run_tests_with_arch): New declaration.
* utils-selftests.c (_initialize_utils_selftests): Add names
when registering selftests.
* utils.c (_initialize_utils): Likewise.
* unittests/array-view-selftests.c
(_initialize_array_view_selftests): Likewise.
* unittests/environ-selftests.c (_initialize_environ_selftests):
Likewise.
* unittests/function-view-selftests.c
(_initialize_function_view_selftests): Likewise.
* unittests/offset-type-selftests.c
(_initialize_offset_type_selftests): Likewise.
* unittests/optional-selftests.c
(_initialize_optional_selftests): Likewise.
* unittests/scoped_restore-selftests.c
(_initialize_scoped_restore_selftests): Likewise.
* NEWS: Document "maintenance selftest" and "maint info
selftests".
2017-09-16 Simon Marchi <simon.marchi@ericsson.com>
* mi/mi-main.c (mi_load_progress): Restore current_uiout using a

View File

@ -59,6 +59,9 @@ QStartupWithShell
* The "maintenance print c-tdesc" command now takes an optional
argument which is the file name of XML target description.
* The "maintenance selftest" command now takes an optional argument to
filter the tests to be run.
* New commands
set|show compile-gcc
@ -69,6 +72,9 @@ set debug separate-debug-file
show debug separate-debug-file
Control the display of debug output about separate debug file search.
maint info selftests
List the registered selftests.
* TUI Single-Key mode now supports two new shortcut keys: `i' for stepi and
`o' for nexti.

View File

@ -3065,8 +3065,10 @@ When on, AArch64 specific debugging is enabled."),
&setdebuglist, &showdebuglist);
#if GDB_SELF_TEST
selftests::register_test (selftests::aarch64_analyze_prologue_test);
selftests::register_test (selftests::aarch64_process_record_test);
selftests::register_test ("aarch64-analyze-prologue",
selftests::aarch64_analyze_prologue_test);
selftests::register_test ("aarch64-process-record",
selftests::aarch64_process_record_test);
#endif
}

View File

@ -9725,7 +9725,7 @@ vfp - VFP co-processor."),
&setdebuglist, &showdebuglist);
#if GDB_SELF_TEST
selftests::register_test (selftests::arm_record_test);
selftests::register_test ("arm-record", selftests::arm_record_test);
#endif
}

View File

@ -20,35 +20,72 @@
#include "common-exceptions.h"
#include "common-debug.h"
#include "selftest.h"
#include <vector>
#include <map>
namespace selftests
{
/* All the tests that have been registered. Using an std::map allows keeping
the order of tests stable and easily looking up whether a test name
exists. */
/* All the tests that have been registered. */
static std::map<std::string, std::unique_ptr<selftest>> tests;
static std::vector<self_test_function *> tests;
/* A selftest that calls the test function without arguments. */
struct simple_selftest : public selftest
{
simple_selftest (self_test_function *function_)
: function (function_)
{}
void operator() () const override
{
function ();
}
self_test_function *function;
};
/* See selftest.h. */
void
register_test (self_test_function *function)
register_test (const std::string &name, selftest *test)
{
tests.push_back (function);
/* Check that no test with this name already exist. */
gdb_assert (tests.find (name) == tests.end ());
tests[name] = std::unique_ptr<selftest> (test);
}
/* See selftest.h. */
void
run_tests (void)
register_test (const std::string &name, self_test_function *function)
{
int failed = 0;
register_test (name, new simple_selftest (function));
}
for (int i = 0; i < tests.size (); ++i)
/* See selftest.h. */
void
run_tests (const char *filter)
{
int ran = 0, failed = 0;
for (const auto &pair : tests)
{
const std::string &name = pair.first;
const std::unique_ptr<selftest> &test = pair.second;
if (filter != NULL && *filter != '\0'
&& name.find (filter) == std::string::npos)
continue;
TRY
{
tests[i] ();
debug_printf (_("Running selftest %s.\n"), name.c_str ());
++ran;
(*test) ();
}
CATCH (ex, RETURN_MASK_ERROR)
{
@ -60,7 +97,16 @@ run_tests (void)
reset ();
}
debug_printf ("Ran %lu unit tests, %d failed\n",
(long) tests.size (), failed);
debug_printf (_("Ran %d unit tests, %d failed\n"),
ran, failed);
}
/* See selftest.h. */
void for_each_selftest (for_each_selftest_ftype func)
{
for (const auto &pair : tests)
func (pair.first);
}
} // namespace selftests

View File

@ -27,18 +27,38 @@ typedef void self_test_function (void);
namespace selftests
{
/* Interface for the various kinds of selftests. */
struct selftest
{
virtual void operator() () const = 0;
};
/* Register a new self-test. */
extern void register_test (self_test_function *function);
extern void register_test (const std::string &name, selftest *test);
/* Register a new self-test. */
extern void register_test (const std::string &name,
self_test_function *function);
/* Run all the self tests. This print a message describing the number
of test and the number of failures. */
of test and the number of failures.
extern void run_tests (void);
If FILTER is not NULL and not empty, only tests with names containing FILTER
will be ran. */
extern void run_tests (const char *filter);
/* Reset GDB or GDBserver's internal state. */
extern void reset ();
typedef void for_each_selftest_ftype (const std::string &name);
/* Call FUNC for each registered selftest. */
extern void for_each_selftest (for_each_selftest_ftype func);
}
/* Check that VALUE is true, and, if not, throw an exception. */

View File

@ -211,7 +211,9 @@ void
_initialize_disasm_selftests (void)
{
#if GDB_SELF_TEST
selftests::register_test_foreach_arch (selftests::print_one_insn_test);
selftests::register_test_foreach_arch (selftests::memory_error_test);
selftests::register_test_foreach_arch ("print_one_insn",
selftests::print_one_insn_test);
selftests::register_test_foreach_arch ("memory_error",
selftests::memory_error_test);
#endif
}

View File

@ -1,3 +1,8 @@
2017-09-16 Simon Marchi <simon.marchi@ericsson.com>
* gdb.texinfo (Maintenance Commands): Document filter parameter
of "maint selftest". Document "maint info selftests" command.
2017-09-11 Tom Tromey <tom@tromey.com>
* python.texi (Events In Python): Document new events.

View File

@ -35067,8 +35067,16 @@ data structures, including its flags and contained types.
@kindex maint selftest
@cindex self tests
@item maint selftest @r{[}@var{filter}@r{]}
Run any self tests that were compiled in to @value{GDBN}. This will
print a message showing how many tests were run, and how many failed.
If a @var{filter} is passed, only the tests with @var{filter} in their
name will by ran.
@kindex "maint info selftests"
@cindex self tests
@item maint info selftests
List the selftests compiled in to @value{GDBN}.
@kindex maint set dwarf always-disassemble
@kindex maint show dwarf always-disassemble

View File

@ -2403,6 +2403,7 @@ _initialize_dwarf2_frame (void)
dwarf2_frame_objfile_data = register_objfile_data ();
#if GDB_SELF_TEST
selftests::register_test_foreach_arch (selftests::execute_cfa_program_test);
selftests::register_test_foreach_arch ("execute_cfa_program",
selftests::execute_cfa_program_test);
#endif
}

View File

@ -4671,6 +4671,6 @@ _initialize_dwarf2loc (void)
&setdebuglist, &showdebuglist);
#if GDB_SELF_TEST
selftests::register_test (selftests::copy_bitwise_tests);
selftests::register_test ("copy_bitwise", selftests::copy_bitwise_tests);
#endif
}

View File

@ -1095,6 +1095,8 @@ void
_initialize_findvar (void)
{
#if GDB_SELF_TEST
selftests::register_test (selftests::findvar_tests::copy_integer_to_size_test);
selftests::register_test
("copy_integer_to_size",
selftests::findvar_tests::copy_integer_to_size_test);
#endif
}

View File

@ -151,6 +151,7 @@ void
_initialize_gdbarch_selftests (void)
{
#if GDB_SELF_TEST
selftests::register_test_foreach_arch (selftests::register_to_value_test);
selftests::register_test_foreach_arch ("register_to_value",
selftests::register_to_value_test);
#endif
}

View File

@ -1,3 +1,10 @@
2017-09-16 Simon Marchi <simon.marchi@ericsson.com>
* server.c (captured_main): Accept argument for --selftest.
Update run_tests call.
* linux-x86-tdesc-selftest.c (initialize_low_tdesc): Add names
when registering selftests.
2017-09-16 Sergio Durigan Junior <sergiodj@redhat.com>
* regcache.c (get_thread_regcache): Update code to use "std::vector"

View File

@ -164,7 +164,7 @@ initialize_low_tdesc ()
init_registers_i386_avx_avx512_linux ();
init_registers_i386_avx_mpx_avx512_pku_linux ();
selftests::register_test (selftests::tdesc::i386_tdesc_test);
selftests::register_test ("i386-tdesc", selftests::tdesc::i386_tdesc_test);
#ifdef __x86_64__
init_registers_x32_linux ();
@ -178,6 +178,6 @@ initialize_low_tdesc ()
init_registers_amd64_avx_avx512_linux ();
init_registers_amd64_avx_mpx_avx512_pku_linux ();
selftests::register_test (selftests::tdesc::amd64_tdesc_test);
selftests::register_test ("amd64-tdesc", selftests::tdesc::amd64_tdesc_test);
#endif
}

View File

@ -3661,6 +3661,7 @@ captured_main (int argc, char *argv[])
volatile int attach = 0;
int was_running;
bool selftest = false;
const char *selftest_filter = NULL;
while (*next_arg != NULL && **next_arg == '-')
{
@ -3781,6 +3782,11 @@ captured_main (int argc, char *argv[])
run_once = 1;
else if (strcmp (*next_arg, "--selftest") == 0)
selftest = true;
else if (startswith (*next_arg, "--selftest="))
{
selftest = true;
selftest_filter = *next_arg + strlen ("--selftest=");
}
else
{
fprintf (stderr, "Unknown argument: %s\n", *next_arg);
@ -3857,7 +3863,7 @@ captured_main (int argc, char *argv[])
if (selftest)
{
selftests::run_tests ();
selftests::run_tests (selftest_filter);
throw_quit ("Quit");
}

View File

@ -957,7 +957,16 @@ show_per_command_cmd (char *args, int from_tty)
static void
maintenance_selftest (char *args, int from_tty)
{
selftests::run_tests ();
selftests::run_tests (args);
}
static void
maintenance_info_selftests (char *arg, int from_tty)
{
printf_filtered ("Registered selftests:\n");
selftests::for_each_selftest ([] (const std::string &name) {
printf_filtered (" - %s\n", name.c_str ());
});
}
@ -1140,11 +1149,14 @@ If you decide you want to use it: maintenance undeprecate 'commandname'"),
add_cmd ("selftest", class_maintenance, maintenance_selftest, _("\
Run gdb's unit tests.\n\
Usage: maintenance selftest\n\
Usage: maintenance selftest [filter]\n\
This will run any unit tests that were built in to gdb.\n\
gdb will abort if any test fails."),
If a filter is given, only the tests with that value in their name will ran."),
&maintenancelist);
add_cmd ("selftests", class_maintenance, maintenance_info_selftests,
_("List the registered selftests."), &maintenanceinfolist);
add_setshow_zinteger_cmd ("watchdog", class_maintenance, &watchdog, _("\
Set watchdog timer."), _("\
Show watchdog timer."), _("\

View File

@ -1773,7 +1773,8 @@ Print the internal register configuration including each register's\n\
remote register number and buffer offset in the g/G packets.\n\
Takes an optional file parameter."),
&maintenanceprintlist);
#if GDB_SELF_TEST
selftests::register_test (selftests::current_regcache_test);
selftests::register_test ("current_regcache", selftests::current_regcache_test);
#endif
}

View File

@ -2779,6 +2779,6 @@ _initialize_rust_exp (void)
gdb_assert (code == 0);
#if GDB_SELF_TEST
selftests::register_test (rust_lex_tests);
selftests::register_test ("rust-lex", rust_lex_tests);
#endif
}

View File

@ -25,12 +25,75 @@
namespace selftests {
static std::vector<self_test_foreach_arch_function *> gdbarch_tests;
/* A kind of selftest that calls the test function once for each gdbarch known
to GDB. */
struct gdbarch_selftest : public selftest
{
gdbarch_selftest (self_test_foreach_arch_function *function_)
: function (function_)
{}
void operator() () const override
{
const char **arches = gdbarch_printable_names ();
bool pass = true;
for (int i = 0; arches[i] != NULL; i++)
{
if (strcmp ("fr300", arches[i]) == 0)
{
/* PR 20946 */
continue;
}
else if (strcmp ("powerpc:EC603e", arches[i]) == 0
|| strcmp ("powerpc:e500mc", arches[i]) == 0
|| strcmp ("powerpc:e500mc64", arches[i]) == 0
|| strcmp ("powerpc:titan", arches[i]) == 0
|| strcmp ("powerpc:vle", arches[i]) == 0
|| strcmp ("powerpc:e5500", arches[i]) == 0
|| strcmp ("powerpc:e6500", arches[i]) == 0)
{
/* PR 19797 */
continue;
}
QUIT;
TRY
{
struct gdbarch_info info;
gdbarch_info_init (&info);
info.bfd_arch_info = bfd_scan_arch (arches[i]);
struct gdbarch *gdbarch = gdbarch_find_by_info (info);
SELF_CHECK (gdbarch != NULL);
function (gdbarch);
}
CATCH (ex, RETURN_MASK_ERROR)
{
pass = false;
exception_fprintf (gdb_stderr, ex,
_("Self test failed: arch %s: "), arches[i]);
}
END_CATCH
reset ();
}
SELF_CHECK (pass);
}
self_test_foreach_arch_function *function;
};
void
register_test_foreach_arch (self_test_foreach_arch_function *function)
register_test_foreach_arch (const std::string &name,
self_test_foreach_arch_function *function)
{
gdbarch_tests.push_back (function);
register_test (name, new gdbarch_selftest (function));
}
void
@ -40,70 +103,5 @@ reset ()
registers_changed ();
reinit_frame_cache ();
}
static void
tests_with_arch ()
{
int failed = 0;
for (const auto &f : gdbarch_tests)
{
const char **arches = gdbarch_printable_names ();
for (int i = 0; arches[i] != NULL; i++)
{
if (strcmp ("fr300", arches[i]) == 0)
{
/* PR 20946 */
continue;
}
else if (strcmp ("powerpc:EC603e", arches[i]) == 0
|| strcmp ("powerpc:e500mc", arches[i]) == 0
|| strcmp ("powerpc:e500mc64", arches[i]) == 0
|| strcmp ("powerpc:titan", arches[i]) == 0
|| strcmp ("powerpc:vle", arches[i]) == 0
|| strcmp ("powerpc:e5500", arches[i]) == 0
|| strcmp ("powerpc:e6500", arches[i]) == 0)
{
/* PR 19797 */
continue;
}
QUIT;
TRY
{
struct gdbarch_info info;
gdbarch_info_init (&info);
info.bfd_arch_info = bfd_scan_arch (arches[i]);
struct gdbarch *gdbarch = gdbarch_find_by_info (info);
SELF_CHECK (gdbarch != NULL);
f (gdbarch);
}
CATCH (ex, RETURN_MASK_ERROR)
{
++failed;
exception_fprintf (gdb_stderr, ex,
_("Self test failed: arch %s: "), arches[i]);
}
END_CATCH
reset ();
}
}
SELF_CHECK (failed == 0);
}
} // namespace selftests
#endif /* GDB_SELF_TEST */
void
_initialize_selftests_foreach_arch ()
{
#if GDB_SELF_TEST
selftests::register_test (selftests::tests_with_arch);
#endif
}

View File

@ -24,7 +24,8 @@ typedef void self_test_foreach_arch_function (struct gdbarch *);
namespace selftests
{
extern void
register_test_foreach_arch (self_test_foreach_arch_function *function);
register_test_foreach_arch (const std::string &name,
self_test_foreach_arch_function *function);
}
#endif /* SELFTEST_ARCH_H */

View File

@ -491,5 +491,6 @@ run_tests ()
void
_initialize_array_view_selftests ()
{
selftests::register_test (selftests::array_view_tests::run_tests);
selftests::register_test ("array_view",
selftests::array_view_tests::run_tests);
}

View File

@ -301,5 +301,6 @@ run_tests ()
void
_initialize_environ_selftests ()
{
selftests::register_test (selftests::gdb_environ_tests::run_tests);
selftests::register_test ("gdb_environ",
selftests::gdb_environ_tests::run_tests);
}

View File

@ -174,5 +174,6 @@ run_tests ()
void
_initialize_function_view_selftests ()
{
selftests::register_test (selftests::function_view::run_tests);
selftests::register_test ("function_view",
selftests::function_view::run_tests);
}

View File

@ -174,5 +174,5 @@ run_tests ()
void
_initialize_offset_type_selftests ()
{
selftests::register_test (selftests::offset_type::run_tests);
selftests::register_test ("offset_type", selftests::offset_type::run_tests);
}

View File

@ -90,5 +90,5 @@ run_tests ()
void
_initialize_optional_selftests ()
{
selftests::register_test (selftests::optional::run_tests);
selftests::register_test ("optional", selftests::optional::run_tests);
}

View File

@ -106,5 +106,6 @@ run_tests ()
void
_initialize_scoped_restore_selftests ()
{
selftests::register_test (selftests::scoped_restore_tests::run_tests);
selftests::register_test ("scoped_restore",
selftests::scoped_restore_tests::run_tests);
}

View File

@ -55,6 +55,6 @@ void
_initialize_utils_selftests (void)
{
#if GDB_SELF_TEST
selftests::register_test (selftests::common_utils_tests);
selftests::register_test ("common-utils", selftests::common_utils_tests);
#endif
}

View File

@ -3267,6 +3267,6 @@ _initialize_utils (void)
add_internal_problem_command (&demangler_warning_problem);
#if GDB_SELF_TEST
selftests::register_test (gdb_realpath_tests);
selftests::register_test ("gdb_realpath", gdb_realpath_tests);
#endif
}