4dd9d9db1d
2007-11-13 Benjamin Kosnik <bkoz@redhat.com> * docs/html/documentation.html: First pass at unified table of contents. * docs/html/abi.html: Move... * docs/html/17_intro/abi.html: ...here. * docs/html/17_intro/porting-howto.html: Update, edit, put resulting pieces into... * docs/html/17_intro/api.html: New. * docs/html/17_intro/c++0x_status.html: New. * docs/html/17_intro/CHECKLIST: Move to... * docs/html/17_intro/c++1998_status.html: ...here. * docs/html/ext/tr1.html: Move ... * docs/html/17_intro/tr1_status.html: ...here. * docs/html/debug_mode.html: Move... * docs/html/ext/debug_mode.html: ...here. * docs/html/parallel_mode.html: Move... * docs/html/ext/parallel_mode.html: ...here * docs/html/17_intro/BUGS: Remove. * docs/html/17_intro/concept_check.diff: Remove. * docs/html/17_intro/HEADER_POLICY: Remove. * docs/html/17_intro/headers_cc.txt: Remove. * docs/html/17_intro/PROBLEMS: Remove. * docs/html/17_intro/RELEASE-NOTES: Remove. * docs/html/explanations.html: Remove. * docs/html/makedoc.awk: Remove. * docs/html/faq/index.txt: Remove. HTML only. * /docs/html/Makefile: Remove. * docs/html/17_intro/configury.html: Editing, updating, consistency check with doxygen conventions. Change libstdc++-v3 to libstdc++. * docs/html/17_intro/howto.html: Same. * docs/html/17_intro/license.html: Same. * docs/html/17_intro/porting.html: Same. * docs/html/18_support/howto.html: Same. * docs/html/19_diagnostics/howto.html: Same. * docs/html/20_util/allocator.html: Same. * docs/html/20_util/howto.html: Same. * docs/html/21_strings/howto.html: Same. * docs/html/22_locale/codecvt.html: Same. * docs/html/22_locale/ctype.html: Same. * docs/html/22_locale/howto.html: Same. * docs/html/22_locale/messages.html: Same. * docs/html/23_containers/howto.html: Same. * docs/html/24_iterators/howto.html: Same. * docs/html/25_algorithms/howto.html: Same. * docs/html/26_numerics/howto.html: Same. * docs/html/27_io/howto.html: Same. * docs/html/configopts.html: Same. * docs/html/debug.html: Same. * docs/html/ext/ballocator_doc.html: Same. * docs/html/ext/howto.html: Same. * docs/html/ext/mt_allocator.html: Same. * docs/html/ext/sgiexts.html: Same. * docs/html/faq/index.html: Same. * docs/html/install.html: Same. * docs/html/test.html: Same. * include/bits/c++config: Change _GLIBCXX_DEPRECATED to _GLIBCXX_DEPRECATED_ATTR, _GLIBCXX_VISIBILITY to _GLIBCXX_VISIBILITY_ATTR. * include/backward/auto_ptr.h: Same. * include/backward/binders.h: Same. * include/bits/stl_function.h: Same. * include/std/memory: Same. * include/std/streambuf: Same. * include/tr1_impl/boost_shared_ptr.h: Same. * src/globals_io.cc: Same. * src/ios_init.cc: Same. From-SVN: r130150
720 lines
24 KiB
HTML
720 lines
24 KiB
HTML
<?xml version="1.0" encoding="ISO-8859-1"?>
|
|
<!DOCTYPE html
|
|
PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
|
|
<head>
|
|
<meta name="AUTHOR" content="bkoz@gcc.gnu.org (Benjamin Kosnik)" />
|
|
<meta name="KEYWORDS" content="c++, libstdc++, test, regression, g++" />
|
|
<meta name="DESCRIPTION" content="README for the GNU libstdc++ effort." />
|
|
<meta name="GENERATOR" content="vi and eight fingers" />
|
|
<title>libstdc++ Testing Instructions</title>
|
|
<link rel="StyleSheet" href="lib3styles.css" />
|
|
</head>
|
|
<body>
|
|
|
|
<h1 class="centered"><a name="top">Testing Details</a></h1>
|
|
|
|
<p class="fineprint"><em>
|
|
The latest version of this document is always available at
|
|
<a href="http://gcc.gnu.org/onlinedocs/libstdc++/test.html">
|
|
http://gcc.gnu.org/onlinedocs/libstdc++/test.html</a>.
|
|
</em></p>
|
|
|
|
<p><em>
|
|
To the <a href="http://gcc.gnu.org/libstdc++/">libstdc++ homepage</a>.
|
|
</em></p>
|
|
|
|
<!-- ####################################################### -->
|
|
<hr />
|
|
<h2>Contents</h2>
|
|
<ul>
|
|
<li><a href="#org">Testsuite organization and naming conventions</a></li>
|
|
<li><a href="#util">Utilities: abicheck and libtestc++</a></li>
|
|
<li><a href="#new">How to write a new test case</a></li>
|
|
<li><a href="#check">Options for running the tests</a></li>
|
|
<li><a href="#debug">Running debug-mode tests</a></li>
|
|
<li><a href="#future">Future</a></li>
|
|
<li><a href="#internals">DejaGNU internals</a></li>
|
|
</ul>
|
|
|
|
<hr />
|
|
|
|
<!-- ####################################################### -->
|
|
|
|
<h2><a name="org">Testsuite organization and naming conventions</a></h2>
|
|
<p>
|
|
The directory <em>libsrcdir/testsuite</em> contains the
|
|
individual test cases organized in sub-directories corresponding
|
|
to chapters of the C++ standard (detailed below), the dejagnu
|
|
test harness support files, and sources to various testsuite
|
|
utilities that are packaged in a separate testing library.
|
|
</p>
|
|
|
|
<p> All test cases for functionality required by the runtime
|
|
components of the C++ standard (ISO 14882) are files within the
|
|
following directories.
|
|
</p>
|
|
|
|
<pre>
|
|
17_intro
|
|
18_support
|
|
19_diagnostics
|
|
20_util
|
|
21_strings
|
|
22_locale
|
|
23_containers
|
|
25_algorithms
|
|
26_numerics
|
|
27_io
|
|
</pre>
|
|
|
|
<p>
|
|
In addition, the following directories include test files:
|
|
</p>
|
|
|
|
<pre>
|
|
tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1).
|
|
backward Tests for backwards compatibility and deprecated features.
|
|
demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler
|
|
ext Tests for extensions.
|
|
performance Tests for performance analysis, and performance regressions.
|
|
thread Tests for threads.
|
|
</pre>
|
|
|
|
<p>
|
|
Some directories don't have test files, but instead contain
|
|
auxiliary information (<a href="#internals">more information</a>):
|
|
</p>
|
|
|
|
<pre>
|
|
config Files for the dejagnu test harness.
|
|
lib Files for the dejagnu test harness.
|
|
libstdc++* Files for the dejagnu test harness.
|
|
data Sample text files for testing input and output.
|
|
util Files for libtestc++, utilities and testing routines.
|
|
</pre>
|
|
|
|
<p>
|
|
Within a directory that includes test files, there may be
|
|
additional subdirectories, or files. Originally, test cases
|
|
were appended to one file that represented a particular section
|
|
of the chapter under test, and was named accordingly. For
|
|
instance, to test items related to <code> 21.3.6.1 -
|
|
basic_string::find [lib.string::find]</code> in the standard,
|
|
the following was used:
|
|
</p>
|
|
<pre>
|
|
21_strings/find.cc
|
|
</pre>
|
|
<p>
|
|
However, that practice soon became a liability as the test cases
|
|
became huge and unwieldy, and testing new or extended
|
|
functionality (like wide characters or named locales) became
|
|
frustrating, leading to aggressive pruning of test cases on some
|
|
platforms that covered up implementation errors. Now, the test
|
|
suite has a policy of one file, one test case, which solves the
|
|
above issues and gives finer grained results and more manageable
|
|
error debugging. As an example, the test case quoted above
|
|
becomes:
|
|
</p>
|
|
<pre>
|
|
21_strings/basic_string/find/char/1.cc
|
|
21_strings/basic_string/find/char/2.cc
|
|
21_strings/basic_string/find/char/3.cc
|
|
21_strings/basic_string/find/wchar_t/1.cc
|
|
21_strings/basic_string/find/wchar_t/2.cc
|
|
21_strings/basic_string/find/wchar_t/3.cc
|
|
</pre>
|
|
|
|
<p>
|
|
All new tests should be written with the policy of one test
|
|
case, one file in mind.
|
|
</p>
|
|
|
|
<p>
|
|
In addition, there are some special names and suffixes that are
|
|
used within the testsuite to designate particular kinds of
|
|
tests.
|
|
</p>
|
|
|
|
<ul>
|
|
<li>
|
|
<em>_xin.cc</em>
|
|
<p>
|
|
This test case expects some kind of interactive input in order
|
|
to finish or pass. At the moment, the interactive tests are not
|
|
run by default. Instead, they are run by hand, like:
|
|
</p>
|
|
<pre>
|
|
g++ 27_io/objects/char/3_xin.cc
|
|
cat 27_io/objects/char/3_xin.in | a.out
|
|
</pre>
|
|
</li>
|
|
<li>
|
|
<em>.in</em>
|
|
<p>
|
|
This file contains the expected input for the corresponding <em>
|
|
_xin.cc</em> test case.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>_neg.cc</em>
|
|
<p>
|
|
This test case is expected to fail: it's a negative test. At the
|
|
moment, these are almost always compile time errors.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>char</em>
|
|
<p>
|
|
This can either be a directory name or part of a longer file
|
|
name, and indicates that this file, or the files within this
|
|
directory are testing the <code>char</code> instantiation of a
|
|
template.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>wchar_t</em>
|
|
<p>
|
|
This can either be a directory name or part of a longer file
|
|
name, and indicates that this file, or the files within this
|
|
directory are testing the <code>wchar_t</code> instantiation of
|
|
a template. Some hosts do not support <code>wchar_t</code>
|
|
functionality, so for these targets, all of these tests will not
|
|
be run.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>thread</em>
|
|
<p>
|
|
This can either be a directory name or part of a longer file
|
|
name, and indicates that this file, or the files within this
|
|
directory are testing situations where multiple threads are
|
|
being used.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>performance</em>
|
|
<p>
|
|
This can either be an enclosing directory name or part of a
|
|
specific file name. This indicates a test that is used to
|
|
analyze runtime performance, for performance regression testing,
|
|
or for other optimization related analysis. At the moment, these
|
|
test cases are not run by default.
|
|
</p>
|
|
</li>
|
|
</ul>
|
|
|
|
<hr />
|
|
<h2><a name="util">Utilities: abi_check and libtestc++</a></h2>
|
|
<p>
|
|
The testsuite directory also contains some files that implement
|
|
functionality that is intended to make writing test cases easier,
|
|
or to avoid duplication, or to provide error checking in a way that
|
|
is consistent across platforms and test harnesses. A stand-alone
|
|
executable, called <em>abi_check</em>, and a static library called
|
|
<em>libtestc++</em> are constructed. Both of these items are not
|
|
installed, and only used during testing.
|
|
</p>
|
|
|
|
<p>
|
|
These files include the following functionality:
|
|
</p>
|
|
|
|
<ul>
|
|
<li>
|
|
<em>testsuite_abi.h</em>,
|
|
<em>testsuite_abi.cc</em>,
|
|
<em>testsuite_abi_check.cc</em>
|
|
<p>
|
|
Creates the executable <em>abi_check</em>.
|
|
Used to check correctness of symbol versioning, visibility of
|
|
exported symbols, and compatibility on symbols in the shared
|
|
library, for hosts that support this feature. More information
|
|
can be found in the ABI documentation <a href="abi.html"> here</a>
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>testsuite_allocator.h</em>,
|
|
<em>testsuite_allocator.cc</em>
|
|
<p>
|
|
Contains specialized allocators that keep track of construction
|
|
and destruction. Also, support for overriding global new and
|
|
delete operators, including verification that new and delete
|
|
are called during execution, and that allocation over max_size
|
|
fails.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>testsuite_character.h</em>
|
|
<p>
|
|
Contains <code>std::char_traits</code> and
|
|
<code>std::codecvt</code> specializations for a user-defined
|
|
POD.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>testsuite_hooks.h</em>,
|
|
<em>testsuite_hooks.cc</em>
|
|
<p>
|
|
A large number of utilities, including:
|
|
</p>
|
|
<ul>
|
|
<li>VERIFY</li>
|
|
<li>set_memory_limits</li>
|
|
<li>verify_demangle</li>
|
|
<li>run_tests_wrapped_locale</li>
|
|
<li>run_tests_wrapped_env</li>
|
|
<li>try_named_locale</li>
|
|
<li>try_mkfifo</li>
|
|
<li>func_callback</li>
|
|
<li>counter</li>
|
|
<li>copy_tracker</li>
|
|
<li>copy_constructor</li>
|
|
<li>assignment_operator</li>
|
|
<li>destructor</li>
|
|
<li>pod_char, pod_int and associated char_traits specializations</li>
|
|
</ul>
|
|
<p></p>
|
|
</li>
|
|
<li>
|
|
<em>testsuite_io.h</em>
|
|
<p>
|
|
Error, exception, and constraint checking for
|
|
<code>std::streambuf, std::basic_stringbuf, std::basic_filebuf</code>.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>testsuite_iterators.h</em>
|
|
<p>
|
|
Wrappers for various iterators.
|
|
</p>
|
|
</li>
|
|
<li>
|
|
<em>testsuite_performance.h</em>
|
|
<p>
|
|
A number of class abstractions for performance counters, and
|
|
reporting functions including:
|
|
</p>
|
|
<ul>
|
|
<li>time_counter</li>
|
|
<li>resource_counter</li>
|
|
<li>report_performance</li>
|
|
</ul>
|
|
<p></p>
|
|
</li>
|
|
</ul>
|
|
|
|
<hr />
|
|
<h2><a name="new">How to write a new test case</a></h2>
|
|
|
|
<p>
|
|
The first step in making a new test case is to choose the correct
|
|
directory and file name, given the organization as previously
|
|
described.
|
|
</p>
|
|
|
|
<p>
|
|
All files are copyright the FSF, and GPL'd: this is very
|
|
important. The first copyright year should correspond to the date
|
|
the file was checked in to SVN.
|
|
</p>
|
|
|
|
<p>
|
|
As per the dejagnu instructions, always return 0 from main to
|
|
indicate success.
|
|
</p>
|
|
|
|
<p>
|
|
A bunch of utility functions and classes have already been
|
|
abstracted out into the testsuite utility library, <code>
|
|
libtestc++</code>. To use this functionality, just include the
|
|
appropriate header file: the library or specific object files will
|
|
automatically be linked in as part of the testsuite run.
|
|
</p>
|
|
|
|
<p>
|
|
For a test that needs to take advantage of the dejagnu test
|
|
harness, what follows below is a list of special keyword that
|
|
harness uses. Basically, a test case contains dg-keywords (see
|
|
dg.exp) indicating what to do and what kinds of behavior are to be
|
|
expected. New test cases should be written with the new style
|
|
DejaGnu framework in mind.
|
|
</p>
|
|
|
|
<p>
|
|
To ease transition, here is the list of dg-keyword documentation
|
|
lifted from dg.exp.
|
|
</p>
|
|
|
|
<pre>
|
|
# The currently supported options are:
|
|
#
|
|
# dg-prms-id N
|
|
# set prms_id to N
|
|
#
|
|
# dg-options "options ..." [{ target selector }]
|
|
# specify special options to pass to the tool (eg: compiler)
|
|
#
|
|
# dg-do do-what-keyword [{ target/xfail selector }]
|
|
# `do-what-keyword' is tool specific and is passed unchanged to
|
|
# ${tool}-dg-test. An example is gcc where `keyword' can be any of:
|
|
# preprocess|compile|assemble|link|run
|
|
# and will do one of: produce a .i, produce a .s, produce a .o,
|
|
# produce an a.out, or produce an a.out and run it (the default is
|
|
# compile).
|
|
#
|
|
# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
|
|
# indicate an error message <regexp> is expected on this line
|
|
# (the test fails if it doesn't occur)
|
|
# Linenum=0 for general tool messages (eg: -V arg missing).
|
|
# "." means the current line.
|
|
#
|
|
# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
|
|
# indicate a warning message <regexp> is expected on this line
|
|
# (the test fails if it doesn't occur)
|
|
#
|
|
# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
|
|
# indicate a bogus error message <regexp> use to occur here
|
|
# (the test fails if it does occur)
|
|
#
|
|
# dg-build regexp comment [{ target/xfail selector }]
|
|
# indicate the build use to fail for some reason
|
|
# (errors covered here include bad assembler generated, tool crashes,
|
|
# and link failures)
|
|
# (the test fails if it does occur)
|
|
#
|
|
# dg-excess-errors comment [{ target/xfail selector }]
|
|
# indicate excess errors are expected (any line)
|
|
# (this should only be used sparingly and temporarily)
|
|
#
|
|
# dg-output regexp [{ target selector }]
|
|
# indicate the expected output of the program is <regexp>
|
|
# (there may be multiple occurrences of this, they are concatenated)
|
|
#
|
|
# dg-final { tcl code }
|
|
# add some tcl code to be run at the end
|
|
# (there may be multiple occurrences of this, they are concatenated)
|
|
# (unbalanced braces must be \-escaped)
|
|
#
|
|
# "{ target selector }" is a list of expressions that determine whether the
|
|
# test succeeds or fails for a particular target, or in some cases whether the
|
|
# option applies for a particular target. If the case of `dg-do' it specifies
|
|
# whether the test case is even attempted on the specified target.
|
|
#
|
|
# The target selector is always optional. The format is one of:
|
|
#
|
|
# { xfail *-*-* ... } - the test is expected to fail for the given targets
|
|
# { target *-*-* ... } - the option only applies to the given targets
|
|
#
|
|
# At least one target must be specified, use *-*-* for "all targets".
|
|
# At present it is not possible to specify both `xfail' and `target'.
|
|
# "native" may be used in place of "*-*-*".
|
|
|
|
Example 1: Testing compilation only
|
|
// { dg-do compile }
|
|
|
|
Example 2: Testing for expected warnings on line 36, which all targets fail
|
|
// { dg-warning "string literals" "" { xfail *-*-* } 36
|
|
|
|
Example 3: Testing for expected warnings on line 36
|
|
// { dg-warning "string literals" "" { target *-*-* } 36
|
|
|
|
Example 4: Testing for compilation errors on line 41
|
|
// { dg-do compile }
|
|
// { dg-error "no match for" "" { target *-*-* } 41 }
|
|
|
|
Example 5: Testing with special command line settings, or without the
|
|
use of pre-compiled headers, in particular the stdc++.h.gch file. Any
|
|
options here will override the DEFAULT_CXXFLAGS and PCH_CXXFLAGS set
|
|
up in the normal.exp file.
|
|
// { dg-options "-O0" { target *-*-* } }
|
|
</pre>
|
|
|
|
<p>
|
|
More examples can be found in the libstdc++/testsuite/*/*.cc files.
|
|
</p>
|
|
|
|
<hr />
|
|
<h2><a name="check">Options for running the tests</a></h2>
|
|
|
|
<p> There are several options for running tests, including testing
|
|
the regression tests, testing a subset of the regression tests,
|
|
testing the performance tests, testing just compilation, testing
|
|
installed tools, etc. In addition, there is a special rule for
|
|
checking the exported symbols of the shared library.
|
|
</p>
|
|
|
|
<p>You can check the status of the build without installing it
|
|
using the dejagnu harness, much like the rest of the gcc tools.</p>
|
|
<pre> make check</pre>
|
|
<p>in the <em>libbuilddir</em> directory.</p>
|
|
<p>or</p>
|
|
<pre> make check-target-libstdc++</pre>
|
|
<p>in the <em>gccbuilddir</em> directory.</p>
|
|
|
|
<p>
|
|
These commands are functionally equivalent and will create a
|
|
'testsuite' directory underneath <em>libbuilddir</em> containing
|
|
the results of the tests. Two results files will be generated:
|
|
<em> libstdc++.sum</em>, which is a PASS/FAIL summary for each
|
|
test, and <em>libstdc++.log</em> which is a log of the exact
|
|
command line passed to the compiler, the compiler output, and
|
|
the executable output (if any).
|
|
</p>
|
|
|
|
|
|
<p>
|
|
To debug the dejagnu test harness during runs, try invoking with a
|
|
specific argument to the variable RUNTESTFLAGS, as below.
|
|
</p>
|
|
|
|
<pre>
|
|
make check-target-libstdc++ RUNTESTFLAGS="-v"
|
|
</pre>
|
|
or
|
|
<pre>
|
|
make check-target-libstdc++ RUNTESTFLAGS="-v -v"
|
|
</pre>
|
|
|
|
<p> To run a subset of the library tests, try using a command like the
|
|
following from the <em>libbuilddir/testsuite</em> directory:
|
|
</p>
|
|
<pre>
|
|
runtest --tool libstdc++ normal.exp="`find $srcdir/17_intro -name *.cc`"
|
|
</pre>
|
|
|
|
|
|
<p>
|
|
There are two ways to run on a simulator: set up DEJAGNU to point to a
|
|
specially crafted site.exp, or pass down --target_board flags.
|
|
</p>
|
|
Example flags to pass down for various embedded builds are as follows:
|
|
<pre>
|
|
--target=powerpc-eabism (libgloss/sim)
|
|
make check-target-libstdc++ RUNTESTFLAGS="--target_board=powerpc-sim"
|
|
|
|
--target=calmrisc32 (libgloss/sid)
|
|
make check-target-libstdc++ RUNTESTFLAGS="--target_board=calmrisc32-sid"
|
|
|
|
--target=xscale-elf (newlib/sim)
|
|
make check-target-libstdc++ RUNTESTFLAGS="--target_board=arm-sim"
|
|
</pre>
|
|
|
|
<p> Also, here is an example of how to run the libstdc++ testsuite for a
|
|
multilibed build directory with different ABI settings:
|
|
</p>
|
|
<pre>
|
|
make check-target-libstdc++ RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"'
|
|
</pre>
|
|
|
|
<p>
|
|
You can run the tests with a compiler and library that have already
|
|
been installed. Make sure that the compiler (e.g., <code>g++</code>)
|
|
is in your <code>PATH</code>. If you are using shared libraries, then
|
|
you must also ensure that the directory containing the shared version
|
|
of libstdc++ is in your <code>LD_LIBRARY_PATH</code>, or equivalent.
|
|
If your GCC source tree is at <code>/path/to/gcc</code>, then you can
|
|
run the tests as follows:
|
|
<pre>
|
|
runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++/testsuite
|
|
</pre>
|
|
The testsuite will create a number of files in the directory in which you
|
|
run this command,. Some of those files might use the same name as
|
|
files created by other testsuites (like the ones for GCC and G++), so
|
|
you should not try to run all the testsuites in parallel from the same
|
|
directory.
|
|
</p>
|
|
|
|
<p> In addition, there are some testing options that are mostly of
|
|
interest to library maintainers and system integrators. As such,
|
|
these tests may not work on all cpu and host combinations, and may need to
|
|
be executed in the <em>libbuilddir/testsuite</em> directory. These options
|
|
include, but are not necessarily limited to, the following:
|
|
</p>
|
|
|
|
<pre>
|
|
make testsuite_files</pre>
|
|
<p>
|
|
Five files are generated that determine what test files
|
|
are run. These files are:
|
|
</p>
|
|
<ul>
|
|
<li>
|
|
<em>testsuite_files </em>
|
|
<p> This is a list of all the test cases that will be run. Each
|
|
test case is on a separate line, given with an absolute path
|
|
from the <em>libsrcdir/testsuite</em> directory.
|
|
</p>
|
|
</li>
|
|
|
|
<li>
|
|
<em>testsuite_files_interactive </em>
|
|
<p> This is a list of all the interactive test cases, using the
|
|
same format as the file list above. These tests are not run by default.
|
|
</p>
|
|
</li>
|
|
|
|
<li>
|
|
<em>testsuite_files_performance</em>
|
|
<p> This is a list of all the performance test cases, using the
|
|
same format as the file list above. These tests are not run by default.
|
|
</p>
|
|
</li>
|
|
|
|
<li>
|
|
<em>testsuite_thread</em>
|
|
<p> This file indicates that the host system can run tests which
|
|
incolved multiple threads.
|
|
</p>
|
|
</li>
|
|
|
|
<li>
|
|
<em>testsuite_wchar_t</em>
|
|
<p> This file indicates that the host system can run the wchar_t
|
|
tests, and corresponds to the macro definition <code>
|
|
_GLIBCXX_USE_WCHAR_T</code> in the file c++config.h.
|
|
</p>
|
|
</li>
|
|
</ul>
|
|
|
|
<pre>
|
|
make check-abi</pre>
|
|
<p>The library ABI can be tested. This involves testing the shared
|
|
library against an ABI-defining previous version of symbol exports. </p>
|
|
|
|
<pre>
|
|
make check-compile</pre>
|
|
<p>This rule compiles, but does not link or execute, the
|
|
<em>testsuite_files</em> test cases and displays the output on stdout.</p>
|
|
|
|
<pre>
|
|
make check-performance</pre>
|
|
<p>This rule runs through the <em>testsuite_files_performance</em>
|
|
test cases and collects information for performance analysis and
|
|
can be used to spot performance regressions. Various timing
|
|
information is collected, as well as number of hard page faults,
|
|
and memory used. This is not run by default, and the implementation
|
|
is in flux.
|
|
</p>
|
|
|
|
<p>
|
|
We are interested in any strange failures of the
|
|
testsuite; please see <a href="faq/index.html#2_4">FAQ 2.4</a>
|
|
for which files to examine.
|
|
</p>
|
|
|
|
<hr/>
|
|
<h2><a name="debug">Running debug-mode tests</a></h2>
|
|
<p>To run the libstdc++ test suite under the <a
|
|
href="debug.html#safe">debug mode</a>,
|
|
edit <code>libstdc++/scripts/testsuite_flags</code> to add the
|
|
compile-time flag <code>-D_GLIBCXX_DEBUG</code> to the result
|
|
printed by the <code>--build-cxx</code> option. Additionally, add
|
|
the <code>-D_GLIBCXX_DEBUG_PEDANTIC</code> flag to turn on pedantic
|
|
checking. The libstdc++ test suite should produce precisely the same
|
|
results under debug mode that it does under release mode: any
|
|
deviation indicates an error in either the library or the test
|
|
suite.</p>
|
|
|
|
<hr />
|
|
<h2><a name="future">Future</a></h2>
|
|
|
|
<p>
|
|
Shared runs need to be implemented, for targets that support shared libraries.
|
|
</p>
|
|
|
|
<p>
|
|
Diffing of expected output to standard streams needs to be finished off.
|
|
</p>
|
|
|
|
<p>
|
|
The V3 testing framework supports, or will eventually support,
|
|
additional keywords for the purpose of easing the job of writing
|
|
test cases. All V3-keywords are of the form <code>@xxx@</code>.
|
|
Currently plans for supported keywords include:
|
|
</p>
|
|
|
|
<dl>
|
|
<dt> <code> @require@ <files> </code> </dt>
|
|
<dd>
|
|
<p>
|
|
The existence of <files> is essential for the test to complete
|
|
successfully. For example, a test case foo.C using bar.baz as
|
|
input file could say
|
|
</p>
|
|
<pre>
|
|
// @require@ bar.baz</pre>
|
|
<p>
|
|
The special variable % stands for the rootname, e.g. the
|
|
file-name without its `.C' extension. Example of use (taken
|
|
verbatim from 27_io/filebuf.cc)
|
|
</p>
|
|
<pre>
|
|
// @require@ %-*.tst %-*.txt</pre>
|
|
</dd>
|
|
<dt> <code> @diff@ <first-list> <second-list> </code> </dt>
|
|
<dd>
|
|
<p>
|
|
After the test case compiles and ran successfully, diff
|
|
<first-list> against <second-list>, these lists should
|
|
have the same length. The test fails if diff returns non-zero a
|
|
pair of files.
|
|
</p>
|
|
</dd>
|
|
</dl>
|
|
|
|
<hr />
|
|
<h2><a name="internals">DejaGNU internals</a></h2>
|
|
|
|
<p>This is information for those looking at making changes to the testsuite
|
|
structure, and/or needing to trace dejagnu's actions with --verbose. This
|
|
will not be useful to people who are "merely" adding new tests to the existing
|
|
structure.
|
|
</p>
|
|
|
|
<p>The first key point when working with dejagnu is the idea of a "tool".
|
|
Files, directories, and functions are all implicitly used when they are
|
|
named after the tool in use. Here, the tool will always be "libstdc++".
|
|
</p>
|
|
|
|
<p>The <code>lib</code> subdir contains support routines. The
|
|
<code>lib/libstdc++.exp</code> file ("support library") is loaded
|
|
automagically, and must explicitly load the others. For example, files can
|
|
be copied from the core compiler's support directory into <code>lib</code>.
|
|
</p>
|
|
|
|
<p>Some routines in <code>lib/libstdc++.exp</code> are callbacks, some are
|
|
our own. Callbacks must be prefixed with the name of the tool. To easily
|
|
distinguish the others, by convention our own routines are named "v3-*".
|
|
</p>
|
|
|
|
<p>The next key point when working with dejagnu is "test files". Any
|
|
directory whose name starts with the tool name will be searched for test files.
|
|
(We have only one.) In those directories, any <code>.exp</code> file is
|
|
considered a test file, and will be run in turn. Our main test file is called
|
|
<code>normal.exp</code>; it runs all the tests in testsuite_files using the
|
|
callbacks loaded from the support library.
|
|
</p>
|
|
|
|
<p>The <code>config</code> directory is searched for any particular "target
|
|
board" information unique to this library. This is currently unused and sets
|
|
only default variables.
|
|
</p>
|
|
|
|
|
|
<!-- ####################################################### -->
|
|
|
|
<hr />
|
|
<p class="fineprint"><em>
|
|
See <a href="17_intro/license.html">license.html</a> for copying conditions.
|
|
Comments and suggestions are welcome, and may be sent to
|
|
<a href="mailto:libstdc++@gcc.gnu.org">the libstdc++ mailing list</a>.
|
|
</em></p>
|
|
|
|
|
|
</body>
|
|
</html>
|