C++ library debug

By default, libstdc++ is built with efficiency in mind, and therefore performs little or no error checking that is not required by the C++ standard. This means that programs that incorrectly use the C++ standard library will exhibit behavior that is not portable and may not even be predictable, because they tread into implementation-specific or undefined behavior. To detect some of these errors before they can become problematic, libstdc++ offers a debug mode that provides additional checking of library facilities, and will report errors in the use of libstdc++ as soon as they can be detected by emitting a description of the problem to standard error and aborting the program. This debug mode is available with GCC 3.4.0 and later versions. The libstdc++ debug mode performs checking for many areas of the C++ standard, but the focus is on checking interactions among standard iterators, containers, and algorithms, including: Safe iterators: Iterators keep track of the container whose elements they reference, so errors such as incrementing a past-the-end iterator or dereferencing an iterator that points to a container that has been destructed are diagnosed immediately. Algorithm preconditions: Algorithms attempt to validate their input parameters to detect errors as early as possible. For instance, the set_intersection algorithm requires that its iterator parameters first1 and last1 form a valid iterator range, and that the sequence [first1, last1) is sorted according to the same predicate that was passed to set_intersection; the libstdc++ debug mode will detect an error if the sequence is not sorted or was sorted by a different predicate.

A program that uses the C++ standard library correctly will maintain the same semantics under debug mode as it had with the normal (release) library. All functional and exception-handling guarantees made by the normal library also hold for the debug mode library, with one exception: performance guarantees made by the normal library may not hold in the debug mode library. For instance, erasing an element in a std::list is a constant-time operation in normal library, but in debug mode it is linear in the number of iterators that reference that particular list. So while your (correct) program won't change its results, it is likely to execute more slowly. libstdc++ includes many extensions to the C++ standard library. In some cases the extensions are obvious, such as the hashed associative containers, whereas other extensions give predictable results to behavior that would otherwise be undefined, such as throwing an exception when a std::basic_string is constructed from a NULL character pointer. This latter category also includes implementation-defined and unspecified semantics, such as the growth rate of a vector. Use of these extensions is not considered incorrect, so code that relies on them will not be rejected by debug mode. However, use of these extensions may affect the portability of code to other implementations of the C++ standard library, and is therefore somewhat hazardous. For this reason, the libstdc++ debug mode offers a "pedantic" mode (similar to GCC's -pedantic compiler flag) that attempts to emulate the semantics guaranteed by the C++ standard. For instance, constructing a std::basic_string with a NULL character pointer would result in an exception under normal mode or non-pedantic debug mode (this is a libstdc++ extension), whereas under pedantic debug mode libstdc++ would signal an error. To enable the pedantic debug mode, compile your program with both -D_GLIBCXX_DEBUG and -D_GLIBCXX_DEBUG_PEDANTIC . (N.B. In GCC 3.4.x and 4.0.0, due to a bug, -D_GLIBXX_DEBUG_PEDANTIC was also needed. The problem has been fixed in GCC 4.0.1 and later versions.) The following library components provide extra debugging capabilities in debug mode: std::basic_string (no safe iterators and see note below) std::bitset std::deque std::list std::map std::multimap std::multiset std::set std::vector std::unordered_map std::unordered_multimap std::unordered_set std::unordered_multiset N.B. although there are precondition checks for some string operations, e.g. operator[], they will not always be run when using the char and wchar_t specialisations (std::string and std::wstring). This is because libstdc++ uses GCC's extern template extension to provide explicit instantiations of std::string and std::wstring, and those explicit instantiations don't include the debug-mode checks. If the containing functions are inlined then the checks will run, so compiling with -O1 might be enough to enable them. Alternatively -D_GLIBCXX_EXTERN_TEMPLATE=0 will suppress the declarations of the explicit instantiations and cause the functions to be instantiated with the debug-mode checks included, but this is unsupported and not guaranteed to work. For full debug-mode support you can use the __gnu_debug::basic_string debugging container directly, which always works correctly.

To use the libstdc++ debug mode, compile your application with the compiler flag -D_GLIBCXX_DEBUG. Note that this flag changes the sizes and behavior of standard class templates such as std::vector, and therefore you can only link code compiled with debug mode and code compiled without debug mode if no instantiation of a container is passed between the two translation units. By default, error messages are formatted to fit on lines of about 78 characters. The environment variable GLIBCXX_DEBUG_MESSAGE_LENGTH can be used to request a different length.

When it is not feasible to recompile your entire application, or only specific containers need checking, debugging containers are available as GNU extensions. These debugging containers are functionally equivalent to the standard drop-in containers used in debug mode, but they are available in a separate namespace as GNU extensions and may be used in programs compiled with either release mode or with debug mode. The following table provides the names and headers of the debugging containers: Container Header Debug container Debug header std::bitset bitset __gnu_debug::bitset <debug/bitset> std::deque deque __gnu_debug::deque <debug/deque> std::list list __gnu_debug::list <debug/list> std::map map __gnu_debug::map <debug/map> std::multimap map __gnu_debug::multimap <debug/map> std::multiset set __gnu_debug::multiset <debug/set> std::set set __gnu_debug::set <debug/set> std::string string __gnu_debug::string <debug/string> std::wstring string __gnu_debug::wstring <debug/string> std::basic_string string __gnu_debug::basic_string <debug/string> std::vector vector __gnu_debug::vector <debug/vector> In addition, when compiling in C++11 mode, these additional containers have additional debug capability. Container Header Debug container Debug header std::unordered_map unordered_map __gnu_debug::unordered_map <debug/unordered_map> std::unordered_multimap unordered_map __gnu_debug::unordered_multimap <debug/unordered_map> std::unordered_set unordered_set __gnu_debug::unordered_set <debug/unordered_set> std::unordered_multiset unordered_set __gnu_debug::unordered_multiset <debug/unordered_set>

The libstdc++ debug mode replaces unsafe (but efficient) standard containers and iterators with semantically equivalent safe standard containers and iterators to aid in debugging user programs. The following goals directed the design of the libstdc++ debug mode: Correctness: the libstdc++ debug mode must not change the semantics of the standard library for all cases specified in the ANSI/ISO C++ standard. The essence of this constraint is that any valid C++ program should behave in the same manner regardless of whether it is compiled with debug mode or release mode. In particular, entities that are defined in namespace std in release mode should remain defined in namespace std in debug mode, so that legal specializations of namespace std entities will remain valid. A program that is not valid C++ (e.g., invokes undefined behavior) is not required to behave similarly, although the debug mode will abort with a diagnostic when it detects undefined behavior. Performance: the additional of the libstdc++ debug mode must not affect the performance of the library when it is compiled in release mode. Performance of the libstdc++ debug mode is secondary (and, in fact, will be worse than the release mode). Usability: the libstdc++ debug mode should be easy to use. It should be easily incorporated into the user's development environment (e.g., by requiring only a single new compiler switch) and should produce reasonable diagnostics when it detects a problem with the user program. Usability also involves detection of errors when using the debug mode incorrectly, e.g., by linking a release-compiled object against a debug-compiled object if in fact the resulting program will not run correctly. Minimize recompilation: While it is expected that users recompile at least part of their program to use debug mode, the amount of recompilation affects the detect-compile-debug turnaround time. This indirectly affects the usefulness of the debug mode, because debugging some applications may require rebuilding a large amount of code, which may not be feasible when the suspect code may be very localized. There are several levels of conformance to this requirement, each with its own usability and implementation characteristics. In general, the higher-numbered conformance levels are more usable (i.e., require less recompilation) but are more complicated to implement than the lower-numbered conformance levels. Full recompilation: The user must recompile his or her entire application and all C++ libraries it depends on, including the C++ standard library that ships with the compiler. This must be done even if only a small part of the program can use debugging features. Full user recompilation: The user must recompile his or her entire application and all C++ libraries it depends on, but not the C++ standard library itself. This must be done even if only a small part of the program can use debugging features. This can be achieved given a full recompilation system by compiling two versions of the standard library when the compiler is installed and linking against the appropriate one, e.g., a multilibs approach. Partial recompilation: The user must recompile the parts of his or her application and the C++ libraries it depends on that will use the debugging facilities directly. This means that any code that uses the debuggable standard containers would need to be recompiled, but code that does not use them (but may, for instance, use IOStreams) would not have to be recompiled. Per-use recompilation: The user must recompile the parts of his or her application and the C++ libraries it depends on where debugging should occur, and any other code that interacts with those containers. This means that a set of translation units that accesses a particular standard container instance may either be compiled in release mode (no checking) or debug mode (full checking), but must all be compiled in the same way; a translation unit that does not see that standard container instance need not be recompiled. This also means that a translation unit A that contains a particular instantiation (say, std::vector<int>) compiled in release mode can be linked against a translation unit B that contains the same instantiation compiled in debug mode (a feature not present with partial recompilation). While this behavior is technically a violation of the One Definition Rule, this ability tends to be very important in practice. The libstdc++ debug mode supports this level of recompilation. Per-unit recompilation: The user must only recompile the translation units where checking should occur, regardless of where debuggable standard containers are used. This has also been dubbed "-g mode", because the -g compiler switch works in this way, emitting debugging information at a per--translation-unit granularity. We believe that this level of recompilation is in fact not possible if we intend to supply safe iterators, leave the program semantics unchanged, and not regress in performance under release mode because we cannot associate extra information with an iterator (to form a safe iterator) without either reserving that space in release mode (performance regression) or allocating extra memory associated with each iterator with new (changes the program semantics).

This section provides an overall view of the design of the libstdc++ debug mode and details the relationship between design decisions and the stated design goals.

The libstdc++ debug mode uses a wrapper model where the debugging versions of library components (e.g., iterators and containers) form a layer on top of the release versions of the library components. The debugging components first verify that the operation is correct (aborting with a diagnostic if an error is found) and will then forward to the underlying release-mode container that will perform the actual work. This design decision ensures that we cannot regress release-mode performance (because the release-mode containers are left untouched) and partially enables mixing debug and release code at link time, although that will not be discussed at this time. Two types of wrappers are used in the implementation of the debug mode: container wrappers and iterator wrappers. The two types of wrappers interact to maintain relationships between iterators and their associated containers, which are necessary to detect certain types of standard library usage errors such as dereferencing past-the-end iterators or inserting into a container using an iterator from a different container.

Iterator wrappers provide a debugging layer over any iterator that is attached to a particular container, and will manage the information detailing the iterator's state (singular, dereferenceable, etc.) and tracking the container to which the iterator is attached. Because iterators have a well-defined, common interface the iterator wrapper is implemented with the iterator adaptor class template __gnu_debug::_Safe_iterator, which takes two template parameters: Iterator: The underlying iterator type, which must be either the iterator or const_iterator typedef from the sequence type this iterator can reference. Sequence: The type of sequence that this iterator references. This sequence must be a safe sequence (discussed below) whose iterator or const_iterator typedef is the type of the safe iterator.

Container wrappers provide a debugging layer over a particular container type. Because containers vary greatly in the member functions they support and the semantics of those member functions (especially in the area of iterator invalidation), container wrappers are tailored to the container they reference, e.g., the debugging version of std::list duplicates the entire interface of std::list, adding additional semantic checks and then forwarding operations to the real std::list (a public base class of the debugging version) as appropriate. However, all safe containers inherit from the class template __gnu_debug::_Safe_sequence, instantiated with the type of the safe container itself (an instance of the curiously recurring template pattern). The iterators of a container wrapper will be safe iterators that reference sequences of this type and wrap the iterators provided by the release-mode base class. The debugging container will use only the safe iterators within its own interface (therefore requiring the user to use safe iterators, although this does not change correct user code) and will communicate with the release-mode base class with only the underlying, unsafe, release-mode iterators that the base class exports. The debugging version of std::list will have the following basic structure: template<typename _Tp, typename _Allocator = allocator<_Tp> class debug-list : public release-list<_Tp, _Allocator>, public __gnu_debug::_Safe_sequence<debug-list<_Tp, _Allocator> > { typedef release-list<_Tp, _Allocator> _Base; typedef debug-list<_Tp, _Allocator> _Self; public: typedef __gnu_debug::_Safe_iterator<typename _Base::iterator, _Self> iterator; typedef __gnu_debug::_Safe_iterator<typename _Base::const_iterator, _Self> const_iterator; // duplicate std::list interface with debugging semantics };

The debug mode operates primarily by checking the preconditions of all standard library operations that it supports. Preconditions that are always checked (regardless of whether or not we are in debug mode) are checked via the __check_xxx macros defined and documented in the source file include/debug/debug.h. Preconditions that may or may not be checked, depending on the debug-mode macro _GLIBCXX_DEBUG, are checked via the __requires_xxx macros defined and documented in the same source file. Preconditions are validated using any additional information available at run-time, e.g., the containers that are associated with a particular iterator, the position of the iterator within those containers, the distance between two iterators that may form a valid range, etc. In the absence of suitable information, e.g., an input iterator that is not a safe iterator, these precondition checks will silently succeed. The majority of precondition checks use the aforementioned macros, which have the secondary benefit of having prewritten debug messages that use information about the current status of the objects involved (e.g., whether an iterator is singular or what sequence it is attached to) along with some static information (e.g., the names of the function parameters corresponding to the objects involved). When not using these macros, the debug mode uses either the debug-mode assertion macro _GLIBCXX_DEBUG_ASSERT , its pedantic cousin _GLIBCXX_DEBUG_PEDASSERT, or the assertion check macro that supports more advance formulation of error messages, _GLIBCXX_DEBUG_VERIFY. These macros are documented more thoroughly in the debug mode source code.

The libstdc++ debug mode is the first debug mode we know of that is able to provide the "Per-use recompilation" (4) guarantee, that allows release-compiled and debug-compiled code to be linked and executed together without causing unpredictable behavior. This guarantee minimizes the recompilation that users are required to perform, shortening the detect-compile-debug bug hunting cycle and making the debug mode easier to incorporate into development environments by minimizing dependencies. Achieving link- and run-time coexistence is not a trivial implementation task. To achieve this goal we use inline namespaces and a complex organization of debug- and release-modes. The end result is that we have achieved per-use recompilation but have had to give up some checking of the std::basic_string class template (namely, safe iterators).

Both the release-mode components and the debug-mode components need to exist within a single translation unit so that the debug versions can wrap the release versions. However, only one of these components should be user-visible at any particular time with the standard name, e.g., std::list. In release mode, we define only the release-mode version of the component with its standard name and do not include the debugging component at all. The release mode version is defined within the namespace std. Minus the namespace associations, this method leaves the behavior of release mode completely unchanged from its behavior prior to the introduction of the libstdc++ debug mode. Here's an example of what this ends up looking like, in C++. namespace std { template<typename _Tp, typename _Alloc = allocator<_Tp> > class list { // ... }; } // namespace std In debug mode we include the release-mode container (which is now defined in the namespace __cxx1998) and also the debug-mode container. The debug-mode container is defined within the namespace __debug, which is associated with namespace std via the C++11 namespace association language feature. This method allows the debug and release versions of the same component to coexist at compile-time and link-time without causing an unreasonable maintenance burden, while minimizing confusion. Again, this boils down to C++ code as follows: namespace std { namespace __cxx1998 { template<typename _Tp, typename _Alloc = allocator<_Tp> > class list { // ... }; } // namespace __gnu_norm namespace __debug { template<typename _Tp, typename _Alloc = allocator<_Tp> > class list : public __cxx1998::list<_Tp, _Alloc>, public __gnu_debug::_Safe_sequence<list<_Tp, _Alloc> > { // ... }; } // namespace __cxx1998 inline namespace __debug { } }

Because each component has a distinct and separate release and debug implementation, there is no issue with link-time coexistence: the separate namespaces result in different mangled names, and thus unique linkage. However, components that are defined and used within the C++ standard library itself face additional constraints. For instance, some of the member functions of std::moneypunct return std::basic_string. Normally, this is not a problem, but with a mixed mode standard library that could be using either debug-mode or release-mode basic_string objects, things get more complicated. As the return value of a function is not encoded into the mangled name, there is no way to specify a release-mode or a debug-mode string. In practice, this results in runtime errors. A simplified example of this problem is as follows. Take this translation unit, compiled in debug-mode: // -D_GLIBCXX_DEBUG #include <string> std::string test02(); std::string test01() { return test02(); } int main() { test01(); return 0; } ... and linked to this translation unit, compiled in release mode: #include <string> std::string test02() { return std::string("toast"); } For this reason we cannot easily provide safe iterators for the std::basic_string class template, as it is present throughout the C++ standard library. For instance, locale facets define typedefs that include basic_string: in a mixed debug/release program, should that typedef be based on the debug-mode basic_string or the release-mode basic_string? While the answer could be "both", and the difference hidden via renaming a la the debug/release containers, we must note two things about locale facets: They exist as shared state: one can create a facet in one translation unit and access the facet via the same type name in a different translation unit. This means that we cannot have two different versions of locale facets, because the types would not be the same across debug/release-mode translation unit barriers. They have virtual functions returning strings: these functions mangle in the same way regardless of the mangling of their return types (see above), and their precise signatures can be relied upon by users because they may be overridden in derived classes. With the design of libstdc++ debug mode, we cannot effectively hide the differences between debug and release-mode strings from the user. Failure to hide the differences may result in unpredictable behavior, and for this reason we have opted to only perform basic_string changes that do not require ABI changes. The effect on users is expected to be minimal, as there are simple alternatives (e.g., __gnu_debug::basic_string), and the usability benefit we gain from the ability to mix debug- and release-compiled translation units is enormous.

The coexistence scheme above was chosen over many alternatives, including language-only solutions and solutions that also required extensions to the C++ front end. The following is a partial list of solutions, with justifications for our rejection of each. Completely separate debug/release libraries: This is by far the simplest implementation option, where we do not allow any coexistence of debug- and release-compiled translation units in a program. This solution has an extreme negative affect on usability, because it is quite likely that some libraries an application depends on cannot be recompiled easily. This would not meet our usability or minimize recompilation criteria well. Add a Debug boolean template parameter: Partial specialization could be used to select the debug implementation when Debug == true, and the state of _GLIBCXX_DEBUG could decide whether the default Debug argument is true or false. This option would break conformance with the C++ standard in both debug and release modes. This would not meet our correctness criteria. Packaging a debug flag in the allocators: We could reuse the Allocator template parameter of containers by adding a sentinel wrapper debug<> that signals the user's intention to use debugging, and pick up the debug<> allocator wrapper in a partial specialization. However, this has two drawbacks: first, there is a conformance issue because the default allocator would not be the standard-specified std::allocator<T>. Secondly (and more importantly), users that specify allocators instead of implicitly using the default allocator would not get debugging containers. Thus this solution fails the correctness criteria. Define debug containers in another namespace, and employ a using declaration (or directive): This is an enticing option, because it would eliminate the need for the link_name extension by aliasing the templates. However, there is no true template aliasing mechanism in C++, because both using directives and using declarations disallow specialization. This method fails the correctness criteria. Use implementation-specific properties of anonymous namespaces. See this post. This method fails the correctness criteria. Extension: allow reopening on namespaces: This would allow the debug mode to effectively alias the namespace std to an internal namespace, such as __gnu_std_debug, so that it is completely separate from the release-mode std namespace. While this will solve some renaming problems and ensure that debug- and release-compiled code cannot be mixed unsafely, it ensures that debug- and release-compiled code cannot be mixed at all. For instance, the program would have two std::cout objects! This solution would fails the minimize recompilation requirement, because we would only be able to support option (1) or (2). Extension: use link name: This option involves complicated re-naming between debug-mode and release-mode components at compile time, and then a g++ extension called link name to recover the original names at link time. There are two drawbacks to this approach. One, it's very verbose, relying on macro renaming at compile time and several levels of include ordering. Two, ODR issues remained with container member functions taking no arguments in mixed-mode settings resulting in equivalent link names, vector::push_back() being one example. See proof-of-concept using link name. Other options may exist for implementing the debug mode, many of which have probably been considered and others that may still be lurking. This list may be expanded over time to include other options that we could have implemented, but in all cases the full ramifications of the approach (as measured against the design goals for a libstdc++ debug mode) should be considered first. The DejaGNU testsuite includes some testcases that check for known problems with some solutions (e.g., the using declaration solution that breaks user specialization), and additional testcases will be added as we are able to identify other typical problem cases. These test cases will serve as a benchmark by which we can compare debug mode implementations.

There are several existing implementations of debug modes for C++ standard library implementations, although none of them directly supports debugging for programs using libstdc++. The existing implementations include: SafeSTL: SafeSTL was the original debugging version of the Standard Template Library (STL), implemented by Cay S. Horstmann on top of the Hewlett-Packard STL. Though it inspired much work in this area, it has not been kept up-to-date for use with modern compilers or C++ standard library implementations. STLport: STLport is a free implementation of the C++ standard library derived from the SGI implementation, and ported to many other platforms. It includes a debug mode that uses a wrapper model (that in some ways inspired the libstdc++ debug mode design), although at the time of this writing the debug mode is somewhat incomplete and meets only the "Full user recompilation" (2) recompilation guarantee by requiring the user to link against a different library in debug mode vs. release mode. Metrowerks CodeWarrior: The C++ standard library that ships with Metrowerks CodeWarrior includes a debug mode. It is a full debug-mode implementation (including debugging for CodeWarrior extensions) and is easy to use, although it meets only the "Full recompilation" (1) recompilation guarantee.