binutils-gdb/gdb/common/new-op.c

99 lines
2.9 KiB
C
Raw Normal View History

gdb: Replace operator new / operator new[] If xmalloc fails allocating memory, usually because something tried a huge allocation, like xmalloc(-1) or some such, GDB asks the user what to do: .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) If the user says "n", that throws a QUIT exception, which is caught by one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the stack. The default implementations of operator new / operator new[] call malloc directly, and on memory allocation failure throw std::bad_alloc. Currently, if that happens, since nothing catches it, the exception escapes out of main, and GDB aborts from unhandled exception. This patch replaces the default operator new variants with versions that, just like xmalloc: #1 - Raise an internal-error on memory allocation failure. #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks continue handling memory allocation problems. A minor complication of #2 is that operator new can _only_ throw std::bad_alloc, or something that extends it: void* operator new (std::size_t size) throw (std::bad_alloc); That means that if we let a gdb QUIT exception escape from within operator new, the C++ runtime aborts due to unexpected exception thrown. So to bridge the gap, this patch adds a new gdb_quit_bad_alloc exception type that inherits both std::bad_alloc and gdb_exception, and throws _that_. If we decide that we should be catching memory allocation errors in fewer places than all the places we currently catch them (everywhere we use RETURN_MASK_ALL currently), then we could change operator new to throw plain std::bad_alloc then. But I'm considering such a change as separate matter from this one -- it'd make sense to do the same to xmalloc at the same time, for instance. Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc. without losing the "virtual memory exhausted" internal-error safeguard. Tested on x86_64 Fedora 23. gdb/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (COMMON_OBS): Add common/new-op.o. (new-op.o): New rule. * common/common-exceptions.h: Include <new>. (struct gdb_quit_bad_alloc): New type. * common/new-op.c: New file. gdb/gdbserver/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (OBS): Add common/new-op.o. (new-op.o): New rule.
2016-09-23 17:42:24 +02:00
/* Replace operator new/new[], for GDB, the GNU debugger.
Copyright (C) 2016-2017 Free Software Foundation, Inc.
gdb: Replace operator new / operator new[] If xmalloc fails allocating memory, usually because something tried a huge allocation, like xmalloc(-1) or some such, GDB asks the user what to do: .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) If the user says "n", that throws a QUIT exception, which is caught by one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the stack. The default implementations of operator new / operator new[] call malloc directly, and on memory allocation failure throw std::bad_alloc. Currently, if that happens, since nothing catches it, the exception escapes out of main, and GDB aborts from unhandled exception. This patch replaces the default operator new variants with versions that, just like xmalloc: #1 - Raise an internal-error on memory allocation failure. #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks continue handling memory allocation problems. A minor complication of #2 is that operator new can _only_ throw std::bad_alloc, or something that extends it: void* operator new (std::size_t size) throw (std::bad_alloc); That means that if we let a gdb QUIT exception escape from within operator new, the C++ runtime aborts due to unexpected exception thrown. So to bridge the gap, this patch adds a new gdb_quit_bad_alloc exception type that inherits both std::bad_alloc and gdb_exception, and throws _that_. If we decide that we should be catching memory allocation errors in fewer places than all the places we currently catch them (everywhere we use RETURN_MASK_ALL currently), then we could change operator new to throw plain std::bad_alloc then. But I'm considering such a change as separate matter from this one -- it'd make sense to do the same to xmalloc at the same time, for instance. Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc. without losing the "virtual memory exhausted" internal-error safeguard. Tested on x86_64 Fedora 23. gdb/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (COMMON_OBS): Add common/new-op.o. (new-op.o): New rule. * common/common-exceptions.h: Include <new>. (struct gdb_quit_bad_alloc): New type. * common/new-op.c: New file. gdb/gdbserver/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (OBS): Add common/new-op.o. (new-op.o): New rule.
2016-09-23 17:42:24 +02:00
This file is part of GDB.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>. */
Don't override operator new if GDB is built with -fsanitize=address Nowadays, if we build GDB with -fsanitize=address, we can get the asan error below, (gdb) quit ================================================================= ==9723==ERROR: AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete) on 0x60200003bf70 #0 0x7f88f3837527 in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x55527) #1 0xac8e13 in __gnu_cxx::new_allocator<void (*)()>::deallocate(void (**)(), unsigned long) /usr/include/c++/4.9/ext/new_allocator.h:110 #2 0xac8cc2 in __gnu_cxx::__alloc_traits<std::allocator<void (*)()> >::deallocate(std::allocator<void (*)()>&, void (**)(), unsigned long) /usr/include/c++/4.9/ext/alloc_traits.h:185 .... 0x60200003bf70 is located 0 bytes inside of 8-byte region [0x60200003bf70,0x60200003bf78) allocated by thread T0 here: #0 0x7f88f38367ef in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x547ef) #1 0xbd2762 in operator new(unsigned long) /home/yao/SourceCode/gnu/gdb/git/gdb/common/new-op.c:42 #2 0xac8edc in __gnu_cxx::new_allocator<void (*)()>::allocate(unsigned long, void const*) /usr/include/c++/4.9/ext/new_allocator.h:104 #3 0xac8d81 in __gnu_cxx::__alloc_traits<std::allocator<void (*)()> >::allocate(std::allocator<void (*)()>&, unsigned long) /usr/include/c++/4.9/ext/alloc_traits.h:182 The reason for this is that we override operator new but don't override operator delete. This patch does the override if the code is NOT compiled with asan. gdb: 2016-10-25 Yao Qi <yao.qi@linaro.org> PR gdb/20716 * common/new-op.c (__has_feature): New macro. Don't override operator new if asan is used.
2016-10-25 12:13:00 +02:00
/* GCC does not understand __has_feature. */
#if !defined(__has_feature)
# define __has_feature(x) 0
#endif
#if !__has_feature(address_sanitizer) && !defined(__SANITIZE_ADDRESS__)
gdb: Replace operator new / operator new[] If xmalloc fails allocating memory, usually because something tried a huge allocation, like xmalloc(-1) or some such, GDB asks the user what to do: .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) If the user says "n", that throws a QUIT exception, which is caught by one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the stack. The default implementations of operator new / operator new[] call malloc directly, and on memory allocation failure throw std::bad_alloc. Currently, if that happens, since nothing catches it, the exception escapes out of main, and GDB aborts from unhandled exception. This patch replaces the default operator new variants with versions that, just like xmalloc: #1 - Raise an internal-error on memory allocation failure. #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks continue handling memory allocation problems. A minor complication of #2 is that operator new can _only_ throw std::bad_alloc, or something that extends it: void* operator new (std::size_t size) throw (std::bad_alloc); That means that if we let a gdb QUIT exception escape from within operator new, the C++ runtime aborts due to unexpected exception thrown. So to bridge the gap, this patch adds a new gdb_quit_bad_alloc exception type that inherits both std::bad_alloc and gdb_exception, and throws _that_. If we decide that we should be catching memory allocation errors in fewer places than all the places we currently catch them (everywhere we use RETURN_MASK_ALL currently), then we could change operator new to throw plain std::bad_alloc then. But I'm considering such a change as separate matter from this one -- it'd make sense to do the same to xmalloc at the same time, for instance. Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc. without losing the "virtual memory exhausted" internal-error safeguard. Tested on x86_64 Fedora 23. gdb/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (COMMON_OBS): Add common/new-op.o. (new-op.o): New rule. * common/common-exceptions.h: Include <new>. (struct gdb_quit_bad_alloc): New type. * common/new-op.c: New file. gdb/gdbserver/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (OBS): Add common/new-op.o. (new-op.o): New rule.
2016-09-23 17:42:24 +02:00
#include "common-defs.h"
#include "host-defs.h"
#include <new>
/* Override operator new / operator new[], in order to internal_error
on allocation failure and thus query the user for abort/core
dump/continue, just like xmalloc does. We don't do this from a
new-handler function instead (std::set_new_handler) because we want
to catch allocation errors from within global constructors too.
Skip overriding if building with -fsanitize=address though.
Address sanitizer wants to override operator new/delete too in
order to detect malloc+delete and new+free mismatches. Our
versions would mask out ASan's, with the result of losing that
useful mismatch detection.
gdb: Replace operator new / operator new[] If xmalloc fails allocating memory, usually because something tried a huge allocation, like xmalloc(-1) or some such, GDB asks the user what to do: .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) If the user says "n", that throws a QUIT exception, which is caught by one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the stack. The default implementations of operator new / operator new[] call malloc directly, and on memory allocation failure throw std::bad_alloc. Currently, if that happens, since nothing catches it, the exception escapes out of main, and GDB aborts from unhandled exception. This patch replaces the default operator new variants with versions that, just like xmalloc: #1 - Raise an internal-error on memory allocation failure. #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks continue handling memory allocation problems. A minor complication of #2 is that operator new can _only_ throw std::bad_alloc, or something that extends it: void* operator new (std::size_t size) throw (std::bad_alloc); That means that if we let a gdb QUIT exception escape from within operator new, the C++ runtime aborts due to unexpected exception thrown. So to bridge the gap, this patch adds a new gdb_quit_bad_alloc exception type that inherits both std::bad_alloc and gdb_exception, and throws _that_. If we decide that we should be catching memory allocation errors in fewer places than all the places we currently catch them (everywhere we use RETURN_MASK_ALL currently), then we could change operator new to throw plain std::bad_alloc then. But I'm considering such a change as separate matter from this one -- it'd make sense to do the same to xmalloc at the same time, for instance. Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc. without losing the "virtual memory exhausted" internal-error safeguard. Tested on x86_64 Fedora 23. gdb/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (COMMON_OBS): Add common/new-op.o. (new-op.o): New rule. * common/common-exceptions.h: Include <new>. (struct gdb_quit_bad_alloc): New type. * common/new-op.c: New file. gdb/gdbserver/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (OBS): Add common/new-op.o. (new-op.o): New rule.
2016-09-23 17:42:24 +02:00
Note that C++ implementations could either have their throw
versions call the nothrow versions (libstdc++), or the other way
around (clang/libc++). For that reason, we replace both throw and
nothrow variants and call malloc directly. */
void *
operator new (std::size_t sz)
{
/* malloc (0) is unpredictable; avoid it. */
if (sz == 0)
sz = 1;
void *p = malloc (sz); /* ARI: malloc */
if (p == NULL)
{
/* If the user decides to continue debugging, throw a
gdb_quit_bad_alloc exception instead of a regular QUIT
gdb_exception. The former extends both std::bad_alloc and a
QUIT gdb_exception. This is necessary because operator new
can only ever throw std::bad_alloc, or something that extends
it. */
TRY
{
malloc_failure (sz);
}
CATCH (ex, RETURN_MASK_ALL)
{
do_cleanups (all_cleanups ());
throw gdb_quit_bad_alloc (ex);
}
END_CATCH
}
return p;
}
void *
operator new (std::size_t sz, const std::nothrow_t&) noexcept
gdb: Replace operator new / operator new[] If xmalloc fails allocating memory, usually because something tried a huge allocation, like xmalloc(-1) or some such, GDB asks the user what to do: .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) If the user says "n", that throws a QUIT exception, which is caught by one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the stack. The default implementations of operator new / operator new[] call malloc directly, and on memory allocation failure throw std::bad_alloc. Currently, if that happens, since nothing catches it, the exception escapes out of main, and GDB aborts from unhandled exception. This patch replaces the default operator new variants with versions that, just like xmalloc: #1 - Raise an internal-error on memory allocation failure. #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks continue handling memory allocation problems. A minor complication of #2 is that operator new can _only_ throw std::bad_alloc, or something that extends it: void* operator new (std::size_t size) throw (std::bad_alloc); That means that if we let a gdb QUIT exception escape from within operator new, the C++ runtime aborts due to unexpected exception thrown. So to bridge the gap, this patch adds a new gdb_quit_bad_alloc exception type that inherits both std::bad_alloc and gdb_exception, and throws _that_. If we decide that we should be catching memory allocation errors in fewer places than all the places we currently catch them (everywhere we use RETURN_MASK_ALL currently), then we could change operator new to throw plain std::bad_alloc then. But I'm considering such a change as separate matter from this one -- it'd make sense to do the same to xmalloc at the same time, for instance. Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc. without losing the "virtual memory exhausted" internal-error safeguard. Tested on x86_64 Fedora 23. gdb/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (COMMON_OBS): Add common/new-op.o. (new-op.o): New rule. * common/common-exceptions.h: Include <new>. (struct gdb_quit_bad_alloc): New type. * common/new-op.c: New file. gdb/gdbserver/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (OBS): Add common/new-op.o. (new-op.o): New rule.
2016-09-23 17:42:24 +02:00
{
/* malloc (0) is unpredictable; avoid it. */
if (sz == 0)
sz = 1;
return malloc (sz); /* ARI: malloc */
}
void *
operator new[] (std::size_t sz)
{
return ::operator new (sz);
}
void*
operator new[] (std::size_t sz, const std::nothrow_t&) noexcept
gdb: Replace operator new / operator new[] If xmalloc fails allocating memory, usually because something tried a huge allocation, like xmalloc(-1) or some such, GDB asks the user what to do: .../src/gdb/utils.c:1079: internal-error: virtual memory exhausted. A problem internal to GDB has been detected, further debugging may prove unreliable. Quit this debugging session? (y or n) If the user says "n", that throws a QUIT exception, which is caught by one of the multiple CATCH(RETURN_MASK_ALL) blocks somewhere up the stack. The default implementations of operator new / operator new[] call malloc directly, and on memory allocation failure throw std::bad_alloc. Currently, if that happens, since nothing catches it, the exception escapes out of main, and GDB aborts from unhandled exception. This patch replaces the default operator new variants with versions that, just like xmalloc: #1 - Raise an internal-error on memory allocation failure. #2 - Throw a QUIT gdb_exception, so that the exact same CATCH blocks continue handling memory allocation problems. A minor complication of #2 is that operator new can _only_ throw std::bad_alloc, or something that extends it: void* operator new (std::size_t size) throw (std::bad_alloc); That means that if we let a gdb QUIT exception escape from within operator new, the C++ runtime aborts due to unexpected exception thrown. So to bridge the gap, this patch adds a new gdb_quit_bad_alloc exception type that inherits both std::bad_alloc and gdb_exception, and throws _that_. If we decide that we should be catching memory allocation errors in fewer places than all the places we currently catch them (everywhere we use RETURN_MASK_ALL currently), then we could change operator new to throw plain std::bad_alloc then. But I'm considering such a change as separate matter from this one -- it'd make sense to do the same to xmalloc at the same time, for instance. Meanwhile, this allows using new/new[] instead of xmalloc/XNEW/etc. without losing the "virtual memory exhausted" internal-error safeguard. Tested on x86_64 Fedora 23. gdb/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (COMMON_OBS): Add common/new-op.o. (new-op.o): New rule. * common/common-exceptions.h: Include <new>. (struct gdb_quit_bad_alloc): New type. * common/new-op.c: New file. gdb/gdbserver/ChangeLog: 2016-09-23 Pedro Alves <palves@redhat.com> * Makefile.in (SFILES): Add common/new-op.c. (OBS): Add common/new-op.o. (new-op.o): New rule.
2016-09-23 17:42:24 +02:00
{
return ::operator new (sz, std::nothrow);
}
Don't override operator new if GDB is built with -fsanitize=address Nowadays, if we build GDB with -fsanitize=address, we can get the asan error below, (gdb) quit ================================================================= ==9723==ERROR: AddressSanitizer: alloc-dealloc-mismatch (malloc vs operator delete) on 0x60200003bf70 #0 0x7f88f3837527 in operator delete(void*) (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x55527) #1 0xac8e13 in __gnu_cxx::new_allocator<void (*)()>::deallocate(void (**)(), unsigned long) /usr/include/c++/4.9/ext/new_allocator.h:110 #2 0xac8cc2 in __gnu_cxx::__alloc_traits<std::allocator<void (*)()> >::deallocate(std::allocator<void (*)()>&, void (**)(), unsigned long) /usr/include/c++/4.9/ext/alloc_traits.h:185 .... 0x60200003bf70 is located 0 bytes inside of 8-byte region [0x60200003bf70,0x60200003bf78) allocated by thread T0 here: #0 0x7f88f38367ef in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.1+0x547ef) #1 0xbd2762 in operator new(unsigned long) /home/yao/SourceCode/gnu/gdb/git/gdb/common/new-op.c:42 #2 0xac8edc in __gnu_cxx::new_allocator<void (*)()>::allocate(unsigned long, void const*) /usr/include/c++/4.9/ext/new_allocator.h:104 #3 0xac8d81 in __gnu_cxx::__alloc_traits<std::allocator<void (*)()> >::allocate(std::allocator<void (*)()>&, unsigned long) /usr/include/c++/4.9/ext/alloc_traits.h:182 The reason for this is that we override operator new but don't override operator delete. This patch does the override if the code is NOT compiled with asan. gdb: 2016-10-25 Yao Qi <yao.qi@linaro.org> PR gdb/20716 * common/new-op.c (__has_feature): New macro. Don't override operator new if asan is used.
2016-10-25 12:13:00 +02:00
#endif