glibc/malloc/tst-malloc-backtrace.c

56 lines
1.5 KiB
C
Raw Normal View History

Avoid deadlock in malloc on backtrace (BZ #16159) When the malloc subsystem detects some kind of memory corruption, depending on the configuration it prints the error, a backtrace, a memory map and then aborts the process. In this process, the backtrace() call may result in a call to malloc, resulting in various kinds of problematic behavior. In one case, the malloc it calls may detect a corruption and call backtrace again, and a stack overflow may result due to the infinite recursion. In another case, the malloc it calls may deadlock on an arena lock with the malloc (or free, realloc, etc.) that detected the corruption. In yet another case, if the program is linked with pthreads, backtrace may do a pthread_once initialization, which deadlocks on itself. In all these cases, the program exit is not as intended. This is avoidable by marking the arena that malloc detected a corruption on, as unusable. The following patch does that. Features of this patch are as follows: - A flag is added to the mstate struct of the arena to indicate if the arena is corrupt. - The flag is checked whenever malloc functions try to get a lock on an arena. If the arena is unusable, a NULL is returned, causing the malloc to use mmap or try the next arena. - malloc_printerr sets the corrupt flag on the arena when it detects a corruption - free does not concern itself with the flag at all. It is not important since the backtrace workflow does not need free. A free in a parallel thread may cause another corruption, but that's not new - The flag check and set are not atomic and may race. This is fine since we don't care about contention during the flag check. We want to make sure that the malloc call in the backtrace does not trip on itself and all that action happens in the same thread and not across threads. I verified that the test case does not show any regressions due to this patch. I also ran the malloc benchmarks and found an insignificant difference in timings (< 2%). * malloc/Makefile (tests): New test case tst-malloc-backtrace. * malloc/arena.c (arena_lock): Check if arena is corrupt. (reused_arena): Find a non-corrupt arena. (heap_trim): Pass arena to unlink. * malloc/hooks.c (malloc_check_get_size): Pass arena to malloc_printerr. (top_check): Likewise. (free_check): Likewise. (realloc_check): Likewise. * malloc/malloc.c (malloc_printerr): Add arena argument. (unlink): Likewise. (munmap_chunk): Adjust. (ARENA_CORRUPTION_BIT): New macro. (arena_is_corrupt): Likewise. (set_arena_corrupt): Likewise. (sysmalloc): Use mmap if there are no usable arenas. (_int_malloc): Likewise. (__libc_malloc): Don't fail if arena_get returns NULL. (_mid_memalign): Likewise. (__libc_calloc): Likewise. (__libc_realloc): Adjust for additional argument to malloc_printerr. (_int_free): Likewise. (malloc_consolidate): Likewise. (_int_realloc): Likewise. (_int_memalign): Don't touch corrupt arenas. * malloc/tst-malloc-backtrace.c: New test case.
2015-05-19 03:10:37 +02:00
/* Verify that backtrace does not deadlock on itself on memory corruption.
Copyright (C) 2015 Free Software Foundation, Inc.
This file is part of the GNU C Library.
The GNU C Library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
version 2.1 of the License, or (at your option) any later version.
The GNU C Library is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public
License along with the GNU C Library; if not, see
<http://www.gnu.org/licenses/>. */
#include <stdlib.h>
#define SIZE 4096
/* Wrap free with a function to prevent gcc from optimizing it out. */
static void
__attribute__((noinline))
call_free (void *ptr)
{
free (ptr);
*(size_t *)(ptr - sizeof (size_t)) = 1;
}
int do_test (void);
#define TEST_FUNCTION do_test ()
#define EXPECTED_SIGNAL SIGABRT
#include "../test-skeleton.c"
Avoid deadlock in malloc on backtrace (BZ #16159) When the malloc subsystem detects some kind of memory corruption, depending on the configuration it prints the error, a backtrace, a memory map and then aborts the process. In this process, the backtrace() call may result in a call to malloc, resulting in various kinds of problematic behavior. In one case, the malloc it calls may detect a corruption and call backtrace again, and a stack overflow may result due to the infinite recursion. In another case, the malloc it calls may deadlock on an arena lock with the malloc (or free, realloc, etc.) that detected the corruption. In yet another case, if the program is linked with pthreads, backtrace may do a pthread_once initialization, which deadlocks on itself. In all these cases, the program exit is not as intended. This is avoidable by marking the arena that malloc detected a corruption on, as unusable. The following patch does that. Features of this patch are as follows: - A flag is added to the mstate struct of the arena to indicate if the arena is corrupt. - The flag is checked whenever malloc functions try to get a lock on an arena. If the arena is unusable, a NULL is returned, causing the malloc to use mmap or try the next arena. - malloc_printerr sets the corrupt flag on the arena when it detects a corruption - free does not concern itself with the flag at all. It is not important since the backtrace workflow does not need free. A free in a parallel thread may cause another corruption, but that's not new - The flag check and set are not atomic and may race. This is fine since we don't care about contention during the flag check. We want to make sure that the malloc call in the backtrace does not trip on itself and all that action happens in the same thread and not across threads. I verified that the test case does not show any regressions due to this patch. I also ran the malloc benchmarks and found an insignificant difference in timings (< 2%). * malloc/Makefile (tests): New test case tst-malloc-backtrace. * malloc/arena.c (arena_lock): Check if arena is corrupt. (reused_arena): Find a non-corrupt arena. (heap_trim): Pass arena to unlink. * malloc/hooks.c (malloc_check_get_size): Pass arena to malloc_printerr. (top_check): Likewise. (free_check): Likewise. (realloc_check): Likewise. * malloc/malloc.c (malloc_printerr): Add arena argument. (unlink): Likewise. (munmap_chunk): Adjust. (ARENA_CORRUPTION_BIT): New macro. (arena_is_corrupt): Likewise. (set_arena_corrupt): Likewise. (sysmalloc): Use mmap if there are no usable arenas. (_int_malloc): Likewise. (__libc_malloc): Don't fail if arena_get returns NULL. (_mid_memalign): Likewise. (__libc_calloc): Likewise. (__libc_realloc): Adjust for additional argument to malloc_printerr. (_int_free): Likewise. (malloc_consolidate): Likewise. (_int_realloc): Likewise. (_int_memalign): Don't touch corrupt arenas. * malloc/tst-malloc-backtrace.c: New test case.
2015-05-19 03:10:37 +02:00
int
do_test (void)
{
void *ptr1 = malloc (SIZE);
void *ptr2 = malloc (SIZE);
/* Avoid unwanted output to TTY after an expected memory corruption. */
ignore_stderr();
Avoid deadlock in malloc on backtrace (BZ #16159) When the malloc subsystem detects some kind of memory corruption, depending on the configuration it prints the error, a backtrace, a memory map and then aborts the process. In this process, the backtrace() call may result in a call to malloc, resulting in various kinds of problematic behavior. In one case, the malloc it calls may detect a corruption and call backtrace again, and a stack overflow may result due to the infinite recursion. In another case, the malloc it calls may deadlock on an arena lock with the malloc (or free, realloc, etc.) that detected the corruption. In yet another case, if the program is linked with pthreads, backtrace may do a pthread_once initialization, which deadlocks on itself. In all these cases, the program exit is not as intended. This is avoidable by marking the arena that malloc detected a corruption on, as unusable. The following patch does that. Features of this patch are as follows: - A flag is added to the mstate struct of the arena to indicate if the arena is corrupt. - The flag is checked whenever malloc functions try to get a lock on an arena. If the arena is unusable, a NULL is returned, causing the malloc to use mmap or try the next arena. - malloc_printerr sets the corrupt flag on the arena when it detects a corruption - free does not concern itself with the flag at all. It is not important since the backtrace workflow does not need free. A free in a parallel thread may cause another corruption, but that's not new - The flag check and set are not atomic and may race. This is fine since we don't care about contention during the flag check. We want to make sure that the malloc call in the backtrace does not trip on itself and all that action happens in the same thread and not across threads. I verified that the test case does not show any regressions due to this patch. I also ran the malloc benchmarks and found an insignificant difference in timings (< 2%). * malloc/Makefile (tests): New test case tst-malloc-backtrace. * malloc/arena.c (arena_lock): Check if arena is corrupt. (reused_arena): Find a non-corrupt arena. (heap_trim): Pass arena to unlink. * malloc/hooks.c (malloc_check_get_size): Pass arena to malloc_printerr. (top_check): Likewise. (free_check): Likewise. (realloc_check): Likewise. * malloc/malloc.c (malloc_printerr): Add arena argument. (unlink): Likewise. (munmap_chunk): Adjust. (ARENA_CORRUPTION_BIT): New macro. (arena_is_corrupt): Likewise. (set_arena_corrupt): Likewise. (sysmalloc): Use mmap if there are no usable arenas. (_int_malloc): Likewise. (__libc_malloc): Don't fail if arena_get returns NULL. (_mid_memalign): Likewise. (__libc_calloc): Likewise. (__libc_realloc): Adjust for additional argument to malloc_printerr. (_int_free): Likewise. (malloc_consolidate): Likewise. (_int_realloc): Likewise. (_int_memalign): Don't touch corrupt arenas. * malloc/tst-malloc-backtrace.c: New test case.
2015-05-19 03:10:37 +02:00
call_free (ptr1);
ptr1 = malloc (SIZE);
/* Not reached. The return statement is to put ptr2 into use so that gcc
doesn't optimize out that malloc call. */
return (ptr1 == ptr2);
}