PTHREAD_MUTEX_NORMAL requires deadlock for nesting, DEFAULT
does not. Since glibc uses the same value (0) disable elision
for any call to pthread_mutexattr_settype() with a 0 value.
This implies that a program can disable elision by doing
pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_NORMAL)
Based on a original proposal by Rich Felker.
Add elision paths to the basic mutex locks.
The normal path has a check for RTM and upgrades the lock
to RTM when available. Trylocks cannot automatically upgrade,
so they check for elision every time.
We use a 4 byte value in the mutex to store the lock
elision adaptation state. This is separate from the adaptive
spin state and uses a separate field.
Condition variables currently do not support elision.
Recursive mutexes and condition variables may be supported at some point,
but are not in the current implementation. Also "trylock" will
not automatically enable elision unless some other lock call
has been already called on the lock.
This version does not use IFUNC, so it means every lock has one
additional check for elision. Benchmarking showed the overhead
to be negligible.
tst-mutex5 and 8 test some behaviour not required by POSIX,
that elision changes. This changes these tests to not check
this when elision is enabled at configure time.
Add Enable/disable flags used internally
Extend the mutex initializers to have the fields needed for
elision. The layout stays the same, and this is not visible
to programs.
These changes are not exposed outside pthread
Lock elision using TSX is a technique to optimize lock scaling
It allows to run locks in parallel using hardware support for
a transactional execution mode in 4th generation Intel Core CPUs.
See http://www.intel.com/software/tsx for more Information.
This patch implements a simple adaptive lock elision algorithm based
on RTM. It enables elision for the pthread mutexes and rwlocks.
The algorithm keeps track whether a mutex successfully elides or not,
and stops eliding for some time when it is not.
When the CPU supports RTM the elision path is automatically tried,
otherwise any elision is disabled.
The adaptation algorithm and its tuning is currently preliminary.
The code adds some checks to the lock fast paths. Micro-benchmarks
show little to no difference without RTM.
This patch implements the low level "lll_" code for lock elision.
Followon patches hook this into the pthread implementation
Changes with the RTM mutexes:
-----------------------------
Lock elision in pthreads is generally compatible with existing programs.
There are some obscure exceptions, which are expected to be uncommon.
See the manual for more details.
- A broken program that unlocks a free lock will crash.
There are ways around this with some tradeoffs (more code in hot paths)
I'm still undecided on what approach to take here; have to wait for testing reports.
- pthread_mutex_destroy of a lock mutex will not return EBUSY but 0.
- There's also a similar situation with trylock outside the mutex,
"knowing" that the mutex must be held due to some other condition.
In this case an assert failure cannot be recovered. This situation is
usually an existing bug in the program.
- Same applies to the rwlocks. Some of the return values changes
(for example there is no EDEADLK for an elided lock, unless it aborts.
However when elided it will also never deadlock of course)
- Timing changes, so broken programs that make assumptions about specific timing
may expose already existing latent problems. Note that these broken programs will
break in other situations too (loaded system, new faster hardware, compiler
optimizations etc.)
- Programs with non recursive mutexes that take them recursively in a thread and
which would always deadlock without elision may not always see a deadlock.
The deadlock will only happen on an early or delayed abort (which typically
happens at some point)
This only happens for mutexes not explicitely set to PTHREAD_MUTEX_NORMAL
or PTHREAD_MUTEX_ADAPTIVE_NP. PTHREAD_MUTEX_NORMAL mutexes do not elide.
The elision default can be set at configure time.
This patch implements the basic infrastructure for elision.
Static applications that call pthread_exit on the main
thread segfault. This is because after a thread terminates
__libc_start_main decrements __nptl_nthreads which is only
defined in pthread_create. Therefore the right solution is
to add a requirement to pthread_create from pthread_exit.
~~~
nptl/
2013-06-24 Vladimir Nikulichev <v.nikulichev@gmail.com>
[BZ #12310]
* pthread_exit.c: Add reference to pthread_create.
This patch introduces two new convenience functions to set the default
thread attributes used for creating threads. This allows a programmer
to set the default thread attributes just once in a process and then
run pthread_create without additional attributes.
Resolves BZ #15618.
pthread_attr_getaffinity_np may write beyond bounds of the input
cpuset buffer if the size of the input buffer is smaller than the
buffer present in the input pthread attributes. Fix is to copy to the
extent of the minimum of the source and the destination.
It is very very possible that the futex syscall returns an
error and that the caller of lll_futex_wake may want to
look at that error and propagate the failure.
This patch allows a caller to see the syscall error.
There are no users of the syscall error at present, but
future cleanups are now be able to check for the error.
--
nplt/
2013-06-10 Carlos O'Donell <carlos@redhat.com>
* sysdeps/unix/sysv/linux/i386/lowlevellock.h
(lll_futex_wake): Return syscall error.
* sysdeps/unix/sysv/linux/x86_64/lowlevellock.h
(lll_futex_wake): Return syscall error.
The sem_post.c file uses atomic functions without including
atomic.h. Add `#include <atomic.h>' to the file to prevent
any compile time warnings when other headers change and
atomic.h isn't implicitly included.
---
nptl/
2013-04-07 Carlos O'Donell <carlos@redhat.com>
* sysdeps/unix/sysv/linux/sem_post.c: Include atomic.h.
Fixes BZ #15337.
Static builds fail with the following warning:
/home/tools/glibc/glibc/nptl/../nptl/sysdeps/unix/sysv/linux/x86_64/cancellation.S:80:
undefined reference to `__GI___pthread_unwind'
When the source is configured with --disable-hidden-plt. This is
because the preprocessor conditional in cancellation.S only checks if
the build is for SHARED, whereas hidden_def is defined appropriately
only for a SHARED build that will have symbol versioning *and* hidden
defs are enabled. The last case is false here.
This reverts the change that allows the POSIX Thread default stack size
to be changed by the environment variable
GLIBC_PTHREAD_DEFAULT_STACKSIZE. It has been requested that more
discussion happen before this change goes into 2.18.
This feature is specifically for the C++ compiler to offload calling
thread_local object destructors on thread program exit, to glibc.
This is to overcome the possible complication of destructors of
thread_local objects getting called after the DSO in which they're
defined is unloaded by the dynamic linker. The DSO is marked as
'unloadable' if it has a constructed thread_local object and marked as
'unloadable' again when all the constructed thread_local objects
defined in it are destroyed.
* sysdeps/unix/sysv/linux/i386/i486/pthread_cond_timedwait.S
(__pthread_cond_timedwait): If possible use FUTEX_WAIT_BITSET to
directly use absolute timeout.
Since the FUTEX_WAIT operation takes a relative timeout, the
pthread_cond_timedwait and other timed function implementations have
to get a relative timeout from the absolute timeout parameter it gets
before it makes the futex syscall. This value is then converted back
into an absolute timeout within the kernel. This is a waste and has
hence been improved upon by a FUTEX_WAIT_BITSET operation (OR'd with
FUTEX_CLOCK_REALTIME to make the kernel use the realtime clock instead
of the default monotonic clock). This was implemented only in the x86
and sh assembly code and not in the C code. This patch implements
support for FUTEX_WAIT_BITSET whenever available (since linux-2.6.29)
for s390 and powerpc.
nptl/
* sysdeps/unix/sysv/linux/sparc/lowlevellock.h (BUSY_WAIT_NOP):
Define when we have v9 instructions available.
* sysdeps/unix/sysv/linux/sparc/sparc64/cpu_relax.S: New file.
* sysdeps/unix/sysv/linux/sparc/sparc32/sparcv9/cpu_relax.S: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc32/sparcv9/Makefile: New
file.
* sysdeps/unix/sysv/linux/sparc/sparc64/Makefile: Add cpu_relax
to libpthread-routines.
The macro pthread_cleanup_push_defer_np in pthread.h has a misaligned
line continuation marker. This marker was previously aligned, but
recent changes have moved it out of alignment. This change realigns
the marker. This also reduces the diff against the hppa version of
pthread.h where the marker is aligned.
[BZ #14652]
When a thread waiting in pthread_cond_wait with a PI mutex is
cancelled after it has returned successfully from the futex syscall
but just before async cancellation is disabled, it enters its
cancellation handler with the mutex held and simply calling a
mutex_lock again will result in a deadlock. Hence, it is necessary to
see if the thread owns the lock and try to lock it only if it doesn't.
[BZ #14568]
* sysdeps/sparc/tls.h (DB_THREAD_SELF_INCLUDE): Delete.
(DB_THREAD_SELF): Use constants for the register offsets. Correct
the case of a 64-bit debugger with a 32-bit inferior.
[BZ #14417]
A futex call with FUTEX_WAIT_REQUEUE_PI returns with the mutex locked
on success. If such a successful thread is pipped to the cond_lock by
another spuriously woken waiter, it could be sent back to wait on the
futex with the mutex lock held, thus causing a deadlock. So it is
necessary that the thread relinquishes the mutex before going back to
sleep.
[BZ #14477]
Add an additional entry in the exception table to jump to
__condvar_w_cleanup2 instead of __condvar_w_cleanup for PI mutexes
when %ebx contains the address of the futex instead of the condition
variable.
Ref gcc.gnu.org/bugzilla/show_bug.cgi?id=52839#c10
Release barriers are needed to ensure that any memory written by
init_routine is seen by other threads before *once_control changes.
In the case of clear_once_control we need to flush any partially
written state.
In some cases, the compiler would optimize out the call to
allocate_and_test and thus result in a false positive for the test
case. Another problem was the fact that the compiler could in some
cases generate additional shifting of the stack pointer, resulting in
alloca moving the stack pointer beyond what is allowed by the
rlimit. Hence, accessing the stackaddr returned by pthread_getattr_np
is safer than relying on the alloca'd result.
Another problem is when RLIMIT may be very large, which may result in
violation of other resource limits. Hence we cap the max stack size to
8M for this test.
When rlimit is small enough to be used as the stacksize to be returned
in pthread_getattr_np, cases where a stack is made executable due to a
DSO load get stack size that is larger than what the kernel
allows. This is because in such a case the stack size does not account
for the pages that have auxv and program arguments.
Additionally, the stacksize for the process derived from this should
be truncated to align to page size to avoid going beyond rlimit.
When a stack is marked executable due to loading a DSO that requires
an executable stack, the logic tends to leave out a portion of stack
after the first frame, thus causing a difference in the value returned
by pthread_getattr_np before and after the stack is marked
executable. It ought to be possible to fix this by marking the rest of
the stack as executable too, but in the interest of marking as less of
the stack as executable as possible, the path this fix takes is to
make pthread_getattr_np also look at the first frame as the underflow
end of the stack and compute size and stack top accordingly.
The above happens only for the main process stack. NPTL thread stacks
are not affected by this change.
There is nothing in the POSIX specification to disallow a
single-threaded program from cancelling itself, so we forcibly enable
multiple_threads to allow the next available cancellation point in the
thread to run.
Also added additional tests to cover various cancellation scenarios.
nptl/
* sysdeps/sparc/sparc64/pthread_spin_unlock.S: Fix thinko, we
always have to return 0, especially for the pthread_spin_init
alias.
* sysdeps/sparc/sparc32/pthread_spin_lock.S: Add missing trailing
newline.
* sysdeps/sparc/sparc32/sparcv9/pthread_spin_lock.S: Likewise.
* sysdeps/sparc/sparc64/pthread_spin_lock.S: Likewise.
* tst-cond16.c (do_test): Use a thread stack size which is either
PTHREAD_STACK_MIN or the page size, whichever is larger.
* tst-cond18.c (do_test): Likewise.
nptl/
[BZ #13844]
* sysdeps/unix/sysv/linux/libc-lowlevellock.c: Include using <..>
instead of "...".
* sysdeps/unix/sysv/linux/sparc/sparc32/libc-lowlevellock.c:
Delete, not needed.
/
* sysdeps/unix/sysv/linux/sparc/sparc64/get_clockfreq.c
(set_obp_int): New function.
(get_obp_int): New function.
(__get_clockfreq_via_dev_openprom): Likewise.
* sysdeps/unix/sysv/linux/sparc/sysdep.h (INTERNAL_SYSCALL_ERROR_P): Avoid
unused variable warnings on 'val' and use builtin_expect.
(INLINE_SYSCALL): Don't wrap INTERNAL_SYSCALL_ERROR_P with builtin_expect.
(INLINE_CLONE_SYSCALL): Likewise.
nptl/
* sysdeps/unix/sysv/linux/sparc/sem_post.c (__new_sem_post): Use
atomic_increment and remove unused local variable.
(__old_sem_post): Likewise.
/
* sysdeps/sparc/Makefile: Add -fPIC when building crt{i,n}.S
* sysdeps/sparc/crti.S: New file.
* sysdeps/sparc/crtn.S: New file.
* sysdeps/sparc/sparc32/Makefile: Remove initfini handling.
* sysdeps/sparc/sparc64/Makefile: Likewise.
nptl/
* sysdeps/sparc/Makefile: Add -fPIC when building pt-crti.S and crtn.S
H.J. Lu
Andi Kleen
Vladimir Nikulichev
Joseph Myers
Adhemerval Zanella
Siddhesh Poyarekar
Carlos O'Donell
Ondrej Bilka
Ryan S. Arnold
Andreas Jaeger
Andreas Schwab
Roland McGrath
David S. Miller
Carlos O'Donell
Allan McRae
Marcus Shawcroft
Chris Metcalf
Aurelien Jarno
Jim Blandy
Carlos O'Donell
Alan Modra
Dmitry V. Levin
Jeff Law
Maxim Kuvyrkov
Andreas Schwab
Thomas Schwinge
Carlos Sánchez de La Lama
Chung-Lin Tang
Rayson Ho
Thomas Schwinge
Paul Pluzhnikov
Paul Eggert
Ulrich Drepper
Richard Henderson
Kaz Kojima
Marek Polacek
Carlos O'Donell
Andreas Schwab
H.J. Lu
Jim Meyering
Roland McGrath