Exec needs that mach_setup_thread does *not* set up TLS since it works on
another task, so we have to split this into mach_setup_tls.
* mach/mach.h (__mach_setup_tls, mach_setup_tls): Add prototypes.
* mach/setup-thread.c (__mach_setup_thread): Move TLS setup to...
(__mach_setup_tls): ... new function.
(mach_setup_tls): New alias.
* hurd/hurdsig.c (_hurdsig_init): Call __mach_setup_tls after
__mach_setup_thread.
* sysdeps/mach/hurd/profil.c (update_waiter): Likewise.
* sysdeps/mach/hurd/setitimer.c (setitimer_locked): Likewise.
* mach/Versions [libc] (mach_setup_tls): Add symbol.
* sysdeps/mach/hurd/i386/libc.abilist (mach_setup_tls): Likewise.
GNU_PROPERTY_X86_FEATURE_1_AND may not be the first property item. We
need to check each property item until we reach the end of the property
or find GNU_PROPERTY_X86_FEATURE_1_AND.
This patch adds 2 tests. The first test checks if IBT is enabled and
the second test reads the output from the first test to check if IBT
is is enabled. The second second test fails if IBT isn't enabled
properly.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
[BZ #23467]
* sysdeps/unix/sysv/linux/x86/Makefile (tests): Add
tst-cet-property-1 and tst-cet-property-2 if CET is enabled.
(CFLAGS-tst-cet-property-1.o): New.
(ASFLAGS-tst-cet-property-dep-2.o): Likewise.
($(objpfx)tst-cet-property-2): Likewise.
($(objpfx)tst-cet-property-2.out): Likewise.
* sysdeps/unix/sysv/linux/x86/tst-cet-property-1.c: New file.
* sysdeps/unix/sysv/linux/x86/tst-cet-property-2.c: Likewise.
* sysdeps/unix/sysv/linux/x86/tst-cet-property-dep-2.S: Likewise.
* sysdeps/x86/dl-prop.h (_dl_process_cet_property_note): Parse
each property item until GNU_PROPERTY_X86_FEATURE_1_AND is found.
All tests should be added to $(tests).
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
[BZ #23458]
* sysdeps/x86/Makefile (tests): Add tst-get-cpu-features-static.
ld.so symbols to be overriden by libc need to be extern to really get
overriden.
* sysdeps/mach/hurd/not-errno.h: New file.
* sysdeps/mach/hurd/i386/localplt.data: Update accordingly.
ld.so symbols to be overriden by libc need to be extern to really get
overriden.
* sysdeps/mach/hurd/dl-unistd.h (__access, __brk, __lseek, __read,
__sbrk): Do not set attribute_hidden.
* sysdeps/mach/hurd/i386/ld.abilist: Update accordingly.
* sysdeps/mach/hurd/i386/localplt.data: Update accordingly.
Simply check if "ptr < ptr_end" since "ptr" is always incremented by 8.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/x86/dl-prop.h (_dl_process_cet_property_note): Don't
parse beyond the note end.
This patch make the OFD tests return unsupported if kernel does not
support OFD locks (it was added on 3.15).
Checked on a ia64-linux-gnu with Linux 3.14.
* sysdeps/unix/sysv/linux/tst-ofdlocks.c: Return unsupported if
kernel does not support OFD locks.
* sysdeps/unix/sysv/linux/tst-ofdlocks-compat.c: Likewise.
ld.so symbols to be overriden by libc need to be extern to really get
overriden.
More fixes are needed to avoid the hidden attribute.
* sysdeps/mach/hurd/Versions (libc): Make __access and
__access_noerrno external so they can override the ld symbols.
(ld): Make __access, __read, __sbrk, __strtoul_internal, __write,
__writev, __open64, __access_noerrno extern so they can be overrided.
* sysdeps/mach/hurd/i386/libc.abilist: Update accordingly.
* sysdeps/mach/hurd/i386/ld.abilist: Update accordingly.
cpu-features.h has
#define bit_cpu_LZCNT (1 << 5)
#define index_cpu_LZCNT COMMON_CPUID_INDEX_1
#define reg_LZCNT
But the LZCNT feature bit is in COMMON_CPUID_INDEX_80000001:
Initial EAX Value: 80000001H
ECX Extended Processor Signature and Feature Bits:
Bit 05: LZCNT available
index_cpu_LZCNT should be COMMON_CPUID_INDEX_80000001, not
COMMON_CPUID_INDEX_1. The VMX feature bit is in COMMON_CPUID_INDEX_1:
Initial EAX Value: 01H
Feature Information Returned in the ECX Register:
5 VMX
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
[BZ # 23456]
* sysdeps/x86/cpu-features.h (index_cpu_LZCNT): Set to
COMMON_CPUID_INDEX_80000001.
Set installed NPTL header as the expected one (instead of an
internal one for glibc testsuite) and add a hurd specific
stdc-predef with __STDC_NO_THREADS__.
Checked on both i686-linux-gnu and i686-gnu that both threads.h
and stdc-predef.h are the expected ones.
* nptl/threads.h: Move to ...
* sysdeps/nptl/threads.h: ... here.
* sysdeps/hurd/stdc-predef.h: New file.
Verify that setcontext works with gaps above and below the newly
allocated shadow stack.
* sysdeps/unix/sysv/linux/x86/Makefile (tests): Add
tst-cet-setcontext-1 if CET is enabled.
(CFLAGS-tst-cet-setcontext-1.c): Add -mshstk.
* sysdeps/unix/sysv/linux/x86/tst-cet-setcontext-1.c: New file.
Remove conformace assumption of NPTL implementation for ISO C threads
and revert wrong libcrypt addition on linknamespace-libs-XPG4.
The i686-gnu target now shows two new conformance failures:
FAIL: conform/ISO11/threads.h/conform
FAIL: conform/ISO11/threads.h/linknamespace
It is expected due missing HTL ISO C threads support and both conformance
.out files indicates the reason ("#error "HTL does not implement ISO C
threads").
Checked on i686-linux-gnu and i686-gnu.
* include/threads.h: Move to ...
* sysdeps/nptl/threads.h: ... here.
* sysdeps/htl/threads.h: New file.
* conform/Makefile (linknamespace-libs-ISO11): Use
static-thread-library instead of linking libpthread.
(linknamespace-libs-XPG4): Revert wrong libcrypt.a addition.
This patch adds a field to ucontext_t to save shadow stack:
1. getcontext and swapcontext are updated to save the caller's shadow
stack pointer and return addresses.
2. setcontext and swapcontext are updated to restore shadow stack and
jump to new context directly.
3. makecontext is updated to allocate a new shadow stack and set the
caller's return address to __start_context.
Since makecontext allocates a new shadow stack when making a new
context and kernel allocates a new shadow stack for clone/fork/vfork
syscalls, we track the current shadow stack base. In setcontext and
swapcontext, if the target shadow stack base is the same as the current
shadow stack base, we unwind the shadow stack. Otherwise it is a stack
switch and we look for a restore token.
We enable shadow stack at run-time only if program and all used shared
objects, including dlopened ones, are shadow stack enabled, which means
that they must be compiled with GCC 8 or above and glibc 2.28 or above.
We need to save and restore shadow stack only if shadow stack is enabled.
When caller of getcontext, setcontext, swapcontext and makecontext is
compiled with smaller ucontext_t, shadow stack won't be enabled at
run-time. We check if shadow stack is enabled before accessing the
extended field in ucontext_t.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/unix/sysv/linux/x86/sys/ucontext.h (ucontext_t): Add
__ssp.
* sysdeps/unix/sysv/linux/x86_64/__start_context.S: Include
<asm/prctl.h> and "ucontext_i.h" when shadow stack is enabled.
(__push___start_context): New.
* sysdeps/unix/sysv/linux/x86_64/getcontext.S: Include
<asm/prctl.h>.
(__getcontext): Record the current shadow stack base. Save the
caller's shadow stack pointer and base.
* sysdeps/unix/sysv/linux/x86_64/makecontext.c: Include
<pthread.h>, <libc-pointer-arith.h> and <sys/prctl.h>.
(__push___start_context): New prototype.
(__makecontext): Call __push___start_context to allocate a new
shadow stack, push __start_context onto the new stack as well
as the new shadow stack.
* sysdeps/unix/sysv/linux/x86_64/setcontext.S: Include
<asm/prctl.h>.
(__setcontext): Restore the target shadow stack.
* sysdeps/unix/sysv/linux/x86_64/swapcontext.S: Include
<asm/prctl.h>.
(__swapcontext): Record the current shadow stack base. Save
the caller's shadow stack pointer and base. Restore the target
shadow stack.
* sysdeps/unix/sysv/linux/x86_64/sysdep.h
(STACK_SIZE_TO_SHADOW_STACK_SIZE_SHIFT): New.
* sysdeps/unix/sysv/linux/x86_64/ucontext_i.sym (oSSP): New.
This will be used to record the current shadow stack base for shadow
stack switching by getcontext, makecontext, setcontext and swapcontext.
If the target shadow stack base is the same as the current shadow stack
base, we unwind the shadow stack. Otherwise it is a stack switch and
we look for a restore token to restore the target shadow stack.
* sysdeps/i386/nptl/tcb-offsets.sym (SSP_BASE_OFFSET): New.
* sysdeps/i386/nptl/tls.h (tcbhead_t): Replace __glibc_reserved2
with ssp_base.
* sysdeps/x86_64/nptl/tcb-offsets.sym (SSP_BASE_OFFSET): New.
* sysdeps/x86_64/nptl/tls.h (tcbhead_t): Replace __glibc_reserved2
with ssp_base.
CET arch_prctl bits should be defined in <asm/prctl.h> from Linux kernel
header files. Add x86 <include/asm/prctl.h> for pre-CET kernel header
files.
Note: sysdeps/unix/sysv/linux/x86/include/asm/prctl.h should be removed
if <asm/prctl.h> from the required kernel header files contains CET
arch_prctl bits.
/* CET features:
IBT: GNU_PROPERTY_X86_FEATURE_1_IBT
SHSTK: GNU_PROPERTY_X86_FEATURE_1_SHSTK
*/
/* Return CET features in unsigned long long *addr:
features: addr[0].
shadow stack base address: addr[1].
shadow stack size: addr[2].
*/
# define ARCH_CET_STATUS 0x3001
/* Disable CET features in unsigned int features. */
# define ARCH_CET_DISABLE 0x3002
/* Lock all CET features. */
# define ARCH_CET_LOCK 0x3003
/* Allocate a new shadow stack with unsigned long long *addr:
IN: requested shadow stack size: *addr.
OUT: allocated shadow stack address: *addr.
*/
# define ARCH_CET_ALLOC_SHSTK 0x3004
/* Return legacy region bitmap info in unsigned long long *addr:
address: addr[0].
size: addr[1].
*/
# define ARCH_CET_LEGACY_BITMAP 0x3005
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/unix/sysv/linux/x86/include/asm/prctl.h: New file.
* sysdeps/unix/sysv/linux/x86/cpu-features.c: Include
<sys/prctl.h> and <asm/prctl.h>.
(get_cet_status): Call arch_prctl with ARCH_CET_STATUS.
* sysdeps/unix/sysv/linux/x86/dl-cet.h: Include <sys/prctl.h>
and <asm/prctl.h>.
(dl_cet_allocate_legacy_bitmap): Call arch_prctl with
ARCH_CET_LEGACY_BITMAP.
(dl_cet_disable_cet): Call arch_prctl with ARCH_CET_DISABLE.
(dl_cet_lock_cet): Call arch_prctl with ARCH_CET_LOCK.
* sysdeps/x86/libc-start.c: Include <startup.h>.
This patch adds the thrd_* definitions from C11 threads (ISO/IEC 9899:2011),
more specifically thrd_create, thrd_curent, rhd_detach, thrd_equal,
thrd_exit, thrd_join, thrd_sleep, thrd_yield, and required types.
Mostly of the definitions are composed based on POSIX conterparts, such as
thrd_t (using pthread_t). For thrd_* function internally direct
POSIX pthread call are used with the exceptions:
1. thrd_start uses pthread_create internal implementation, but changes
how to actually calls the start routine. This is due the difference
in signature between POSIX and C11, where former return a 'void *'
and latter 'int'.
To avoid calling convention issues due 'void *' to int cast, routines
from C11 threads are started slight different than default pthread one.
Explicit cast to expected return are used internally on pthread_create
and the result is stored back to void also with an explicit cast.
2. thrd_sleep uses nanosleep internal direct syscall to avoid clobbering
errno and to handle expected standard return codes. It is a
cancellation entrypoint to be consistent with both thrd_join and
cnd_{timed}wait.
3. thrd_yield also uses internal direct syscall to avoid errno clobbering.
Checked with a build for all major ABI (aarch64-linux-gnu, alpha-linux-gnu,
arm-linux-gnueabi, i386-linux-gnu, ia64-linux-gnu, m68k-linux-gnu,
microblaze-linux-gnu [1], mips{64}-linux-gnu, nios2-linux-gnu,
powerpc{64le}-linux-gnu, s390{x}-linux-gnu, sparc{64}-linux-gnu,
and x86_64-linux-gnu).
Also ran a full check on aarch64-linux-gnu, x86_64-linux-gnu, i686-linux-gnu,
arm-linux-gnueabhf, and powerpc64le-linux-gnu.
[BZ #14092]
* conform/Makefile (conformtest-headers-ISO11): Add threads.h.
(linknamespace-libs-ISO11): Add libpthread.a.
* conform/data/threads.h-data: New file: add C11 thrd_* types and
functions.
* include/stdc-predef.h (__STDC_NO_THREADS__): Remove definition.
* nptl/Makefile (headers): Add threads.h.
(libpthread-routines): Add new C11 thread thrd_create, thrd_current,
thrd_detach, thrd_equal, thrd_exit, thrd_join, thrd_sleep, and
thrd_yield.
* nptl/Versions (libpthread) [GLIBC_2.28]): Add new C11 thread
thrd_create, thrd_current, thrd_detach, thrd_equal, thrd_exit,
thrd_join, thrd_sleep, and thrd_yield symbols.
* nptl/descr.h (struct pthread): Add c11 field.
* nptl/pthreadP.h (ATTR_C11_THREAD): New define.
* nptl/pthread_create.c (START_THREAD_DEFN): Call C11 thread start
routine with expected function prototype.
(__pthread_create_2_1): Add C11 threads check based on attribute
value.
* sysdeps/unix/sysdep.h (INTERNAL_SYSCALL_CANCEL): New macro.
* nptl/thrd_create.c: New file.
* nptl/thrd_current.c: Likewise.
* nptl/thrd_detach.c: Likewise.
* nptl/thrd_equal.c: Likewise.
* nptl/thrd_exit.c: Likewise.
* nptl/thrd_join.c: Likewise.
* nptl/thrd_priv.h: Likewise.
* nptl/thrd_sleep.c: Likewise.
* nptl/thrd_yield.c: Likewise.
* include/threads.h: Likewise.
Add <bits/indirect-return.h> and include it in <ucontext.h>.
__INDIRECT_RETURN defined in <bits/indirect-return.h> indicates if
swapcontext requires special compiler treatment. The default
__INDIRECT_RETURN is empty.
On x86, when shadow stack is enabled, __INDIRECT_RETURN is defined
with indirect_return attribute, which has been added to GCC 9, to
indicate that swapcontext returns via indirect branch. Otherwise
__INDIRECT_RETURN is defined with returns_twice attribute.
When shadow stack is enabled, remove always_inline attribute from
prepare_test_buffer in string/tst-xbzero-opt.c to avoid:
tst-xbzero-opt.c: In function ‘prepare_test_buffer’:
tst-xbzero-opt.c:105:1: error: function ‘prepare_test_buffer’ can never be inlined because it uses setjmp
prepare_test_buffer (unsigned char *buf)
when indirect_return attribute isn't available.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* bits/indirect-return.h: New file.
* misc/sys/cdefs.h (__glibc_has_attribute): New.
* sysdeps/x86/bits/indirect-return.h: Likewise.
* stdlib/Makefile (headers): Add bits/indirect-return.h.
* stdlib/ucontext.h: Include <bits/indirect-return.h>.
(swapcontext): Add __INDIRECT_RETURN.
* string/tst-xbzero-opt.c (ALWAYS_INLINE): New.
(prepare_test_buffer): Use it.
The shadow stack prevents us from pushing the saved return PC onto
the stack and returning normally. Instead we pop the shadow stack
and return directly. This is the safest way to return and ensures
any stack manipulations done by the vfork'd child doesn't cause the
parent to terminate when CET is enabled.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/unix/sysv/linux/i386/vfork.S (SYSCALL_ERROR_HANDLER):
Redefine if shadow stack is enabled.
(SYSCALL_ERROR_LABEL): Likewise.
(__vfork): Pop shadow stack and jump back to to caller directly
when shadow stack is in use.
* sysdeps/unix/sysv/linux/x86_64/vfork.S (SYSCALL_ERROR_HANDLER):
Redefine if shadow stack is enabled.
(SYSCALL_ERROR_LABEL): Likewise.
(__vfork): Pop shadow stack and jump back to to caller directly
when shadow stack is in use.
Add endbr64 to tst-quadmod1.S and tst-quadmod2.S so that func and foo
can be called indirectly.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/x86_64/tst-quadmod1.S (func): Add endbr64 if IBT is
enabled.
(foo): Likewise.
* sysdeps/x86_64/tst-quadmod2.S (func) : Likewise.
(foo): Likewise.
* scripts/check-execstack.awk: Consider `xfail' variable containing a
list
of libraries whose stack executability is expected.
* elf/Makefile ($(objpfx)check-execstack.out): Pass
$(check-execstack-xfail) to check-execstack.awk through `xfail'
variable.
* sysdeps/mach/hurd/i386/Makefile (check-execstack-xfail): Set to ld.so
libc.so libpthread.so.
* sysdeps/mach/hurd/pipe2.c: New file, copy from pipe.c. Evolve it to
implement __pipe2.
* sysdeps/mach/hurd/pipe.c (__pipe): Reimplement using __pipe2.
i386 add_n.S and sub_n.S use a trick to implment jump tables with LEA.
We can't use conditional branches nor normal jump tables since jump
table entries use EFLAGS set by jump table index. This patch adds
_CET_ENDBR to indirect jump targets and adjust destination for
_CET_ENDBR.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
* sysdeps/i386/add_n.S: Include <sysdep.h>, instead of
"sysdep.h".
(__mpn_add_n): Save and restore %ebx if IBT is enabed. Add
_CET_ENDBR to indirect jump targets and adjust jump destination
for _CET_ENDBR.
* sysdeps/i386/i686/add_n.S: Include <sysdep.h>, instead of
"sysdep.h".
(__mpn_add_n): Save and restore %ebx if IBT is enabed. Add
_CET_ENDBR to indirect jump targets and adjust jump destination
for _CET_ENDBR.
* sysdeps/i386/sub_n.S: Include <sysdep.h>, instead of
"sysdep.h".
(__mpn_sub_n): Save and restore %ebx if IBT is enabed. Add
_CET_ENDBR to indirect jump targets and adjust jump destination
for _CET_ENDBR.
Samuel Thibault
Adhemerval Zanella
H.J. Lu
Joseph Myers
Florian Weimer
Thomas Schwinge