This patch adds a generic implementation of HAVE_RM_CTX using standard
fenv calls. As a result math functions using SET_RESTORE_ROUND* macros
do not suffer from a large slowdown on targets which do not implement
optimized libc_fe*_ctx inline functions. Most of the libc_fe* inline
functions are now unused and could be removed in the future (there are
a few math functions left which use a mixture of standard fenv calls
and libc_fe* inline functions - they could be updated to use
SET_RESTORE_ROUND or improved to avoid expensive fenv manipulations
across just a few FP instructions).
libc_feholdsetround*_noex_ctx is added to enable better optimization of
SET_RESTORE_ROUND_NOEX* implementations.
Performance measurements on ARM and x86 of sin() show significant gains
over the current default, fairly close to a highly optimized fenv_private:
ARM x86
no fenv_private : 100% 100%
generic HAVE_RM_CTX : 250% 350%
fenv_private (CTX) : 250% 450%
2014-06-23 Will Newton <will.newton@linaro.org>
Wilco <wdijkstr@arm.com>
* sysdeps/generic/math_private.h: Add generic HAVE_RM_CTX
implementation. Include get-rounding-mode.h.
[!HAVE_RM_CTX]: Define HAVE_RM_CTX to zero.
[!libc_feholdsetround_noex_ctx]: Define
libc_feholdsetround_noex_ctx.
[!libc_feholdsetround_noexf_ctx]: Define
libc_feholdsetround_noexf_ctx.
[!libc_feholdsetround_noexl_ctx]: Define
libc_feholdsetround_noexl_ctx.
(libc_feholdsetround_ctx): New function.
(libc_feresetround_ctx): New function.
(libc_feholdsetround_noex_ctx): New function.
(libc_feresetround_noex_ctx): New function.
This patch updates glibc headers for changes / new definitions in
Linux 3.15. In the course of my review I noticed that
IPV6_PMTUDISC_INTERFACE was absent from glibc despite the inclusion of
IP_PMTUDISC_INTERFACE; I added it along with IP_PMTUDISC_OMIT and
IPV6_PMTUDISC_OMIT. I did not add FALLOC_FL_NO_HIDE_STALE given the
kernel header comment that it is reserved.
Tested x86_64.
* sysdeps/unix/sysv/linux/bits/fcntl-linux.h [__USE_GNU]
(FALLOC_FL_COLLAPSE_RANGE): New macro.
[__USE_GNU] (FALLOC_FL_ZERO_RANGE): Likewise.
* sysdeps/unix/sysv/linux/bits/in.h (IP_PMTUDISC_OMIT): Likewise.
(IPV6_PMTUDISC_INTERFACE): Likewise.
(IPV6_PMTUDISC_OMIT): Likewise.
Linux commit dd58a092c4202f2bd490adab7285b3ff77f8e467 added the
PPC_FEATURE2_VEC_CRYPTO auvx capability to indicate whether to
hardware supports vector crypto hardware instructions. This patch
adds its definition to powerpc hwcap bits.
Convert all uses of __builtin_expect to __glibc_likely and
__glibc_unlikely. Most of these are trivial boolean expressions
but a few were not. In particular the use of __builtin_expect in
the switch expression in do_lookup_x has been removed. Verified
that there are no code changes on x86_64 and ARM aside from line
numbers.
ChangeLog:
2014-06-23 Will Newton <will.newton@linaro.org>
* elf/dl-lookup.c: Use __glibc_unlikely and __glibc_likely
rather than __builtin_expect.
undefined_msg is only used once contrary to the comment.
ChangeLog:
2014-06-23 Will Newton <will.newton@linaro.org>
* elf/dl-lookup.c (undefined_msg): Remove variable.
(_dl_lookup_symbol_x): Replace undefined_msg with string
literal.
Move handling of STB_GNU_UNIQUE symbols to a separate function
from do_lookup_x in order to make the code more readable.
The new function gets inlined with gcc 4.8 on ARM and the
do_lookup_x code becomes a few bytes smaller.
ChangeLog:
2014-06-23 Will Newton <will.newton@linaro.org>
* elf/dl-lookup.c (do_lookup_unique): New function.
(do_lookup_x): Move STB_GNU_UNIQUE handling code
to a separate function.
At the moment the test skeleton uses a mixture of stdout and
stderr for error message output. Using stdout for all test output
keeps all output correctly ordered and properly redirected to the
output file. The suggestion to use stdout is also made on the wiki:
https://sourceware.org/glibc/wiki/Testing/Testsuite#Writing_a_test_case
ChangeLog:
2014-06-23 Will Newton <will.newton@linaro.org>
* test-skeleton.c (signal_handler): Use printf and %m
rather than perror. Use printf rather than fprintf to
stderr. Use puts rather than fputs to stderr.
(main): Likewise.
The attached patch removed the unused ‘thread_info_t’ typedef and the
‘thread_info’ variable from nscd.c. The former conflicts with a GNU Mach
typedef, and the latter conflicts with a GNU Mach function declaration:
<https://lists.gnu.org/archive/html/bug-hurd/2014-06/msg00101.html>.
Tested on x86_64-linux-gnu.
This patch removes ARM __ASSUME_SIGFRAME_V2 now that the
2.6.18-and-later signal frame layout can be assumed, renaming the
affected functions accordingly now only one version of them is needed
in glibc. (sigrestorer.S did not in fact include <kernel-features.h>
and it appears that, unlike other such cases, it didn't get the header
indirectly, so the v1 functions would have been compiled in even when
sigaction.c didn't reference them.)
(alpha and hppa also have architecture-specific __ASSUME_* macros that
should now be removed: __ASSUME_FDATASYNC and __ASSUME_LWS_CAS
respectively. I don't have any plans to do anything on that myself.)
Tested on ARM.
* sysdeps/unix/sysv/linux/arm/kernel-features.h
(__ASSUME_SIGFRAME_V2): Remove macro.
* sysdeps/unix/sysv/linux/arm/sigrestorer.S: Update comment.
[!__ASSUME_SIGFRAME_V2]: Remove conditional code.
(__default_sa_restorer_v2): Rename to __default_sa_restorer.
(__default_rt_sa_restorer_v2): Rename to __default_rt_sa_restorer.
* sysdeps/unix/sysv/linux/arm/sigaction.c (__default_sa_restorer):
Declare as function. Remove conditional macro definitions.
(__default_rt_sa_restorer): Likewise.
(__default_sa_restorer_v1): Remove declaration.
(__default_sa_restorer_v2): Likewise.
(__default_rt_sa_restorer_v1): Likewise.
(__default_rt_sa_restorer_v2): Likewise.
* sysdeps/unix/sysv/linux/arm/Versions (GLIBC_PRIVATE): Remove
__default_sa_restorer_v1, __default_rt_sa_restorer_v1,
__default_sa_restorer_v2 and __default_rt_sa_restorer_v2.
This patch makes files using __ASSUME_* macros include
<kernel-features.h> explicitly, rather than relying on some other
header (such as tls.h, lowlevellock.h or pthreadP.h) to include it
implicitly. (I omitted cases where I've already posted or am testing
the patch that stops the file from needing __ASSUME_* at all.) This
accords with the general principle of making source files include the
headers for anything they use, and also helps make it safe to remove
<kernel-features.h> includes from any file that doesn't use
__ASSUME_* (some of those may be stray includes left behind after
increasing the minimum kernel version, others may never have been
needed or may have become obsolete after some other change).
Tested x86_64 that the disassembly of installed shared libraries is
unchanged by this patch.
* nptl/pthread_cond_wait.c: Include <kernel-features.h>.
* nptl/pthread_rwlock_timedrdlock.c: Likewise.
* nptl/pthread_rwlock_timedwrlock.c: Likewise.
* nptl/sysdeps/unix/sysv/linux/lowlevelrobustlock.c: Likewise.
* nscd/nscd.c: Likewise.
* sysdeps/i386/nptl/tcb-offsets.sym: Likewise.
* sysdeps/powerpc/nptl/tcb-offsets.sym: Likewise.
* sysdeps/sh/nptl/tcb-offsets.sym: Likewise.
* sysdeps/x86_64/nptl/tcb-offsets.sym: Likewise.
This patch removes conditionals on __ASSUME_SOCK_CLOEXEC, and on
SOCK_CLOEXEC being defined, in Linux-specific code, now that all
supported Linux kernel versions can be assumed to have this
functionality. (The macro is also used in OS-independent code and is
not defined for Hurd.)
Tested x86_64 that the disassembly of installed shared libraries is
unchanged by this patch.
* nptl/sysdeps/unix/sysv/linux/mq_notify.c: Do not include
<kernel-features.h>.
(init_mq_netlink): Remove conditional have_sock_cloexec
definitions. Remove code conditional on have_sock_cloexec < 0.
(init_mq_netlink) [!SOCK_CLOEXEC]: Remove conditional code.
(init_mq_netlink) [!__ASSUME_SOCK_CLOEXEC]: Likewise.
* sysdeps/unix/sysv/linux/opensock.c: Do not include
<kernel-features.h>.
(__opensock) [SOCK_CLOEXEC]: Make code unconditional.
(__opensock) [!__ASSUME_SOCK_CLOEXEC]: Remove conditional code.
This patch adds ifunc tests for x86_64 memset_chk and memset. It also
defines HAS_AVX2 with AVX2_Usable since AVX2 may not be usable even if
processor has AVX2.
* sysdeps/x86_64/multiarch/ifunc-impl-list.c (__libc_ifunc_impl_list):
Add tests for memset_chk and memset.
* sysdeps/x86_64/multiarch/init-arch.h (HAS_AVX2): Defined
with AVX2_Usable.
This fixes an off-by-one argc interpretation error (argv[0] is included
in the count) in stdlib/tst-qsort2.c causing a segfault if the program
is called with one command-line argument only (argv[2] is NULL in that
case).
This patch removes __ASSUME_F_GETOWN_EX now it can be assumed to be
true unconditionally.
Tested x86_64 that disassembly of installed shared libraries is
unchanged by this patch.
* sysdeps/unix/sysv/linux/kernel-features.h
(__ASSUME_F_GETOWN_EX): Remove macro.
* sysdeps/unix/sysv/linux/fcntl.c: Do not include
<kernel-features.h>.
(miss_F_GETOWN_EX): Remove variable or macro.
(do_fcntl): Do not check miss_F_GETOWN_EX.
(do_fcntl) [!__ASSUME_F_GETOWN_EX]: Remove conditional code.
This patch removes __ASSUME_AT_RANDOM now it can be assumed to be true
unconditionally.
Tested x86_64 that the disassembly of installed shared libraries is
unchanged by this patch.
* sysdeps/unix/sysv/linux/kernel-features.h (__ASSUME_AT_RANDOM):
Remove macro.
* sysdeps/unix/sysv/linux/dl-osinfo.h (_dl_setup_stack_chk_guard)
[!__ASSUME_AT_RANDOM]: Remove conditional code.
(_dl_setup_pointer_guard) [!__ASSUME_AT_RANDOM]: Likewise.
This patch removes the __ASSUME_ADJ_OFFSET_SS_READ macro (and
conditionals on whether ADJ_OFFSET_SS_READ is defined), now it can be
unconditionally assumed to be true and ADJ_OFFSET_SS_READ can be
assumed to be defined.
Tested x86_64 that the disassembly of installed shared libraries is
unchanged by this patch.
* sysdeps/unix/sysv/linux/kernel-features.h
(__ASSUME_ADJ_OFFSET_SS_READ): Remove macro.
* sysdeps/unix/sysv/linux/adjtime.c (ADJTIME)
[ADJ_OFFSET_SS_READ]: Make code unconditional.
(ADJTIME) [!ADJ_OFFSET_SS_READ]: Remove conditional code.
This fixes the calculation of R_ARM_TLS_DESC relocations for lazy global
symbol references, i.e. created with `-z lazy' in effect with the static
linker, where immediate resolution is requested with LD_BIND_NOW.
This patch cleans up for __ASSUME_ATFCTS now always being true for the
supported Linux kernel versions by removing conditional code in
sysdeps/unix/sysv/linux. Several fchownat.c files that were only
present because of differences in the fallback syscalls used
(depending on the architecture-specific names of chown-related
syscalls for 32-bit uids) are removed. Files that looks like they
could be replaced by syscalls.list entries have the standard "Consider
moving to syscalls.list." comment (see bug 14138) added. Conditionals
on the relevant __NR_* syscall numbers being defined are also removed,
since my analysis indicated that the relevant syscalls are always
defined for all relevant kernel versions using any affected file.
Much of the removed fallback code had unbounded stack allocations, so
this reduces the number of cases to consider for anyone reviewing uses
of alloca and VLAs in glibc.
There remain tests of __ASSUME_ATFCTS in io/openat.c (to determine
whether to define __have_atfcts) and sysdeps/posix/getcwd.c (which
also uses __have_atfcts); thus, the definition of __ASSUME_ATFCTS
remains in kernel-features.h. The logical condition relevant there is
whether openat64_not_cancel_3 is known to work. Hurd doesn't use this
version of getcwd at all, so the conditionals in getcwd.c are always
true in glibc. However, this code is also used in gnulib. So the
best way to deal with the conditionals there may be for gnulib people
to deal with merging all relevant changes in both directions between
the glibc and gnulib versions of this file, at the end of which the
openat conditionals should be in whatever form is best for gnulib, and
hardcoded in the _LIBC case to having openat supported.
Tested by comparing before-and-after disassembly of installed
(stripped) shared libraries, on x86_64 and x86. On x86 the patch made
no change to the disassembly; on x86_64, the only changes were in
readlinkat, where formerly the return value from the readlinkat
syscall was stored in an int variable before being converted to
ssize_t for the return, and now the return value is returned directly
without truncation to int. I think it's clearly correct not to
truncate the return value (although I also think the truncation would
not have been a user-visible bug because the kernel would never have
returned a value it could have affected).
* include/fcntl.h (__atfct_seterrno): Remove prototype.
(__atfct_seterrno_2): Likewise.
* sysdeps/unix/sysv/linux/alpha/dl-fxstatat64.c: Do not include
<kernel-features.h>.
(__ASSUME_ATFCTS): Do not undefine and redefine.
* sysdeps/unix/sysv/linux/alpha/fxstatat.c [__ASSUME_ATFCTS]
(__have_atfcts): Remove conditional definition.
(__fxstatat([__NR_fstatat64]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code and code
unreachable if [__ASSUME_ATFCTS].
* sysdeps/unix/sysv/linux/dl-fxstatat64.c (__ASSUME_ATFCTS): Do
not undefine and redefine.
* sysdeps/unix/sysv/linux/faccessat.c: Do not include
<kernel-features.h>.
(faccessat) [__NR_faccessat]: Make code unconditional.
(faccessat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fchmodat.c: Do not include
<kernel-features.h>.
(fchmodat) [__NR_fchmodat]: Make code unconditional.
(fchmodat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fchownat.c: Do not include
<kernel-features.h>.
(fchownat) [__NR_fchownat]: Make code unconditional.
(fchownat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/futimesat.c: Do not include
<kernel-features.h>.
(futimesat) [__NR_futimesat]: Make code unconditional.
(futimesat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fxstatat.c: Do not include
<kernel-features.h>.
(__fxstatat) [__NR_newfstatat]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/fxstatat64.c: Do not include
<kernel-features.h>.
(__fxstatat64) [__NR_fstatat64]: Make code unconditional.
(__fxstatat64) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/i386/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/i386/fxstatat.c: Do not include
<kernel-features.h>.
(__fxstatat) [__NR_fstatat64]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/linkat.c: Do not include
<kernel-features.h>.
(linkat) [__NR_linkat]: Make code unconditional.
(linkat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/m68k/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/mips/mips64/fxstatat64.c: Do not include
<kernel-features.h>.
(__fxstatat64) [__NR_newfstatat]: Make code unconditional.
(__fxstatat64) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/mkdirat.c: Do not include
<kernel-features.h>.
(mkdirat) [__NR_mkdirat]: Make code unconditional.
(mkdirat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/openat.c: Do not include
<kernel-features.h>.
[!__ASSUME_ATFCTS] (__atfct_seterrno): Remove function.
[!__ASSUME_ATFCTS] (__have_atfcts): Remove variable.
(OPENAT_NOT_CANCEL) [__NR_openat]: Make code unconditional.
(OPENAT_NOT_CANCEL) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/powerpc/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/readlinkat.c: Do not include
<kernel-features.h>.
(readlinkat) [__NR_readlinkat]: Make code unconditional.
(readlinkat) [!__ASSUME_ATFCTS]: Remove conditional code. Return
result of INLINE_SYSCALL directly, not via int variable.
* sysdeps/unix/sysv/linux/renameat.c: Do not include
<kernel-features.h>.
[!__ASSUME_ATFCTS] (__atfct_seterrno_2): Remove function.
(renameat) [__NR_renameat]: Make code unconditional.
(renameat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/s390/s390-32/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/sh/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/sparc/sparc32/fchownat.c: Remove file.
* sysdeps/unix/sysv/linux/sparc/sparc64/dl-fxstatat64.c
(__ASSUME_ATFCTS): Do not undefine and redefine.
* sysdeps/unix/sysv/linux/symlinkat.c: Do not include
<kernel-features.h>.
(symlinkat) [__NR_symlinkat]: Make code unconditional.
(symlinkat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/unlinkat.c: Do not include
<kernel-features.h>.
(unlinkat) [__NR_unlinkat]: Make code unconditional.
(unlinkat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/wordsize-64/dl-fxstatat64.c
(__ASSUME_ATFCTS): Do not undefine and redefine.
* sysdeps/unix/sysv/linux/wordsize-64/fxstatat.c: Do not include
<kernel-features.h>.
(__fxstatat) [__NR_newfstatat]: Make code unconditional.
(__fxstatat) [!__ASSUME_ATFCTS]: Remove conditional code.
* sysdeps/unix/sysv/linux/xmknodat.c: Do not include
<kernel-features.h>.
(__xmknodat) [__NR_mknodat]: Make code unconditional.
(__xmknodat) [!__ASSUME_ATFCTS]: Remove conditional code.
Errno is not set and the testcases will fail.
Now the scalbln-aliases are removed in i386/m68
and the wrappers are used when calling the scalbln-functions.
On ia64 only scalblnf has its own implementation.
For scalbln and scalblnl the ieee754/dbl-64 and ieee754/ldbl-96 are used, thus
the wrappers are needed, too.
Since long is 4 bytes for x32, we should use 3 bytes for __pad1 when
a long __pad1 is replaced by a byte __rwelision and __pad1.
* sysdeps/x86/nptl/bits/pthreadtypes.h (pthread_rwlock_t): Use
3 bytes for __pad1 for x32.
(__PTHREAD_RWLOCK_ELISION_EXTRA): Likewise.
In this patch we take advantage of HSW memory bandwidth, manage to
reduce miss branch prediction by avoiding using branch instructions and
force destination to be aligned with avx & avx2 instruction.
The CPU2006 403.gcc benchmark indicates this patch improves performance
from 26% to 59%.
* sysdeps/x86_64/multiarch/Makefile: Add memset-avx2.
* sysdeps/x86_64/multiarch/memset-avx2.S: New file.
* sysdeps/x86_64/multiarch/memset.S: Likewise.
* sysdeps/x86_64/multiarch/memset_chk.S: Likewise.
* sysdeps/x86_64/multiarch/rtld-memset.S: Likewise.
Due to my bad review suggestion for the fix for BZ #15089 a check
was removed from systrim to prevent sbrk being called with a zero
argument. Add the check back to avoid this useless work.
ChangeLog:
2014-06-19 Will Newton <will.newton@linaro.org>
* malloc/malloc.c (systrim): If extra is zero then return
early.
Implementation of strchr for AArch64. Speedups taken from micro-bench
show the improvements relative to the standard C code.
The use of LD1 means we have identical code for both big- and
little-endian systems.
This patch fixes __ieee754_logl (-LDBL_MAX) on x86_64 and x86 not to
subtract 1 from its argument and so cause spurious overflow in
FE_DOWNWARD mode. (For any argument strictly less than -1, it doesn't
matter whether or not 1 is subtracted before computing log1p, as long
as the result doesn't overflow to -Inf.)
Tested x86_64 and x86. (This particular case lacks test coverage,
since the testsuite doesn't cover -lieee, but it will be covered by
tests after the following patch to test pow in all rounding modes,
which was the context in which this bug was found.)
[BZ #17022]
* sysdeps/i386/fpu/e_logl.S (__ieee754_logl): Do not subtract 1
from arguments -2 or below.
* sysdeps/i386/i686/fpu/e_logl.S (__ieee754_logl): Likewise.
* sysdeps/x86_64/fpu/e_logl.S (__ieee754_logl): Likewise.
The glibc makefiles have a standard variable, $(rtld-prefix), to run
the dynamic linker with a default --library-path option; this is used
as the basis of lots of other variables for running programs compiled
with the newly built library.
A few places however use $(elf-objpfx)ld.so or
$(elf-objpfx)${rtld-installed-name} directly, with such a
--library-path option. This patch makes such places use
$(rtld-prefix) instead. I'm not aware of any significance in these
cases to the choice of ld.so or ${rtld-installed-name} when running
the dynamic linker, or to whether $(patsubst
%,:%,$(sysdep-library-path)) is included in the library-path as it is
in $(rtld-prefix) and just one of the places being changed.
Tested x86_64.
* elf/Makefile ($(objpfx)tst-unused-dep.out): Use $(rtld-prefix).
* iconvdata/Makefile ($(inst_gconvdir)/gconv-modules)
[$(cross-compiling) = no]: Likewise.
* sysdeps/s390/s390-64/Makefile ($(inst_gconvdir)/gconv-modules)
[$(cross-compiling) = no]: Likewise.
localedata/ChangeLog:
* Makefile (LOCALEDEF): Use $(rtld-prefix).
This patch fixes few failures in nearbyintl() where the fraction part is
close to 0.5.i The new tests added report few extra failures in
nearbyint_downward and nearbyint_towardzero which is a known issue.
Fixes#17031.
Wilco Dijkstra
Roland McGrath
Joseph Myers
Andreas Schwab
Adhemerval Zanella
Will Newton
Ludovic Courtès
Allan McRae
H.J. Lu
Maciej W. Rozycki
Andreas Schwab
Siddhesh Poyarekar
Stefan Liebler
Ling Ma
Richard Earnshaw
Rajalakshmi Srinivasaraghavan