This patch provides optimized versions of strcmp and wcscmp with the z13
vector instructions.
The architecture specific string.h had a typo, which leads to ommiting the
inline version in this file if __USE_STRING_INLINES is defined.
Tested this inline version by tweaking test-strcmp.c.
ChangeLog:
* sysdeps/s390/multiarch/strcmp-vx.S: New File.
* sysdeps/s390/multiarch/strcmp.c: Likewise.
* sysdeps/s390/multiarch/wcscmp-c.c: Likewise.
* sysdeps/s390/multiarch/wcscmp-vx.S: Likewise.
* sysdeps/s390/multiarch/wcscmp.c: Likewise.
* sysdeps/s390/s390-32/multiarch/strcmp.c: Likewise.
* sysdeps/s390/s390-64/multiarch/strcmp.c: Likewise.
* sysdeps/s390/multiarch/Makefile (sysdep_routines): Add strcmp and
wcscmp functions.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(__libc_ifunc_impl_list): Add ifunc test for strcmp, wcscmp.
* string/strcmp.c (STRCMP): Define and use macro.
* benchtests/bench-wcscmp.c: New File.
* benchtests/Makefile (wcsmbs-bench): Add wcscmp.
* sysdeps/s390/bits/string.h: Fix typo: _HAVE_STRING_ARCH_strcmp
instead of _HAVE_STRING_ARCH_memchr.
This patch provides optimized versions of strlen and wcslen with the z13 vector
instructions.
The helper macro IFUNC_VX_IMPL is introduced and is used to register all
__<func>_c() and __<func>_vx() functions within __libc_ifunc_impl_list()
to the ifunc test framework.
ChangeLog:
* sysdeps/s390/multiarch/Makefile: New File.
* sysdeps/s390/multiarch/strlen-c.c: Likewise.
* sysdeps/s390/multiarch/strlen-vx.S: Likewise.
* sysdeps/s390/multiarch/strlen.c: Likewise.
* sysdeps/s390/multiarch/wcslen-c.c: Likewise.
* sysdeps/s390/multiarch/wcslen-vx.S: Likewise.
* sysdeps/s390/multiarch/wcslen.c: Likewise.
* string/strlen.c (STRLEN): Define and use macro.
* sysdeps/s390/multiarch/ifunc-impl-list.c
(IFUNC_VX_IMPL): New macro function.
(__libc_ifunc_impl_list): Add ifunc test for strlen, wcslen.
* benchtests/Makefile (wcsmbs-bench): New variable.
(string-bench-all): Added wcsmbs-bench.
* benchtests/bench-wcslen.c: New File.
I think the last clause of the conditional,
|| __n <= __bos (__dest)
may be backward. The code should call the runtime-checking function
if __n is not constant, or if __n is known to be LARGER than the size
of the destination.
This patches uses the default strcpy/stpcpy implementation for
POWER7/PPC64. This is faster in mostly inputs for benchtests
and for multiarch the implementation uses the POWER7 strlen and
memcpy.
* string/stpcpy.c (__stpcpy): Use STPCPY to redefine symbol name and
cleanup macro usage.
* string/strcpy.c (strcpt): Use STRCPY to redefine symbol name.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy-power7.S: Remove file.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy-ppc64.S: Likewise.
* sysdeps/powerpc/powerpc64/multiarch/strcpy-power7.S: Likewise.
* sysdeps/powerpc/powerpc64/multiarch/strcpy-ppc64.S: Likewise.
* sysdeps/powerpc/powerpc64/power7/stpcpy.S: Likewise.
* sysdeps/powerpc/powerpc64/power7/strcpy.S: Likewise.
* sysdeps/powerpc/powerpc64/power7/strcpy.c: Likewise.
* sysdeps/powerpc/powerpc64/stpcpy.S: Likewise.
* sysdeps/powerpc/powerpc64/strcpy.S: Likewise.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy.c
[SHARED && IS_IN (libc)]: Include <string/strcpy.c>.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy.c
[SHARED && IS_IN (libc)]: Include <string/stpcpy.c>.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy-power7.c: New file.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy-ppc64.c: Likewise.
* sysdeps/powerpc/powerpc64/multiarch/strcpy-power7.c: Likewise.
* sysdeps/powerpc/powerpc64/multiarch/strcpy-ppc64.c: Likewise.
* sysdeps/powerpc/powerpc64/power7/strcpy.c: Likewise.
These tests were skipped by the use-test-skeleton conversion done in
commit 29955b5d because they did not have an `int main (void)'
declaration. Instead their `main' functions were declared with arguments
(i.e. argc, argv) even though they didn't use them.
Remove these arguments and include the test skeleton in these tests.
on bench-strncpy is 1.9-2.1x faster on average. I tried several variations, and using a tailcall and
calling memset conditionally gave the best overall results.
Various code in glibc uses __strnlen instead of strnlen for namespace
reasons. However, __strnlen does not use libc_hidden_proto /
libc_hidden_def (as is normally done for any function defined and
called within the same library, whether or not exported from the
library and whatever namespace it is in), so the compiler does not
know that those calls are to a function within libc.
This patch uses libc_hidden_proto / libc_hidden_def with __strnlen.
On x86_64, it makes no difference to the installed stripped shared
libraries. On 32-bit x86, it causes __strnlen calls to go to the same
place as strnlen calls (the fallback strnlen implementation), rather
than through a PLT entry for the strnlen IFUNC; I'm not sure of the
logic behind when calls from within libc should use IFUNCs versus when
they should go direct to a particular function implementation, but
clearly it doesn't make sense for strnlen and __strnlen to be handled
differently in this regard.
Tested for x86_64 and x86 (testsuite, and comparison of installed
shared libraries as described above).
* string/strnlen.c [!STRNLEN] (__strnlen): Use libc_hidden_def.
* include/string.h (__strnlen): Use libc_hidden_proto.
* sysdeps/aarch64/strnlen.S (__strnlen): Use libc_hidden_def.
* sysdeps/i386/i686/multiarch/strnlen-c.c [SHARED]
(libc_hidden_def): Define __GI___strnlen as well as __GI_strnlen.
* sysdeps/powerpc/powerpc32/power4/multiarch/strnlen-power7.S
(libc_hidden_def): Undefine and redefine.
* sysdeps/powerpc/powerpc32/power4/multiarch/strnlen-ppc32.c
[SHARED] (libc_hidden_def): Define __GI___strnlen as well as
__GI_strnlen.
* sysdeps/powerpc/powerpc32/power7/strnlen.S (__strnlen): Use
libc_hidden_def.
* sysdeps/tile/tilegx/strnlen.c (__strnlen): Likewise.
pathconf (sysdeps/unix/sysv/linux/pathconf.c) uses basename. But
pathconf is in POSIX back to 1990 while basename is only reserved with
external linkage in those standards including XPG functions. This
patch fixes this namespace issue in the usual way, renaming basename
to __basename and making it into a weak alias.
Tested for x86_64 and x86 (testsuite, and that disassembly of
installed shared libraries is unchanged by the patch).
[BZ #18444]
* string/basename.c (basename): Rename to __basename and define as
weak alias of __basename. Use libc_hidden_weak.
* include/string.h (__basename): Declare. Use libc_hidden_proto.
* sysdeps/unix/sysv/linux/pathconf.c (distinguish_extX): Call
__basename instead of basename.
* conform/Makefile (test-xfail-POSIX2008/unistd.h/linknamespace):
Remove variable.
(test-xfail-XOPEN2K8/unistd.h/linknamespace): Likewise.
[BZ #18206]
* wcsmbs/wcsncmp.c (wcsncmp): Compare as wchar_t, not wint_t.
Use signed comparision instead of substraction to avoid
overflow bug.
* localedata/tests-mbwc/tst_wcsncmp.c (tst_wcsncmp):
Take the sign of ret.
* localedata/tests-mbwc/dat_wcsncmp.c (tst_wcsncmp_loc):
Do not expect precise return values. Only the sign matters.
* wcsmbs/Makefile (strop-tests): Add wcsncmp.
* wcsmbs/test-wcsncmp.c: New File.
* string/test-strncmp.c: Add wcsncmp support.
[Modified from the original email by Siddhesh Poyarekar]
This patch solves bug #16009 by implementing an additional path in
strxfrm that does not depend on caching the weight and rule indices.
In detail the following changed:
* The old main loop was factored out of strxfrm_l into the function
do_xfrm_cached to be able to alternativly use the non-caching version
do_xfrm.
* strxfrm_l allocates a a fixed size array on the stack. If this is not
sufficiant to store the weight and rule indices, the non-caching path is
taken. As the cache size is not dependent on the input there can be no
problems with integer overflows or stack allocations greater than
__MAX_ALLOCA_CUTOFF. Note that malloc-ing is not possible because the
definition of strxfrm does not allow an oom errorhandling.
* The uncached path determines the weight and rule index for every char
and for every pass again.
* Passing all the locale data array by array resulted in very long
parameter lists, so I introduced a structure that holds them.
* Checking for zero src string has been moved a bit upwards, it is
before the locale data initialization now.
* To verify that the non-caching path works correct I added a test run
to localedata/sort-test.sh & localedata/xfrm-test.c where all strings
are patched up with spaces so that they are too large for the caching path.
gcc now warns when the arguments to memset may have been accidentally
transposed (i.e. length set to zero instead of the byte), so we don't
need that bit of the code in glibc headers anymore.
Tested on x86_64. Coe generated by gcc 4.8 is identical with or
without the patch. I also tested gcc master, which does not result in
any new failures. It does fail quite a few FORTIFY_SOURCE tests, but
those failures are not due to this patch.
TEST_IFUNC is only tested in two headers, bench-string.h and
test-string.h, after it gets defined by those headers, and it never
gets undefined.
Thus no defines of TEST_IFUNC are needed, and the *-ifunc.c tests that
just define TEST_IFUNC and include other tests are also redundant, as
is the code to remove $(tests-ifunc) and $(xtests-ifunc) conditionally
from tests and xtests. This patch removes the useless defines and
tests of TEST_IFUNC and the associated useless tests and makefile
code. It thereby fixes a series of warnings
"../string/test-string.h:21:0: warning: "TEST_IFUNC" redefined" where
test-string.h defines TEST_IFUNC to empty, other files define it to 1
and this produces warnings.
Tested for x86_64.
* debug/test-stpcpy_chk-ifunc.c: Remove file.
* debug/test-strcpy_chk-ifunc.c: Likewise.
* wcsmbs/test-wcschr-ifunc.c: Likewise.
* wcsmbs/test-wcscmp-ifunc.c: Likewise.
* wcsmbs/test-wcscpy-ifunc.c: Likewise.
* wcsmbs/test-wcslen-ifunc.c: Likewise.
* wcsmbs/test-wcsrchr-ifunc.c: Likewise.
* wcsmbs/test-wmemcmp-ifunc.c: Likewise.
* Rules [$(multi-arch) = no] (tests): Do not filter out
$(tests-ifunc).
[$(multi-arch) = no] (xtests): Do not filter out $(xtests-ifunc).
* debug/Makefile (tests-ifunc): Remove variable.
(tests): Do not add $(tests-ifunc).
* wcsmbs/Makefile (tests-ifunc): Remove variable.
(tests): Do not add $(tests-ifunc).
* benchtests/bench-string.h (TEST_IFUNC): Remove macro.
[TEST_IFUNC]: Remove conditionals.
* string/test-string.h (TEST_IFUNC): Remove macro.
[TEST_IFUNC]: Remove conditionals.
string/test-strchr.c is used for both wide and narrow string testing,
but produces a series of warnings for wide string testing because of
hardcoded use of narrow characters in the function check1. This patch
fixes that function to use macros abstracting away the wide / narrow
string choice, adding a new such macro to handle the string and
character constants.
Tested for x86_64.
* string/test-strchr.c [!WIDE] (L): New macro.
[WIDE] (L): Likewise.
(check1): Use CHAR instead of char. Use L on string and character
constants.
This patch fixes the build of C mempcpy and stpcpy by disabling the
redirection to __mempcpy and __stpcpy asm names if
NO_MEMPCPY_STPCPY_REDIRECT is defined, and defining that macro in the
relevant source files.
Tested for powerpc32 that the build is fixed.
* include/string.h [NO_MEMPCPY_STPCPY_REDIRECT] (mempcpy): Do not
redeclare with asm name.
[NO_MEMPCPY_STPCPY_REDIRECT] (stpcpy): Likewise.
* string/mempcpy.c (NO_MEMPCPY_STPCPY_REDIRECT): Define before
including <string.h>.
* string/stpcpy.c (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
* sysdeps/powerpc/powerpc32/power4/multiarch/mempcpy.c
[!NOT_IN_libc] (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
* sysdeps/powerpc/powerpc64/multiarch/mempcpy.c
[!NOT_IN_libc] (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy.c
[SHARED && !NOT_IN_libc] (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
Locale code, brought in by ISO C functions, calls memmem, which is not
an ISO C function. This isn't an ISO C conformance bug, because all
mem* names are reserved, but glibc practice is not to rely on that
reservation (thus, memmem is only declared in string.h if __USE_GNU
even though ISO C would allow it to be declared unconditionally, for
example). This patch changes that code to use __memmem.
Note: there are uses of memmem elsewhere in glibc that I didn't
change, although it may turn out some of those also need to use
__memmem.
Tested for x86_64 (testsuite, and that disassembly of installed shared
libraries is unchanged by this patch).
[BZ #17585]
* string/memmem.c [!_LIBC] (__memmem): Define to memmem.
(memmem): Rename to __memmem and define as weak alias of
__memmem. Use libc_hidden_weak.
(__memmem): Use libc_hidden_def.
* include/string.h (__memmem): Declare. Use libc_hidden_proto.
* locale/findlocale.c (valid_locale_name): Use __memmem instead of
memmem.
Modifies the test examination in test-skeleton.c so that a test can be
successful if it is interrupted or it returns uninterrupted with the
expected status. For this both EXPECTED_SIGNAL and EXPECTED_STATUS
have to be set, as is done in tst-strcoll-overflow.c.
this
will improve performance even on targets which don't have an optimized strlen. It is about twice
as
fast as the original strncat in bench-strncat.
implementation, so this improves performance even on targets which don't have an optimized
strlen and strcpy - it is 25% faster in bench-strcat. On targets which don't provide an
optimized strcat but which do have an optimized strlen and strcpy, performance gain is > 2x.
Stefan Liebler
Wilco Dijkstra
Paul Eggert
Zack Weinberg
Adhemerval Zanella
Arjun Shankar
Leonhard Holz
Joseph Myers
Andreas Schwab
Ondřej Bílka
Richard Earnshaw
Torvald Riegel
Siddhesh Poyarekar