[BZ #18206]
* wcsmbs/wcsncmp.c (wcsncmp): Compare as wchar_t, not wint_t.
Use signed comparision instead of substraction to avoid
overflow bug.
* localedata/tests-mbwc/tst_wcsncmp.c (tst_wcsncmp):
Take the sign of ret.
* localedata/tests-mbwc/dat_wcsncmp.c (tst_wcsncmp_loc):
Do not expect precise return values. Only the sign matters.
* wcsmbs/Makefile (strop-tests): Add wcsncmp.
* wcsmbs/test-wcsncmp.c: New File.
* string/test-strncmp.c: Add wcsncmp support.
[Modified from the original email by Siddhesh Poyarekar]
This patch solves bug #16009 by implementing an additional path in
strxfrm that does not depend on caching the weight and rule indices.
In detail the following changed:
* The old main loop was factored out of strxfrm_l into the function
do_xfrm_cached to be able to alternativly use the non-caching version
do_xfrm.
* strxfrm_l allocates a a fixed size array on the stack. If this is not
sufficiant to store the weight and rule indices, the non-caching path is
taken. As the cache size is not dependent on the input there can be no
problems with integer overflows or stack allocations greater than
__MAX_ALLOCA_CUTOFF. Note that malloc-ing is not possible because the
definition of strxfrm does not allow an oom errorhandling.
* The uncached path determines the weight and rule index for every char
and for every pass again.
* Passing all the locale data array by array resulted in very long
parameter lists, so I introduced a structure that holds them.
* Checking for zero src string has been moved a bit upwards, it is
before the locale data initialization now.
* To verify that the non-caching path works correct I added a test run
to localedata/sort-test.sh & localedata/xfrm-test.c where all strings
are patched up with spaces so that they are too large for the caching path.
gcc now warns when the arguments to memset may have been accidentally
transposed (i.e. length set to zero instead of the byte), so we don't
need that bit of the code in glibc headers anymore.
Tested on x86_64. Coe generated by gcc 4.8 is identical with or
without the patch. I also tested gcc master, which does not result in
any new failures. It does fail quite a few FORTIFY_SOURCE tests, but
those failures are not due to this patch.
TEST_IFUNC is only tested in two headers, bench-string.h and
test-string.h, after it gets defined by those headers, and it never
gets undefined.
Thus no defines of TEST_IFUNC are needed, and the *-ifunc.c tests that
just define TEST_IFUNC and include other tests are also redundant, as
is the code to remove $(tests-ifunc) and $(xtests-ifunc) conditionally
from tests and xtests. This patch removes the useless defines and
tests of TEST_IFUNC and the associated useless tests and makefile
code. It thereby fixes a series of warnings
"../string/test-string.h:21:0: warning: "TEST_IFUNC" redefined" where
test-string.h defines TEST_IFUNC to empty, other files define it to 1
and this produces warnings.
Tested for x86_64.
* debug/test-stpcpy_chk-ifunc.c: Remove file.
* debug/test-strcpy_chk-ifunc.c: Likewise.
* wcsmbs/test-wcschr-ifunc.c: Likewise.
* wcsmbs/test-wcscmp-ifunc.c: Likewise.
* wcsmbs/test-wcscpy-ifunc.c: Likewise.
* wcsmbs/test-wcslen-ifunc.c: Likewise.
* wcsmbs/test-wcsrchr-ifunc.c: Likewise.
* wcsmbs/test-wmemcmp-ifunc.c: Likewise.
* Rules [$(multi-arch) = no] (tests): Do not filter out
$(tests-ifunc).
[$(multi-arch) = no] (xtests): Do not filter out $(xtests-ifunc).
* debug/Makefile (tests-ifunc): Remove variable.
(tests): Do not add $(tests-ifunc).
* wcsmbs/Makefile (tests-ifunc): Remove variable.
(tests): Do not add $(tests-ifunc).
* benchtests/bench-string.h (TEST_IFUNC): Remove macro.
[TEST_IFUNC]: Remove conditionals.
* string/test-string.h (TEST_IFUNC): Remove macro.
[TEST_IFUNC]: Remove conditionals.
string/test-strchr.c is used for both wide and narrow string testing,
but produces a series of warnings for wide string testing because of
hardcoded use of narrow characters in the function check1. This patch
fixes that function to use macros abstracting away the wide / narrow
string choice, adding a new such macro to handle the string and
character constants.
Tested for x86_64.
* string/test-strchr.c [!WIDE] (L): New macro.
[WIDE] (L): Likewise.
(check1): Use CHAR instead of char. Use L on string and character
constants.
This patch fixes the build of C mempcpy and stpcpy by disabling the
redirection to __mempcpy and __stpcpy asm names if
NO_MEMPCPY_STPCPY_REDIRECT is defined, and defining that macro in the
relevant source files.
Tested for powerpc32 that the build is fixed.
* include/string.h [NO_MEMPCPY_STPCPY_REDIRECT] (mempcpy): Do not
redeclare with asm name.
[NO_MEMPCPY_STPCPY_REDIRECT] (stpcpy): Likewise.
* string/mempcpy.c (NO_MEMPCPY_STPCPY_REDIRECT): Define before
including <string.h>.
* string/stpcpy.c (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
* sysdeps/powerpc/powerpc32/power4/multiarch/mempcpy.c
[!NOT_IN_libc] (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
* sysdeps/powerpc/powerpc64/multiarch/mempcpy.c
[!NOT_IN_libc] (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
* sysdeps/powerpc/powerpc64/multiarch/stpcpy.c
[SHARED && !NOT_IN_libc] (NO_MEMPCPY_STPCPY_REDIRECT): Likewise.
Locale code, brought in by ISO C functions, calls memmem, which is not
an ISO C function. This isn't an ISO C conformance bug, because all
mem* names are reserved, but glibc practice is not to rely on that
reservation (thus, memmem is only declared in string.h if __USE_GNU
even though ISO C would allow it to be declared unconditionally, for
example). This patch changes that code to use __memmem.
Note: there are uses of memmem elsewhere in glibc that I didn't
change, although it may turn out some of those also need to use
__memmem.
Tested for x86_64 (testsuite, and that disassembly of installed shared
libraries is unchanged by this patch).
[BZ #17585]
* string/memmem.c [!_LIBC] (__memmem): Define to memmem.
(memmem): Rename to __memmem and define as weak alias of
__memmem. Use libc_hidden_weak.
(__memmem): Use libc_hidden_def.
* include/string.h (__memmem): Declare. Use libc_hidden_proto.
* locale/findlocale.c (valid_locale_name): Use __memmem instead of
memmem.
Modifies the test examination in test-skeleton.c so that a test can be
successful if it is interrupted or it returns uninterrupted with the
expected status. For this both EXPECTED_SIGNAL and EXPECTED_STATUS
have to be set, as is done in tst-strcoll-overflow.c.
this
will improve performance even on targets which don't have an optimized strlen. It is about twice
as
fast as the original strncat in bench-strncat.
implementation, so this improves performance even on targets which don't have an optimized
strlen and strcpy - it is 25% faster in bench-strcat. On targets which don't provide an
optimized strcat but which do have an optimized strlen and strcpy, performance gain is > 2x.
this is a path that should solve bug 15884. It complains about the performance
of strcoll(). It was found out that the runtime of strcoll() is actually bound
to strlen which is needed for calculating the size of a cache that was
installed to improve the comparison performance.
The idea for this patch was that the cache is only useful in rare cases
(strings of same length and same first-level-chars) and that it would be
better to avoid memory allocation at all. To prove this I wrote a performance
test bench-strcoll.c with test data in benchtests-strcoll.tar.gz. Also
modifications in benchtests/Makefile and localedata/Makefile are necessary to
make it work.
After removing the cache the strcoll method showed the predicted behavior
(getting slightly faster) in all but the test case for hindi word sorting.
This was due the hindi text having much more equal words than the other ones.
For equal strings the performance was worse since all comparison levels were
run through and from the second level on the cache improved the comparison
performance of the original version.
Therefore I added a bytewise test via strcmp iff the first level comparison
found that both strings did match because in this case it is very likely that
equal strings are compared. This solved the problem with the hindi test case
and improved the performance of the others.
Performance comparison:
glibc files -33.77%
vi_VN.UTF-8 -34.12%
en_US.UTF-8 -42.42%
ar_SA.UTF-8 -27.49%
zh_CN.UTF-8 +07.90%
cs_CZ.UTF-8 -29.67%
en_GB.UTF-8 -28.50%
da_DK.UTF-8 -36.57%
pl_PL.UTF-8 -39.31%
fr_FR.UTF-8 -28.57%
pt_PT.UTF-8 -22.82%
el_GR.UTF-8 -26.77%
ru_RU.UTF-8 -35.81%
iw_IL.UTF-8 -35.34%
es_ES.UTF-8 -34.46%
hi_IN.UTF-8 -00.38%
sv_SE.UTF-8 -36.99%
hu_HU.UTF-8 -16.35%
tr_TR.UTF-8 -27.80%
is_IS.UTF-8 -33.24%
it_IT.UTF-8 -24.39%
sr_RS.UTF-8 -37.55%
ja_JP.UTF-8 +02.84%
This patch adds an optimized memmove optimization for POWER7/powerpc64.
Basically the idea is to use the memcpy for POWER7 on non-overlapped
memory regions and a optimized backward memcpy for memory regions
that overlap (similar to the idea of string/memmove.c).
The backward memcpy algorithm used is similar the one use for memcpy for
POWER7, with adjustments done for alignment. The difference is memory
is always aligned to 16 bytes before using VSX/altivec instructions.
Merge most of the gnulib implementation of memchr. The changes that
remain are:
- copyright header
- bp-sym.h removed
- reg_char removed
- allow MEMCHR to be redefined
- non-conforming whitespace changes
The merged code fixes a number of -Wundef warnings and also introduces
an optimized algorithm. I haven't detected any performance difference
in the new code which I believe is down to the quite specific
circumstances required to hit it. However the new code is approximately
half the size of the old code on AArch64 (which uses generic memchr).
ChangeLog:
2014-07-04 Will Newton <will.newton@linaro.org>
* string/memchr.c: Merge from gnulib.
[_LIBC]: Remove conditionals.
(__ptr_t): Remove define.
(LONG_MAX_32_BITS): Likewise.
(LONG_MAX): Likewise.
(MEMCHR): Use ANSI prototype and optimize algorithm.
The PAGE_COPY_THRESHOLD macro is meant to be overridden by
architecture-specific pagecopy.h, but it is currently done only by
mach; all other architectures use the default. Check to see if the
macro is defined in addition to whether it is set to a non-zero value.
Various glibc build / install / test code has C locale settings that
are redundant with LC_ALL=C.
LC_ALL takes precedence over LANG, so anywhere that sets LC_ALL=C
(explicitly, or through it being in the default environment for
running tests) does not need to set LANG=C. LC_ALL=C also takes
precedence over LANGUAGE, since
2001-01-02 Ulrich Drepper <drepper@redhat.com>
* intl/dcigettext.c (guess_category_value): Rewrite so that LANGUAGE
value is ignored if the selected locale is the C locale.
* intl/tst-gettext.c: Set locale for above change.
* intl/tst-translit.c: Likewise.
and so settings of LANGUAGE=C are also redundant when LC_ALL=C is
set. One test also had LC_ALL=C in its -ENV setting, although it's
part of the default environment used for tests.
This patch removes the redundant settings. It removes a suggestion in
install.texi of setting LANGUAGE=C LC_ALL=C for "make install"; the
Makefile.in target "install" already sets LC_ALL_C so there's no need
for the user to set it (and nor should there be any need for the user
to set it).
If some build machine tool used by "make install" uses a version of
libintl predating that 2001 change, and the user has LANGUAGE set, the
removal of LANGUAGE=C from the Makefile.in "install" rule could in
principle affect the user's installation. However, I don't think we
need to be concerned about pre-2001 build tools.
Tested x86_64.
* Makefile (install): Don't set LANGUAGE.
* Makefile.in (install): Likewise.
* assert/Makefile (test-assert-ENV): Remove variable.
(test-assert-perr-ENV): Likewise.
* elf/Makefile (neededtest4-ENV): Likewise.
* iconvdata/Makefile ($(inst_gconvdir)/gconv-modules)
[$(cross-compiling) = no]: Don't set LANGUAGE.
* io/ftwtest-sh (LANG): Remove variable.
* libio/Makefile (tst-widetext-ENV): Likewise.
* manual/install.texi (Running make install): Don't refer to
environment settings for make install.
* INSTALL: Regenerated.
* nptl/tst-tls6.sh: Don't set LANG.
* posix/globtest.sh (LANG): Remove variable.
* string/Makefile (tester-ENV): Likewise.
(inl-tester-ENV): Likewise.
(noinl-tester-ENV): Likewise.
* sysdeps/s390/s390-64/Makefile ($(inst_gconvdir)/gconv-modules)
[$(cross-compiling) = no]: Don't set LANGUAGE.
* timezone/Makefile (build-testdata): Use $(built-program-cmd)
without explicit environment settings.
localedata/ChangeLog:
* tst-fmon.sh: Don't set LANGUAGE.
* tst-locale.sh: Likewise.
Tests run using the default $(make-test-out) automatically get
GCONV_PATH and LC_ALL set, whether or not those environment variables
are actually needed for the individual test. However, they do not get
LOCPATH set, meaning that a large number of tests have -ENV settings
just to set LOCPATH.
This patch moves LOCPATH into the default environment used for all
tests, on the principle that like GCONV_PATH any settings needed to
use files associated with the newly built library, rather than any old
installed files, are appropriate to use by default.
A further motivation is that various tests using .sh files also set
some combination of LC_ALL, GCONV_PATH and LOCPATH. Preferably .sh
files should also use the default environment with any additions
required for the individual test. Now, it was suggested in
<https://sourceware.org/ml/libc-alpha/2014-05/msg00715.html> that
various Makefile variables used in testing should be derived by
composing the -before-env and -after-env variables used when explicit
environment settings are required. With such a change, it's also
natural for those variables to include the default settings (via some
intermediate makefile variable also used in make-test-out).
Because some .sh files only set variables that correspond to the
default settings, or a subset thereof, and this applies to more of the
.sh files once LOCPATH is in the default settings, doing so reduces
the size of a revised version of
<https://sourceware.org/ml/libc-alpha/2014-05/msg00596.html>: scripts
only needing the (expanded) default settings will not need to receive
the separate -before-env and -after-env variables, only the single
variable they do at present. So moving LOCPATH into the default
settings can reduce churn caused by subsequent patches.
Tested x86_64 and x86.
* Rules (make-test-out): Include
LOCPATH=$(common-objpfx)localedata in default environment.
* debug/Makefile (tst-chk1-ENV): Remove variable.
(tst-chk2-ENV): Likewise.
(tst-chk3-ENV): Likewise.
(tst-chk4-ENV): Likewise.
(tst-chk5-ENV): Likewise.
(tst-chk6-ENV): Likewise.
(tst-lfschk1-ENV): Likewise.
(tst-lfschk2-ENV): Likewise.
(tst-lfschk3-ENV): Likewise.
(tst-lfschk4-ENV): Likewise.
(tst-lfschk5-ENV): Likewise.
(tst-lfschk6-ENV): Likewise.
* iconvdata/Makefile (bug-iconv6-ENV): Likewise.
(tst-iconv7-ENV): Likewise.
* intl/Makefile (LOCPATH-ENV): Likewise.
(tst-codeset-ENV): Likewise.
(tst-gettext3-ENV): Likewise.
(tst-gettext5-ENV): Likewise.
* libio/Makefile (tst-widetext-ENV): Don't set LOCPATH.
(tst-fopenloc-ENV): Likewise.
(tst-fgetws-ENV): Remove variable.
(tst-ungetwc1-ENV): Likewise.
(tst-ungetwc2-ENV): Likewise.
(bug-ungetwc2-ENV): Likewise.
(tst-swscanf-ENV): Likewise.
(bug-ftell-ENV): Likewise.
(tst-fgetwc-ENV): Likewise.
(tst-fseek-ENV): Likewise.
(tst-ftell-partial-wide-ENV): Likewise.
(tst-ftell-active-handler-ENV): Likewise.
(tst-ftell-append-ENV): Likewise.
* posix/Makefile (tst-fnmatch-ENV): Likewise.
(tst-regexloc-ENV): Likewise.
(bug-regex1-ENV): Likewise.
(tst-regex-ENV): Likewise.
(tst-regex2-ENV): Likewise.
(bug-regex5-ENV): Likewise.
(bug-regex6-ENV): Likewise.
(bug-regex17-ENV): Likewise.
(bug-regex18-ENV): Likewise.
(bug-regex19-ENV): Likewise.
(bug-regex20-ENV): Likewise.
(bug-regex22-ENV): Likewise.
(bug-regex23-ENV): Likewise.
(bug-regex25-ENV): Likewise.
(bug-regex26-ENV): Likewise.
(bug-regex30-ENV): Likewise.
(bug-regex32-ENV): Likewise.
(bug-regex33-ENV): Likewise.
(bug-regex34-ENV): Likewise.
(bug-regex35-ENV): Likewise.
(tst-rxspencer-ENV): Likewise.
(tst-rxspencer-no-utf8-ENV): Likewise.
* stdio-common/Makefile (tst-sprintf-ENV): Likewise.
(tst-sscanf-ENV): Likewise.
(tst-swprintf-ENV): Likewise.
(tst-swscanf-ENV): Likewise.
(test-vfprintf-ENV): Likewise.
(scanf13-ENV): Likewise.
(bug14-ENV): Likewise.
(tst-grouping-ENV): Likewise.
* stdlib/Makefile (tst-strtod-ENV): Likewise.
(tst-strtod3-ENV): Likewise.
(tst-strtod4-ENV): Likewise.
(tst-strtod5-ENV): Likewise.
(testmb2-ENV): Likewise./
* string/Makefile (tst-strxfrm-ENV): Likewise.
(tst-strxfrm2-ENV): Likewise.
(bug-strcoll1-ENV): Likewise.
(test-strcasecmp-ENV): Likewise.
(test-strncasecmp-ENV): Likewise.
* time/Makefile (tst-strptime-ENV): Likewise.
(tst-ftime_l-ENV): Likewise.
* wcsmbs/Makefile (tst-btowc-ENV): Likewise.
(tst-mbrtowc-ENV): Likewise.
(tst-wcrtomb-ENV): Likewise.
(tst-mbrtowc2-ENV): Likewise.
(tst-c16c32-1-ENV): Likewise.
(tst-mbsnrtowcs-ENV): Likewise.
localedata/ChangeLog:
* Makefile (TEST_MBWC_ENV): Remove variable.
(tst_iswalnum-ENV): Likewise.
(tst_iswalpha-ENV): Likewise.
(tst_iswcntrl-ENV): Likewise.
(tst_iswctype-ENV): Likewise.
(tst_iswdigit-ENV): Likewise.
(tst_iswgraph-ENV): Likewise.
(tst_iswlower-ENV): Likewise.
(tst_iswprint-ENV): Likewise.
(tst_iswpunct-ENV): Likewise.
(tst_iswspace-ENV): Likewise.
(tst_iswupper-ENV): Likewise.
(tst_iswxdigit-ENV): Likewise.
(tst_mblen-ENV): Likewise.
(tst_mbrlen-ENV): Likewise.
(tst_mbrtowc-ENV): Likewise.
(tst_mbsrtowcs-ENV): Likewise.
(tst_mbstowcs-ENV): Likewise.
(tst_mbtowc-ENV): Likewise.
(tst_strcoll-ENV): Likewise.
(tst_strfmon-ENV): Likewise.
(tst_strxfrm-ENV): Likewise.
(tst_swscanf-ENV): Likewise.
(tst_towctrans-ENV): Likewise.
(tst_towlower-ENV): Likewise.
(tst_towupper-ENV): Likewise.
(tst_wcrtomb-ENV): Likewise.
(tst_wcscat-ENV): Likewise.
(tst_wcschr-ENV): Likewise.
(tst_wcscmp-ENV): Likewise.
(tst_wcscoll-ENV): Likewise.
(tst_wcscpy-ENV): Likewise.
(tst_wcscspn-ENV): Likewise.
(tst_wcslen-ENV): Likewise.
(tst_wcsncat-ENV): Likewise.
(tst_wcsncmp-ENV): Likewise.
(tst_wcsncpy-ENV): Likewise.
(tst_wcspbrk-ENV): Likewise.
(tst_wcsrtombs-ENV): Likewise.
(tst_wcsspn-ENV): Likewise.
(tst_wcsstr-ENV): Likewise.
(tst_wcstod-ENV): Likewise.
(tst_wcstok-ENV): Likewise.
(tst_wcstombs-ENV): Likewise.
(tst_wcswidth-ENV): Likewise.
(tst_wcsxfrm-ENV): Likewise.
(tst_wctob-ENV): Likewise.
(tst_wctomb-ENV): Likewise.
(tst_wctrans-ENV): Likewise.
(tst_wctype-ENV): Likewise.
(tst_wcwidth-ENV): Likewise.
(tst-digits-ENV): Likewise.
(tst-mbswcs6-ENV): Likewise.
(tst-xlocale1-ENV): Likewise.
(tst-xlocale2-ENV): Likewise.
(tst-strfmon1-ENV): Likewise.
(tst-strptime-ENV): Likewise.
(tst-setlocale-ENV): Don't set LOCPATH.
(bug-iconv-trans-ENV): Remove variable.
(tst-sscanf-ENV): Likewise.
(tst-leaks-ENV): Don't set LOCPATH.
(bug-setlocale1-ENV): Remove variable.
(bug-setlocale1-static-ENV): Likewise.
(tst-setlocale2-ENV): Likewise.
[BZ #16885]
* sysdeps/sparc/sparc64/strcmp.S: Fix end comparison handling when
multiple zero bytes exist at the end of a string.
Reported by Aurelien Jarno <aurelien@aurel32.net>
* string/test-strcmp.c (check): Add explicit test for situations where
there are multiple zero bytes after the first.
Clean up string functions that do not have a version in gnulib on
the assumption that glibc is the canonical upstream copy of this
code. basename has a copy in gnulib but it is largely written to
handle Windows paths so merging it is not really viable. The changes
mostly consist of switching to ANSI function prototypes and removing
unused includes.
As many of these functions do not get built in a typical build due
to architecture optimized versions being used instead I built these
by hand to verify there were no build warnings and the code was
identical.
2014-04-07 Will Newton <will.newton@linaro.org>
* string/basename.c [HAVE_CONFIG_H]: Remove #ifdef and
and contents. [!_LIBC] Remove #ifndef and contents.
(basename): Use ANSI prototype. [_LIBC] Remove #idef.
* string/memccpy.c (__memccpy): Use ANSI prototype.
* string/memfrob.c (memfrob): Likewise.
* string/strcoll.c (STRCOLL): Likewise.
* string/strlen.c (strlen): Likewise.
* string/strtok.c (STRTOK): Likewise.
* string/strcat.c: Remove unused #include of memcopy.h.
(strcat): Use ANSI prototype.
* string/strchr.c: Remove unused #include of memcopy.h.
(strchr): Use ANSI prototype.
* string/strcmp.c: Remove unused #include of memcopy.h.
(strcmp): Use ANSI prototype.
* string/strcpy.c: Remove unused #include of memcopy.h.
(strcpy): Use ANSI prototype.
This patch add an optimized strpbrk for POWER7 by using a different
algorithm than default implementation: it constructs a table based on
the 'accept' argument and use this table to check for any occurance on
the input string. The idea is similar as x86_64 uses.
For PowerPC some tunings were added, such as unroll loops and memory
clear using VSX instructions.
This patch add a optimized strcspn for POWER7 by using a different
algorithm than default implementation: it constructs a table based on
the 'accept' argument and use this table to check for any occurance
on the input string. The idea is similar as x86_64 uses.
For PowerPC some tunings were added, such as unroll loops and align
stack memory to table to 16 bytes (so VSX clean can ran without
alignment issues).
The optimization is achieved by following techniques:
> hashing of needle.
> hashing avoids scanning of duplicate entries in needle across the string.
> initializing the hash table with Vector instructions (VSX) by quadword access.
> unrolling when scanning for character in string across hash table.
This patch is a revised and updated version of
<https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html>.
In order to generate overall summaries of the results of all tests in
the glibc testsuite, we need to identify and concatenate the files
with the results of individual tests.
Tomas Dohnalek's patch used $(common-objpfx)*/*.test-result for this.
However, the normal glibc approach is explicit enumeration of the
expected set of files with a given property, rather than all files
matching some pattern like that. Furthermore, we would like to be
able to mark tests as UNRESOLVED if the file with their results is for
some reason missing, and in future we would like to be able to mark
tests as UNSUPPORTED if they are disabled for a particular
configuration (rather than simply having them missing from the list of
tests as at present). Such handling of tests that were not run or did
not record results requires an explicit enumeration of tests.
For the tests following the default makefile rules, $(tests) (and
$(xtests)) provides such an enumeration. Others, however, are added
directly as dependencies of the "tests" and "xtests" makefile
targets. This patch changes the makefiles to put them in variables
tests-special and xtests-special, with appropriate dependencies on the
tests listed there then being added centrally.
Those variables are used in Rules and so need to be set before Rules
is included in a subdirectory makefile, which is often earlier in the
makefile than the dependencies were present before. We previously
discussed the question of where to include Rules; see the question at
<https://sourceware.org/ml/libc-alpha/2012-11/msg00798.html>, and a
discussion in
<https://sourceware.org/ml/libc-alpha/2013-01/msg00337.html> of why
Rules is included early rather than late in subdirectory makefiles.
It was necessary to avoid an indirection through the check-abi target
and get the check-abi-* targets for individual libraries into the
tests-special variable. The intl/ test $(objpfx)tst-gettext.out,
previously built only because of dependencies from other tests, was
also added to tests-special for the same reason.
The entries in tests-special are the full makefile targets, complete
with $(objpfx) and .out. If a future change causes tests to be named
consistently with a .out suffix, this can be changed to include just
the path relative to $(objpfx), without .out.
Tested x86_64, including that the same set of files is generated in
the build directory by a build and testsuite run both before and after
the patch (except for changes to the
elf/tst-null-argv.debug.out.<number> file name), and a build with
run-built-tests=no to verify there aren't any more obvious instances
of the issue Marcus Shawcroft reported with a previous version in
<https://sourceware.org/ml/libc-alpha/2014-01/msg00462.html>.
* Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(tests): Depend on $(tests-special).
* Makerules (check-abi-list): New variable.
(check-abi): Depend on $(check-abi-list).
[$(subdir) = elf] (tests-special): Add
$(objpfx)check-abi-libc.out.
[$(build-shared) = yes && subdir] (tests-special): Add
$(check-abi-list).
[$(build-shared) = yes && subdir] (tests): Do not depend on
check-abi.
* Rules (tests): Depend on $(tests-special).
(xtests): Depend on $(xtests-special).
* catgets/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* conform/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* elf/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* grp/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* iconv/Makefile (xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* iconvdata/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* intl/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable. Also add
$(objpfx)tst-gettext.out.
* io/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* libio/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* malloc/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* misc/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* nptl/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* nptl_db/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* posix/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* resolv/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(xtests): Change dependencies to ....
(xtests-special): ... additions to this variable.
* stdio-common/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
(do-tst-unbputc): Remove target.
(do-tst-printf): Likewise.
* stdlib/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* string/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
* sysdeps/x86/Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
localedata:
* Makefile (tests): Change dependencies to ....
(tests-special): ... additions to this variable.
This patch optimizes strrchr() for ppc64. It uses aligned memory
access along with cmpb instruction and CPU prefetch to avoid
cache misses for speed improvement.
In <https://sourceware.org/ml/libc-alpha/2014-01/msg00196.html> I
noted it was necessary to add includes of Makeconfig early in various
subdirectory makefiles for the tests-special variable settings added
by that patch to be conditional on configuration information. No-one
commented on the general question there of whether Makeconfig should
always be included immediately after the definition of subdir.
This patch implements that early inclusion of Makeconfig in each
directory (which is a lot easier than consistent placement of includes
of Rules). Includes are added if needed, or moved up if already
present. Subdirectory "all:" targets are removed, since Makeconfig
provides one.
There is potential for further cleanups I haven't done. Rules and
Makerules have code such as
ifneq "$(findstring env,$(origin headers))" ""
headers :=
endif
to override to empty any value of various variables that came from the
environment. I think there is a case for Makeconfig setting all the
subdirectory variables (other than subdir) to empty to ensure no
outside value is going to take effect if a subdirectory fails to
define a variable. (A list of such variables, possibly out of date
and incomplete, is in manual/maint.texi.) Rules and Makerules would
give errors if Makeconfig hadn't already been included, instead of
including it themselves. The special code to override values coming
from the environment would then be obsolete and could be removed.
Tested x86_64, including that installed binaries are identical before
and after the patch.
* argp/Makefile: Include Makeconfig immediately after defining
subdir.
* assert/Makefile: Likewise.
* benchtests/Makefile: Likewise.
* catgets/Makefile: Likewise.
* conform/Makefile: Likewise.
* crypt/Makefile: Likewise.
* csu/Makefile: Likewise.
(all): Remove target.
* ctype/Makefile: Include Makeconfig immediately after defining
subdir.
* debug/Makefile: Likewise.
* dirent/Makefile: Likewise.
* dlfcn/Makefile: Likewise.
* gmon/Makefile: Likewise.
* gnulib/Makefile: Likewise.
* grp/Makefile: Likewise.
* gshadow/Makefile: Likewise.
* hesiod/Makefile: Likewise.
* hurd/Makefile: Likewise.
(all): Remove target.
* iconvdata/Makefile: Include Makeconfig immediately after
defining subdir.
* inet/Makefile: Likewise.
* intl/Makefile: Likewise.
* io/Makefile: Likewise.
* libio/Makefile: Likewise.
(all): Remove target.
* locale/Makefile: Include Makeconfig immediately after defining
subdir.
* login/Makefile: Likewise.
* mach/Makefile: Likewise.
(all): Remove target.
* malloc/Makefile: Include Makeconfig immediately after defining
subdir.
(all): Remove target.
* manual/Makefile: Include Makeconfig immediately after defining
subdir.
* math/Makefile: Likewise.
* misc/Makefile: Likewise.
* nis/Makefile: Likewise.
* nss/Makefile: Likewise.
* po/Makefile: Likewise.
(all): Remove target.
* posix/Makefile: Include Makeconfig immediately after defining
subdir.
* pwd/Makefile: Likewise.
* resolv/Makefile: Likewise.
* resource/Makefile: Likewise.
* rt/Makefile: Likewise.
* setjmp/Makefile: Likewise.
* shadow/Makefile: Likewise.
* signal/Makefile: Likewise.
* socket/Makefile: Likewise.
* soft-fp/Makefile: Likewise.
* stdio-common/Makefile: Likewise.
* stdlib/Makefile: Likewise.
* streams/Makefile: Likewise.
* string/Makefile: Likewise.
* sunrpc/Makefile: Likewise.
(all): Remove target.
* sysvipc/Makefile: Include Makeconfig immediately after defining
subdir.
* termios/Makefile: Likewise.
* time/Makefile: Likewise.
* timezone/Makefile: Likewise.
(all): Remove target.
* wcsmbs/Makefile: Include Makeconfig immediately after defining
subdir.
* wctype/Makefile: Likewise.
libidn/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
localedata/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
(all): Remove target.
nptl/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
nptl_db/ChangeLog:
* Makefile: Include Makeconfig immediately after defining subdir.
This patch splits makefile rules that generate a file then run cmp to
check the contents of that file into separate rules to generate and
compare the file. This simplifies making those tests generate PASS /
FAIL results, by removing the need to insert && between commands in
the test so that a $(evaluate-test) call is reached. It also avoids
the oddity of the .out file being an intermediate file rather than the
final result generated, as noted for some of these tests in
<https://sourceware.org/ml/libc-alpha/2012-10/msg00894.html>.
In many cases, the rule to run the program was no longer needed
because the default rules for running test programs on the host to
generate a .out file sufficed. (I'm not asserting the commands run
after this patch are *exactly* the same as before, simply that the
rules did nothing special that appeared deliberate or relevant to
anything about what the tests were testing. In cases where the rules
redirected stderr as well as stdout, I left the existing rule's
redirection in place to avoid changing what gets compared with the
expected results.)
It's clear there is a lot in common between the various -cmp.out rules
and it might be possible in future to refactor them into more generic
support for the case of comparing test output against a baseline.
(Some baselines are *.exp, some *.expect, some directly embedded in
the makefiles, and nptl/tst-cleanupx0.expect appears unused.)
Tested x86_64.
* elf/Makefile ($(objpfx)order.out): Remove rule.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)order-cmp.out.
($(objpfx)order-cmp.out): New rule.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)tst-array1-cmp.out, $(objpfx)tst-array1-static-cmp.out,
$(objpfx)tst-array2-cmp.out, $(objpfx)tst-array3-cmp.out,
$(objpfx)tst-array4-cmp.out, $(objpfx)tst-array5-cmp.out and
$(objpfx)tst-array5-static-cmp.out.
($(objpfx)tst-array1.out): Remove rule.
($(objpfx)tst-array1-cmp.out): New rule.
($(objpfx)tst-array1-static.out): Remove rule.
($(objpfx)tst-array1-static-cmp.out): New rule.
($(objpfx)tst-array2.out): Remove rule.
($(objpfx)tst-array2-cmp.out): New rule.
($(objpfx)tst-array3.out): Remove rule.
($(objpfx)tst-array3-cmp.out): New rule.
($(objpfx)tst-array4.out): Remove rule.
($(objpfx)tst-array4-cmp.out): New rule.
($(objpfx)tst-array5.out): Remove rule.
($(objpfx)tst-array5-cmp.out): New rule.
($(objpfx)tst-array5-static.out): Remove rule.
($(objpfx)tst-array5-static-cmp.out): New rule.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)order2-cmp.out.
($(objpfx)order2.out): Remove rule.
($(objpfx)order2-cmp.out): New rule.
($(objpfx)tst-initorder.out): Remove rule.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)tst-initorder-cmp.out.
($(objpfx)tst-initorder-cmp.out): New rule.
($(objpfx)tst-initorder2.out): Remove rule.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)tst-initorder2-cmp.out.
($(objpfx)tst-initorder2-cmp.out): New rule.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)tst-unused-dep-cmp.out.
($(objpfx)tst-unused-dep-cmp.out): Do not run cmp.
($(objpfx)tst-unused-dep-cmp.out): New rule.
* stdio-common/Makefile [$(run-built-tests) = yes] (tests): Depend
on $(objpfx)tst-setvbuf1-cmp.out.
($(objpfx)tst-setvbuf1.out): Do not run cmp.
($(objpfx)tst-setvbuf1-cmp.out): New rule.
* string/Makefile [$(run-built-tests) = yes] (tests): Depend
$(objpfx)tst-svc-cmp.out instead of $(objpfx)tst-svc.out.
($(objpfx)tst-svc.out): Remove rule.
($(objpfx)tst-svc-cmp.out): New rule.
nptl:
* Makefile ($(objpfx)tst-cleanup0.out): Do not run cmp.
[$(run-built-tests) = yes] (tests): Depend on
$(objpfx)tst-cleanup0-cmp.out.
($(objpfx)tst-cleanup0-cmp.out): New rule.
This patch cleans up cases of __USE_MISC that are trivially redundant
after the recent substitution of __USE_MISC for __USE_BSD and
__USE_SVID: either in constructs such as "defined __USE_MISC ||
defined __USE_MISC", or else (in the bits/mman.h case) a conditional
on __USE_MISC nested inside another __USE_MISC conditional. (The
cleanups remaining after this patch are still quite large, but it
seems a reasonable piece to separate out.)
Tested x86_64.
* bits/mman.h [__USE_MISC]: Remove redundant conditionals.
* ctype/ctype.h [__USE_MISC]: Likewise.
* dirent/dirent.h [__USE_MISC]: Likewise.
* grp/grp.h [__USE_MISC]: Likewise.
* io/fcntl.h [__USE_MISC]: Likewise.
* io/sys/stat.h [__USE_MISC]: Likewise.
* libio/stdio.h [__USE_MISC]: Likewise.
* posix/unistd.h [__USE_MISC]: Likewise.
* pwd/pwd.h [__USE_MISC]: Likewise.
* stdlib.h [__USE_MISC]: Likewise.
* string/bits/string2.h [__USE_MISC]: Likewise.
* string/string.h [__USE_MISC]: Likewise.
* time/time.h [__USE_MISC]: Likewise.
In the string/string.h and string/strings.h headers, we have a couple
of macros that "tell the caller that we provide correct C++
prototypes" according to the comment; they are used to determine
whether to wrap some prototypes in "extern "C++"" (and provide
multiple overloads of them, and some other magic) when __cplusplus is
defined.
The macros are set to check for sufficiently-recent GCC versions (4.4
and later), but this is not the right check for non-GCC compilers. In
particular, these macros should also be set when using Clang -- if
they are not set, then Clang will be unable to correctly diagnose a
number of subtle bugs that will be errors in GCC compilations.
As per discussion on earlier versions of this patch, rather than
restrict the fix to Clang per se, we assume that all C++ compilers that
claim to fully support C++98 are using a standard-conforming C++
standard library, which seems pretty reasonable. Clang has been
providing an appropriate value of __cplusplus since May 2012.
http://sourceware.org/ml/libc-alpha/2013-08/msg00095.html
I found this useful at one stage when I was seeing a huge number of
memrchr failures all of test number 10.
* string/tester.c (test_memrchr): Increment reported test cycle.
http://sourceware.org/ml/libc-alpha/2013-08/msg00094.html
Using plain %s here runs the risk of segfaulting when displaying the
string. src and dst aren't zero terminated strings.
* string/test-memcpy.c (do_one_test): When reporting errors, print
string address and don't overrun end of string.
strcoll is implemented using a cache for indices and weights of
collation sequences in the strings so that subsequent passes do not
have to search through collation data again. For very large string
inputs, the cache size computation could overflow. In such a case,
use the fallback function that does not cache indices and weights of
collation sequences.
Fixes CVE-2012-4412.
strcoll currently falls back to alloca if malloc fails, resulting in a
possible stack overflow. This patch implements sequence traversal and
comparison without caching indices and rules.
Fixes CVE-2012-4424.
Check wheter the compiler has the option -fno-tree-loop-distribute-patterns
to inhibit loop transformation to library calls and uses it on memset
and memmove default implementation to avoid recursive calls.
Leonhard Holz
Stefan Liebler
Andreas Schwab
Wilco Dijkstra
Ondřej Bílka
Joseph Myers
Richard Earnshaw
Torvald Riegel
Siddhesh Poyarekar
Arjun Shankar
Roland McGrath
Adhemerval Zanella
Will Newton
David S. Miller
Vidya Ranganathan
Rajalakshmi Srinivasaraghavan
Allan McRae
Brooks Moses
Samuel Thibault
Alan Modra
Andreas Schwab
H.J. Lu
Ryan S. Arnold
Andreas Jaeger
Tom de Vries
2012-12-27 Bruno Haible
Maxim Kuvyrkov