this is a path that should solve bug 15884. It complains about the performance
of strcoll(). It was found out that the runtime of strcoll() is actually bound
to strlen which is needed for calculating the size of a cache that was
installed to improve the comparison performance.
The idea for this patch was that the cache is only useful in rare cases
(strings of same length and same first-level-chars) and that it would be
better to avoid memory allocation at all. To prove this I wrote a performance
test bench-strcoll.c with test data in benchtests-strcoll.tar.gz. Also
modifications in benchtests/Makefile and localedata/Makefile are necessary to
make it work.
After removing the cache the strcoll method showed the predicted behavior
(getting slightly faster) in all but the test case for hindi word sorting.
This was due the hindi text having much more equal words than the other ones.
For equal strings the performance was worse since all comparison levels were
run through and from the second level on the cache improved the comparison
performance of the original version.
Therefore I added a bytewise test via strcmp iff the first level comparison
found that both strings did match because in this case it is very likely that
equal strings are compared. This solved the problem with the hindi test case
and improved the performance of the others.
Performance comparison:
glibc files -33.77%
vi_VN.UTF-8 -34.12%
en_US.UTF-8 -42.42%
ar_SA.UTF-8 -27.49%
zh_CN.UTF-8 +07.90%
cs_CZ.UTF-8 -29.67%
en_GB.UTF-8 -28.50%
da_DK.UTF-8 -36.57%
pl_PL.UTF-8 -39.31%
fr_FR.UTF-8 -28.57%
pt_PT.UTF-8 -22.82%
el_GR.UTF-8 -26.77%
ru_RU.UTF-8 -35.81%
iw_IL.UTF-8 -35.34%
es_ES.UTF-8 -34.46%
hi_IN.UTF-8 -00.38%
sv_SE.UTF-8 -36.99%
hu_HU.UTF-8 -16.35%
tr_TR.UTF-8 -27.80%
is_IS.UTF-8 -33.24%
it_IT.UTF-8 -24.39%
sr_RS.UTF-8 -37.55%
ja_JP.UTF-8 +02.84%
strcoll is implemented using a cache for indices and weights of
collation sequences in the strings so that subsequent passes do not
have to search through collation data again. For very large string
inputs, the cache size computation could overflow. In such a case,
use the fallback function that does not cache indices and weights of
collation sequences.
Fixes CVE-2012-4412.
strcoll currently falls back to alloca if malloc fails, resulting in a
possible stack overflow. This patch implements sequence traversal and
comparison without caching indices and rules.
Fixes CVE-2012-4424.
2004-03-14 Ulrich Drepper <drepper@redhat.com>
Make the non-_l functions wrappers around the _l functions.
* include/monetary.h: Declare __vstrmon_l.
* include/string.h: Add libc_hidden_proto for __strcoll_l and
__strxfrm_l.
* include/time.h: Define ptime_locale_status. Declare
__strptime_internal.
* include/wchar.h: Add libc_hidden_proto for __wcscoll_l and
__wcsxfrm_l.
* stdlib/strfmon.c: Move the code to strfmon_l.c. Add little
wrapper around __vstrfmon_l.
* stdlib/strfmon_l.c: Add real implementation. Split into new
function __vstrfmon_l to allow calling it from strfmon.
* stdlib/strtod.c: Move real code to strtod_l.c and add wrapper.
* stdlib/strtod_l.c: Add real implementation.
* stdlib/strtof.c: Adjust to changed strtod.c.
* stdlib/strtof_l.c: Include strtod_l.c now.
* stdlib/strtold.c: New file.
* stdlib/strtold_l.c: Removed.
* string/strcoll.c: Move real code to strcoll_l.c: Add wrapper.
* string/strcoll_l.c: Add real implementation.
* string/strxfrm.c: Move real code to strxfrm_l.c: Add wrapper.
* string/strxfrm_l.c: Add real implementation.
* sysdeps/generic/strtol.c: Move real implementation to strtol_l.c.
Add wrappers.
* sysdeps/generic/strtol_l.c: Add real implementation.
* sysdeps/generic/strtold.c: Removed.
* sysdeps/generic/strtold_l.c: New file.
* sysdeps/generic/strtoll_l.c: Include strtol_l.c now. Adjust
#defines.
* sysdeps/generic/strtoul_l.c: Likewise.
* sysdeps/generic/strtoull_l.c: Likewise.
* sysdeps/generic/wcstol_l.c: Likewise.
* sysdeps/generic/wcstoll_l.c: Likewise.
* sysdeps/generic/wcstoul_l.c: Likewise.
* sysdeps/generic/wcstoull_l.c: Likewise.
* sysdeps/ieee754/ldbl-128/strtold.c: Removed.
* sysdeps/ieee754/ldbl-128/strtold_l.c: New file.
* sysdeps/ieee754/ldbl-96/strtold.c: Removed.
* sysdeps/ieee754/ldbl-96/strtold_l.c: New file.
* sysdeps/m68k/strtold.c: Removed.
* sysdeps/m68k/strtold_l.c: New file.
* time/strftime.c: Move real code to strftime_l.c. Add wrapper.
* time/strftime_l.c: Add real implementation.
* time/strptime.c: Move real code to strptime_l.c. Add wrapper.
* time/strptime_l.c: Add real implementation.
* time/wcsftime.c: Simplify since only wrappers are defined in
strftime.c.
* time/wcsftime_l.c: Include strftime_l.c.
* wcsmbs/wcscoll.c: Simplify since the file is not used by wcscoll_l.c
anymore.
* wcsmbs/wcscoll_l.c: Include strcoll_l.c.
* wcsmbs/wcsxfrm.c: Simplify since the file is not used by wcsxfrm_l.c
anymore.
* wcsmbs/wcsxfrm_l.c: Include strxfrm_l.c.
* wcsmbs/wcstod.c: Prepare to include new strtod.c.
* wcsmbs/wcstod_l.c: Include strtod_l.c.
* wcsmbs/wcstof.c: Prepare to include new strtof.c.
* wcsmbs/wcstof_l.c: Include strtof_l.c.
* wcsmbs/wcstold.c: Prepare to include new strtold.c.
* wcsmbs/wcstold_l.c: Include strtold_l.c.
* locale/uselocale.c: Use _NL_CURRENT_LOCALE instead of __libc_tsd_get.
* sysdeps/generic/strcasecmp.c: Optimize a bit. It's better to get
a reference to the current locale and then use the _l functions.
* sysdeps/generic/strncase.c: Likewise.
2001-07-06 Paul Eggert <eggert@twinsun.com>
* manual/argp.texi: Remove ignored LGPL copyright notice; it's
not appropriate for documentation anyway.
* manual/libc-texinfo.sh: "Library General Public License" ->
"Lesser General Public License".
2001-07-06 Andreas Jaeger <aj@suse.de>
* All files under GPL/LGPL version 2: Place under LGPL version
2.1.
Joseph Myers
Leonhard Holz
Siddhesh Poyarekar
Roland McGrath
Allan McRae
Samuel Thibault
Paul Eggert
Andreas Schwab
Ulrich Drepper
Roland McGrath
Andreas Jaeger