Change the tcache->counts[] entries to uint16_t - this removes
the limit set by char and allows a larger tcache. Remove a few
redundant asserts.
bench-malloc-thread with 4 threads is ~15% faster on Cortex-A72.
Reviewed-by: DJ Delorie <dj@redhat.com>
* malloc/malloc.c (MAX_TCACHE_COUNT): Increase to UINT16_MAX.
(tcache_put): Remove redundant assert.
(tcache_get): Remove redundant asserts.
(__libc_malloc): Check tcache count is not zero.
* manual/tunables.texi (glibc.malloc.tcache_count): Update maximum.
(cherry picked from commit 1f50f2ad854c84ead522bfc7331b46dbe6057d53)
The kernel is evolving this interface (e.g., removal of the
restriction on cross-device copies), and keeping up with that
is difficult. Applications which need the function should
run kernels which support the system call instead of relying on
the imperfect glibc emulation.
Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
(cherry picked from commit 5a659ccc0ec217ab02a4c273a1f6d346a359560a)
The tcache counts[] array is a char, which has a very small range and thus
may overflow. When setting tcache_count tunable, there is no overflow check.
However the tunable must not be larger than the maximum value of the tcache
counts[] array, otherwise it can overflow when filling the tcache.
[BZ #24531]
* malloc/malloc.c (MAX_TCACHE_COUNT): New define.
(do_set_tcache_count): Only update if count is small enough.
* manual/tunables.texi (glibc.malloc.tcache_count): Document max value.
(cherry picked from commit 5ad533e8e65092be962e414e0417112c65d154fb)
* NEWS: Add the list of bugs fixed in 2.29.
* manual/contrib.texi: Update contributors list with some more
names.
* manual/install.texi: Update latest versions of packages
tested.
* INSTALL: Regenerated.
The full representation of the alternative calendar year (%EY)
typically includes an internal use of "%Ey". As a GNU extension,
apply any flags on "%EY" (e.g. "%_EY", "%-EY") to the internal "%Ey",
allowing users of "%EY" to control how the year is padded.
Reviewed-by: Rafal Luzynski <digitalfreak@lingonborough.com>
Reviewed-by: Zack Weinberg <zackw@panix.com>
ChangeLog:
[BZ #24096]
* manual/time.texi (strftime): Document "%EC" and "%EY".
* time/Makefile (tests): Add tst-strftime2.
(LOCALES): Add ja_JP.UTF-8, lo_LA.UTF-8, and th_TH.UTF-8.
* time/strftime_l.c (__strftime_internal): Add argument yr_spec to
override padding for "%Ey".
If an optional flag ('_' or '-') is specified to "%EY", interpret the
"%Ey" in the subformat as if decorated with that flag.
* time/tst-strftime2.c: New file.
In Japanese locales, strftime's alternative year format (%Ey) produces
a year numbered within a time period called an _era_. A new era
typically begins when a new emperor is enthroned. The result of "%Ey"
is therefore usually a one- or two-digit number.
Many programs that display Japanese era dates assume that the era year
is two digits wide. To improve how these programs display dates
during the first nine years of a new era, change "%Ey" to pad one-
digit numbers on the left with a zero. This change applies to all
locales. It is expected to be harmless for other locales that use the
alternative year format (e.g. lo_LA and th_TH, in which "%Ey" produces
the year of the Buddhist calendar) as those calendars' year numbers
are already more than two digits wide, and this is not expected to
change.
This change needs to be in place before 2019-05-01 CE, as a new era is
scheduled to begin on that date.
Reviewed-by: Zack Weinberg <zackw@panix.com>
Reviewed-by: Rafal Luzynski <digitalfreak@lingonborough.com>
ChangeLog:
[BZ #23758]
* manual/time.texi (strftime): Document "%Ey".
* time/strftime_l.c (__strftime_internal): Set the default width
padding with zero of "%Ey" to 2.
Add Ares to the midr_el0 list and support ifunc dispatch. Since Ares
supports 2 128-bit loads/stores, use Neon registers for memcpy by
selecting __memcpy_falkor by default (we should rename this to
__memcpy_simd or similar).
* manual/tunables.texi (glibc.cpu.name): Add ares tunable.
* sysdeps/aarch64/multiarch/memcpy.c (__libc_memcpy): Use
__memcpy_falkor for ares.
* sysdeps/unix/sysv/linux/aarch64/cpu-features.h (IS_ARES):
Add new define.
* sysdeps/unix/sysv/linux/aarch64/cpu-features.c (cpu_list):
Add ares cpu.
@var is intended for placeholders (such as function parameters).
Actual variables need to use @code because @var causes upper-case
output, resulting in a different C identifier.
This patch updates some miscellaneous files from their upstream
sources (thereby bringing in copyright date updates for some of those
files).
Tested for x86_64, including "make pdf".
* manual/texinfo.tex: Update to version 2018-12-28.17 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2019-01-01.
* scripts/config.sub: Update to version 2019-01-01.
* scripts/move-if-change: Update from gnulib.
We know that building glibc with GCC 4.9 is broken on various
platforms (bug 23993). As it's more than a year since we last
increased the minimum GCC version to build glibc, this patch changes
the requirement to be GCC 5 or later (indeed, based on 4.9 having been
required for building 2.26, it would be consistent in terms of timing
to require GCC 6 or later from the 2.30 release onwards). It
deliberately just updates the configure test and corresponding
documentation, leaving removal of no-longer-needed __GNUC_PREREQ tests
for a separate patch.
In the NEWS entry, the requirement for a newer GCC version for
powerpc64le is reiterated (as in the entry for the 4.9 requirement in
2.26) to avoid suggesting the version requirement there has gone down.
(If that version goes up further as part of support for binary128 long
double, of course the wording would change at that time.)
Tested for x86_64.
[BZ #23993]
* configure.ac (libc_cv_compiler_ok): Require GCC 5 or later.
* configure: Regenerated.
* manual/install.texi (Tools for Compilation): Update minimum GCC
version.
* INSTALL: Regenerated.
Tested with 'make check' on x86_64-linux-gnu and i686-linux.gnu.
* include/time.h
(__localtime64): Add.
* manual/maint.texi: Document Y2038 symbol handling.
* time/localtime.c
(__localtime64): Add.
[__TIMERSIZE != 64] (__localtime): Turn into a wrapper.
This patch eliminates the gen-py-const.awk variant of gen-as-const,
switching to use of gnu-as-const.py (with a new --python option) to
process .pysym files (i.e., to generate nptl_lock_constants.py), as
the syntax of those files is identical to that of .sym files.
Note that the generated nptl_lock_constants.py is *not* identical to
the version generated by the awk script. Apart from the trivial
changes (comment referencing the new script, and output being sorted),
the constant FUTEX_WAITERS, PTHREAD_MUTEXATTR_FLAG_BITS,
PTHREAD_MUTEXATTR_FLAG_PSHARED and PTHREAD_MUTEX_PRIO_CEILING_MASK are
now output as positive rather than negative constants (on x86_64
anyway; maybe not necessarily on 32-bit systems):
< FUTEX_WAITERS = -2147483648
---
> FUTEX_WAITERS = 2147483648
< PTHREAD_MUTEXATTR_FLAG_BITS = -251662336
< PTHREAD_MUTEXATTR_FLAG_PSHARED = -2147483648
---
> PTHREAD_MUTEXATTR_FLAG_BITS = 4043304960
> PTHREAD_MUTEXATTR_FLAG_PSHARED = 2147483648
< PTHREAD_MUTEX_PRIO_CEILING_MASK = -524288
---
> PTHREAD_MUTEX_PRIO_CEILING_MASK = 4294443008
This is because gen-as-const has a cast of the constant value to long
int, which gen-py-const lacks.
I think the positive values are more logically correct, since the
constants in question are in fact unsigned in C. But to reliably
produce gen-as-const.py output for constants that always (in C and
Python) reflects the signedness of values with the high bit of "long
int" set would mean more complicated logic needs to be used in
computing values.
The more correct positive values by themselves produce a failure of
nptl/test-mutexattr-printers, because masking with
~PTHREAD_MUTEXATTR_FLAG_BITS & ~PTHREAD_MUTEX_NO_ELISION_NP now leaves
a bit -1 << 32 in the Python value, resulting in a KeyError exception.
To avoid that, places masking with ~ of one of the constants in
question are changed to mask with 0xffffffff as well (this reflects
how ~ in Python applies to an infinite-precision integer whereas ~ in
C does not do any promotions beyond the width of int).
Tested for x86_64.
* scripts/gen-as-const.py (main): Handle --python option.
* scripts/gen-py-const.awk: Remove.
* Makerules (py-const-script): Use gen-as-const.py.
($(py-const)): Likewise.
* nptl/nptl-printers.py (MutexPrinter.read_status_no_robust): Mask
with 0xffffffff together with ~(PTHREAD_MUTEX_PRIO_CEILING_MASK).
(MutexAttributesPrinter.read_values): Mask with 0xffffffff
together with ~PTHREAD_MUTEXATTR_FLAG_BITS and
~PTHREAD_MUTEX_NO_ELISION_NP.
* manual/README.pretty-printers: Update reference to
gen-py-const.awk.
This patch updates various miscellaneous files from their upstream
sources.
Tested for x86_64, including "make pdf".
* manual/texinfo.tex: Update to version 2018-09-21.20 with
trailing whitespace removed.
* scripts/config.guess: Update to version 2018-11-28.
* scripts/config.sub: Update to version 2018-11-28.
* scripts/install-sh: Update to version 2018-03-11.20.
* scripts/mkinstalldirs: Update to version 2018-03-07.03.
* scripts/move-if-change: Update to version 2018-03-07 03:47.
This patch does not have any functionality change, we only provide a spin
count tunes for pthread adaptive spin mutex. The tunable
glibc.pthread.mutex_spin_count tunes can be used by system administrator to
squeeze system performance according to different hardware capabilities and
workload characteristics.
The maximum value of spin count is limited to 32767 to avoid the overflow
of mutex->__data.__spins variable with the possible type of short in
pthread_mutex_lock ().
The default value of spin count is set to 100 with the reference to the
previous number of times of spinning via trylock. This value would be
architecture-specific and can be tuned with kinds of benchmarks to fit most
cases in future.
I would extend my appreciation sincerely to H.J.Lu for his help to refine
this patch series.
* manual/tunables.texi (POSIX Thread Tunables): New node.
* nptl/Makefile (libpthread-routines): Add pthread_mutex_conf.
* nptl/nptl-init.c: Include pthread_mutex_conf.h
(__pthread_initialize_minimal_internal) [HAVE_TUNABLES]: Call
__pthread_tunables_init.
* nptl/pthreadP.h (MAX_ADAPTIVE_COUNT): Remove.
(max_adaptive_count): Define.
* nptl/pthread_mutex_conf.c: New file.
* nptl/pthread_mutex_conf.h: New file.
* sysdeps/generic/adaptive_spin_count.h: New file.
* sysdeps/nptl/dl-tunables.list: New file.
* nptl/pthread_mutex_lock.c (__pthread_mutex_lock): Use
max_adaptive_count () not MAX_ADAPTIVE_COUNT.
* nptl/pthread_mutex_timedlock.c (__pthrad_mutex_timedlock):
Likewise.
Suggested-by: Andi Kleen <andi.kleen@intel.com>
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Signed-off-by: Kemi.wang <kemi.wang@intel.com>
On platforms where long double used to have the same format as double,
but later switched to a different format (alpha, s390, sparc, and
powerpc), accessing the older behavior is possible and it happens via
__nldbl_* functions (not on the API, but accessible from header
redirection and from compat symbols). These functions write to the
global flag __ldbl_is_dbl, which tells other functions that long double
variables should be handled as double. This patch takes the first step
towards removing this global flag and creates __vstrfmon_l_internal,
which takes an explicit flags parameter.
This change arguably makes the generated code slightly worse on
architectures where __ldbl_is_dbl is never true; right now, on those
architectures, it's a compile-time constant; after this change, the
compiler could theoretically prove that __vstrfmon_l_internal was
never called with a nonzero flags argument, but it would probably need
LTO to do it. This is not performance critical code and I tend to
think that the maintainability benefits of removing action at a
distance are worth it. However, we _could_ wrap the runtime flag
check with a macro that was defined to ignore its argument and always
return false on architectures where __ldbl_is_dbl is never true, if
people think the codegen benefits are important.
Tested for powerpc and powerpc64le.
In the description of gcvt, add missing closing parenthesis in the
comparison with sprintf.
* manual/arith.texi (gcvt): Add missing closing parenthesis.
This patch makes Python 3.4 or later a required tool for building
glibc, so allowing changes of awk, perl etc. code used in the build
and test to Python code without any such changes needing makefile
conditionals or to handle older Python versions.
This patch makes the configure test for Python check the version and
give an error if Python is missing or too old, and removes makefile
conditionals that are no longer needed. It does not itself convert
any code from another language to Python, and does not remove any
compatibility with older Python versions from existing scripts.
Tested for x86_64.
* configure.ac (PYTHON_PROG): Use AC_CHECK_PROG_VER. Set
critic_missing for versions before 3.4.
* configure: Regenerated.
* manual/install.texi (Tools for Compilation): Document
requirement for Python to build glibc.
* INSTALL: Regenerated.
* Rules [PYTHON]: Make code unconditional.
* benchtests/Makefile [PYTHON]: Likewise.
* conform/Makefile [PYTHON]: Likewise.
* manual/Makefile [PYTHON]: Likewise.
* math/Makefile [PYTHON]: Likewise.
This patch extends gen-libm-test.py to generate the ulps table for the
manual, so meaning there is only a single ulps file parser needed and
another Perl script is eliminated. As with the introduction of
gen-libm-test.py, this is designed to generate exactly the same
libm-err.texi as libm-err-tab.pl did. (gen-libm-test.py is still
shorter in lines than the old gen-libm-test.pl even after this patch.)
Note that this introduces a Python dependency for building the manual,
which is thus noted in install.texi and NEWS.
Tested building html / info / pdf versions of the manual.
* math/gen-libm-test.py: Import os.
(ALL_FLOATS_MANUAL): New constant.
(ALL_FLOATS_SUFFIX): Likewise.
(Ulps.all_functions): New function.
(real_all_ulps): Likewise.
(generate_err_table_sub): Likewise.
(generate_err_table): Likewise.
(main): Handle -s and -m options.
* manual/libm-err-tab.pl: Remove.
* manual/Makefile ($(objpfx)stamp-libm-err): Use gen-libm-test.py
instead of libm-err-tab.pl.
[$(PERL) != no]: Change condition to [$(if $(PYTHON),$(PERL),no)
!= no].
* manual/install.texi (Tools for Compilation): Document
requirement for Python to build manual.
* INSTALL: Regenerated.
Job control was made mandatory in POSIX.1-2001: compare
<http://pubs.opengroup.org/onlinepubs/7990989775/xsh/unistd.h.html> with
<http://pubs.opengroup.org/onlinepubs/009695399/basedefs/unistd.h.html>.
Seventeen years later, we need not devote an entire manual @node to
warning people that this was once an optional POSIX feature.
* manual/job.texi (Job Control is Optional): Remove node, as
job control has not been optional in quite some time.
(Job Control): Mention briefly that systems older than
POSIX.1-2001 might not support job control.
* manual/conf.texi (_POSIX_JOB_CONTROL): Will always be
defined on systems conforming to POSIX.1-2001.
For architectures and ABIs that are added in version 2.29 or later the
option --enable-obsolete-nsl is no longer available, and no libnsl
compatibility library is built.
The install.texi documentation of uses of Perl and Python is
substantially out of date.
The description of Perl is "to test the installation" (which I
interpret as referring to test-installation.pl), but it's used for
more tests than that, and to build the manual, and to regenerate one
file in the source tree.
The description of Python is only for pretty-printer tests, but it's
used for other tests / benchmarks as well (and for other internal uses
such as updating Unicode data, for which we already require Python 3,
but I think install.texi only needs to describe uses from the main
glibc Makefiles).
This patch updates the descriptions of what those tools are used for.
The Python information (and information about other tools for testing
pretty printers) was awkwardly in the middle of the general
description of building and testing glibc, rather than with the rest
of information about tools used in glibc build and test; this patch
moves the information about those tools into the main list.
Tested with regeneration of INSTALL as well as "make info" and "make
pdf".
* manual/install.texi (Configuring and compiling): Do not list
tools used for testing pretty printers here.
(Tools for Compilation): List Python, PExpect and GDB here.
Update descriptions of uses of Perl and Python.
* INSTALL: Regenerate.
The convenience install target 'install-locale-files' is created
to allow distributions to install all of the SUPPORTED locales as
files instead of into the locale-archive.
You invoke the new convenience target like this:
make localedata/install-locale-files DESTDIR=<prefix>
The glibc.tune namespace is vaguely named since it is a 'tunable', so
give it a more specific name that describes what it refers to. Rename
the tunable namespace to 'cpu' to more accurately reflect what it
encompasses. Also rename glibc.tune.cpu to glibc.cpu.name since
glibc.cpu.cpu is weird.
* NEWS: Mention the change.
* elf/dl-tunables.list: Rename tune namespace to cpu.
* sysdeps/powerpc/dl-tunables.list: Likewise.
* sysdeps/x86/dl-tunables.list: Likewise.
* sysdeps/aarch64/dl-tunables.list: Rename tune.cpu to
cpu.name.
* elf/dl-hwcaps.c (_dl_important_hwcaps): Adjust.
* elf/dl-hwcaps.h (GET_HWCAP_MASK): Likewise.
* manual/README.tunables: Likewise.
* manual/tunables.texi: Likewise.
* sysdeps/powerpc/cpu-features.c: Likewise.
* sysdeps/unix/sysv/linux/aarch64/cpu-features.c
(init_cpu_features): Likewise.
* sysdeps/x86/cpu-features.c: Likewise.
* sysdeps/x86/cpu-features.h: Likewise.
* sysdeps/x86/cpu-tunables.c: Likewise.
* sysdeps/x86_64/Makefile: Likewise.
* sysdeps/x86/dl-cet.c: Likewise.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
This patch updates the manual and adds a new chapter to the manual,
explaining types macros, constants and functions defined by ISO C11
threads.h standard.
[BZ# 14092]
* manual/debug.texi: Update adjacent chapter name.
* manual/probes.texi: Likewise.
* manual/threads.texi (ISO C Threads): New section.
(POSIX Threads): Convert to a section.
The implementation falls back to renameat if renameat2 is not available
in the kernel (or in the kernel headers) and the flags argument is zero.
Without kernel support, a non-zero argument returns EINVAL, not ENOSYS.
This mirrors what the kernel does for invalid renameat2 flags.
$(file …) appears to be the only convenient way to create files
with newlines and make substitution variables. This needs make 4.0
(released in 2013), so update the requirement to match.
Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Some Linux distributions are experimenting with a new, separately
maintained and hopefully more agile implementation of the crypt
API. To facilitate this, add a configure option which disables
glibc's embedded libcrypt. When this option is given, libcrypt.*
and crypt.h will not be built nor installed.
This is a major rewrite of the description of 'crypt', 'getentropy',
and 'getrandom'.
A few highlights of the content changes:
- Throughout the manual, public headers, and user-visible messages,
I replaced the term "password" with "passphrase", the term
"password database" with "user database", and the term
"encrypt(ion)" with "(one-way) hashing" whenever it was applied to
passphrases. I didn't bother making this change in internal code
or tests. The use of the term "password" in ruserpass.c survives,
because that refers to a keyword in netrc files, but it is adjusted
to make this clearer.
There is a note in crypt.texi explaining that they were
traditionally called passwords but single words are not good enough
anymore, and a note in users.texi explaining that actual passphrase
hashes are found in a "shadow" database nowadays.
- There is a new short introduction to the "Cryptographic Functions"
section, explaining how we do not intend to be a general-purpose
cryptography library, and cautioning that there _are_, or have
been, legal restrictions on the use of cryptography in many
countries, without getting into any kind of detail that we can't
promise to keep up to date.
- I added more detail about what a "one-way function" is, and why
they are used to obscure passphrases for storage. I removed the
paragraph saying that systems not connected to a network need no
user authentication, because that's a pretty rare situation
nowadays. (It still says "sometimes it is necessary" to
authenticate the user, though.)
- I added documentation for all of the hash functions that glibc
actually supports, but not for the additional hash functions
supported by libxcrypt. If we're going to keep this manual section
around after the transition is more advanced, it would probably
make sense to add them then.
- There is much more detailed discussion of how to generate a salt,
and the failure behavior for crypt is documented. (Returning an
invalid hash on failure is what libxcrypt does; Solar Designer's
notes say that this was done "for compatibility with old programs
that assume crypt can never fail".)
- As far as I can tell, the header 'crypt.h' is entirely a GNU
invention, and never existed on any other Unix lineage. The
function 'crypt', however, was in Issue 1 of the SVID and is now
in the XSI component of POSIX. I tried to make all of the
@standards annotations consistent with this, but I'm not sure I got
them perfectly right.
- The genpass.c example has been improved to use getentropy instead
of the current time to generate the salt, and to use a SHA-256 hash
instead of MD5. It uses more random bytes than is strictly
necessary because I didn't want to complicate the code with proper
base64 encoding.
- The testpass.c example has three hardwired hashes now, to
demonstrate that different one-way functions produce different
hashes for the same input. It also demonstrates how DES hashing
only pays attention to the first eight characters of the input.
- There is new text explaining in more detail how a CSPRNG differs
from a regular random number generator, and how
getentropy/getrandom are not exactly a CSPRNG. I tried not to make
specific falsifiable claims here. I also tried to make the
blocking/cancellation/error behavior of both getentropy and
getrandom clearer.
In preparation for a major revision of the documentation for
crypt(_r), getentropy, and getrandom, reorganize crypt.texi. This
patch does not change any text; it only deletes and moves text.
The description of 'getpass' moves to terminal.texi, since all it does
is read a password from the controlling terminal with echo disabled.
The "Legal Problems" section of crypt.texi is dropped, and the
introductory text is shifted down to the "Encrypting Passwords"
section; the next patch will add some new introductory text.
Also, it is no longer true that crypt.texi's top @node needs to have
no pointers. That was a vestige of crypt/ being an add-on. (makeinfo
itself doesn't need @node pointers anymore, but the scripts that
assemble the libc manual's topmost node rely on each chapter-level
node having them.)
The functions encrypt, setkey, encrypt_r, setkey_r, cbc_crypt,
ecb_crypt, and des_setparity should not be used in new programs,
because they use the DES block cipher, which is unacceptably weak by
modern standards. Demote all of them to compatibility symbols, and
remove their prototypes from installed headers. cbc_crypt, ecb_crypt,
and des_setparity were already compat symbols when glibc was
configured with --disable-obsolete-rpc.
POSIX requires encrypt and setkey to be available when _XOPEN_CRYPT
is defined, so this change also removes the definition of X_OPEN_CRYPT
from <unistd.h>.
The entire "DES Encryption" section is dropped from the manual, as is
the mention of AUTH_DES and FIPS 140-2 in the introduction to
crypt.texi. The documentation of 'memfrob' cross-referenced the DES
Encryption section, which is replaced by a hyperlink to libgcrypt, and
while I was in there I spruced up the actual documentation of
'memfrob' and 'strfry' a little. It's still fairly jokey, because
those functions _are_ jokes, but they do also have real use cases, so
people trying to use them for real should have all the information
they need.
DES-based authentication for Sun RPC is also insecure and should be
deprecated or even removed, but maybe that can be left as TI-RPC's
problem.
DJ Delorie
Wilco Dijkstra
Florian Weimer
Siddhesh Poyarekar
TAMUKI Shoichi
Wilco Dijkstra
Szabolcs Nagy
Joseph Myers
H.J. Lu
Albert ARIBAUD (3ADEV)
Kemi Wang
DJ Delorie
Zack Weinberg
Gabriel F. T. Gomes
Samuel Thibault
Andreas Schwab
Carlos O'Donell
Rical Jasan
Martin Sebor