Commit Graph

251 Commits

Author SHA1 Message Date
Joseph Myers 4d602bcea8 Add build-many-glibcs.py bot-cycle action.
This patch continues the process of setting up build-many-glibcs.py to
run as a bot monitoring for and reporting on build issues by adding a
bot-cycle action to the script.  When this action is used, it will run
the checkout action (re-execing itself if it was changed by that
action), then rebuild whichever of host-libraries, compilers, glibcs
should be rebuilt based on changed versions, time elapsed and state of
previous builds.  Email is sent with the results of the build (for
each build action done).

The rebuild logic is: if previous build time or versions aren't
recorded, rebuild that component.  If the script has changed, rebuild
everything.  If any relevant component version has changed, rebuild,
except for not rebuilding compilers if the time indicated in the bot
configuration has not passed since the last build of the compilers.
If one piece is rebuilt then rebuild subsequent pieces as well.

Using bot-cycle requires a configuration file bot-config.json in the
toplevel directory used by build-many-glibcs.py.  It might contain
e.g.

{
  "compilers-rebuild-delay": 604800,
  "email-from": "Example Name <user@example.org>",
  "email-server": "localhost",
  "email-subject": "GCC 6 %(action)s %(build-time)s build results",
  "email-to": "libc-testresults@sourceware.org"
}

My next intended step is adding a further action "bot" which loops
running bot-cycle then sleeping for an amount of time given in
bot-config.json.  Then I'll set up a bot using that action (building
with GCC 6 branch; a bot using GCC mainline may wait until the SH
out-of-memory issues
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78460> are fixed; I
expect the bot to mail to me until it seems ready to switch to mailing
to gcc-testresults).

	* scripts/build-many-glibcs.py: Add bot-cycle to usage message.
	Import email.mime.text, email.utils and smtplib modules.
	(Context.__init__): Initialize self.bot_config_json.
	(Context.run_builds): Handle bot-cycle action.
	(Context.load_bot_config_json): New function.
	(Context.part_build_old): Likewise.
	(Context.bot_cycle): Likewise.
	(Context.bot_build_mail): Likewise.
	(Context.bot_run_self): Likewise.
	(get_parser): Allow bot-cycle action.
2016-11-30 18:56:37 +00:00
Joseph Myers bf469f0ce9 Make build-many-glibcs.py store more information about builds.
This patch makes build-many-glibcs.py store information about builds
in JSON format.  This is part of preparing it for use in a bot
checking for regressions.

The information stored is: time of last build (of host-libraries,
compilers or glibcs); versions of components used in the last build
(for compilers, host library versions are properly copied from those
used for the previous host-libraries build, and for glibcs, component
versions other than that of glibc are similarly copied from the last
compilers build); PASS/FAIL/UNRESOLVED results of the individual build
steps; a list of changed results; a list of tests (that are still run
at all) that have ever been recorded to PASS.

The first pieces of information are intended to be used by a bot to
decide whether a rebuild is appropriate (based on some combination of
elapsed time and changes to versions; a bot might want to rebuild
glibcs if there had been any change but only rebuild compilers after
enough time had elapsed, for example).  All the information is
intended to be used in generating mails with results information.

This state is specifically for full builds (no individual configs for
building compilers or glibcs specified).  If individual configs are
specified, build-time and build-versions information is cleared (since
it will no longer accurately reflect the install directory contents),
while the other information is left unchanged.  This reflects the
motivation of providing information for a bot checking for
regressions; the contents of build-state.json in a tree used for
manual builds that may be only for some configurations are not
particularly important.

	* scripts/build-many-glibcs.py: Import datetime module.
	(Context.__init__): Load JSON build state.  Initialize list of
	status logs.
	(Context.run_builds): Update saved build state.
	(Context.add_makefile_cmdlist): Update list of status logs.
	(Context.load_build_state_json): New function.
	(Context.store_build_state_json): Likewise.
	(Context.clear_last_build_state): Likewise.
	(Context.update_build_state): Likewise.
	(CommandList.status_logs): Likewise.
2016-11-26 00:10:24 +00:00
Joseph Myers a1c9859baf Make build-many-glibcs.py re-exec itself if changed by checkout.
Updating build-many-glibcs.py may result in changes to the default
versions of components, or to the set of components (if e.g. Hurd
support is added and that requires a new component).

It's desirable for the checkout process to leave a source tree that is
ready to use.  If the checkout updated the script itself, that means
it needs to be rerun to cause any new versions or components
referenced by the new script version to be properly checked out.  This
patch makes the script check if it was modified by the checkout
process, and re-exec itself (with the same arguments) if so.

	* scripts/build-many-glibcs.py (Context.__init__): Save text of
	script being executed.
	(Context.get_script_text): New function.
	(Context.exec_self): Likewise.
	(Context.checkout): Re-exec script if changed by checkout process.
2016-11-25 00:58:22 +00:00
Joseph Myers 02c78f02a9 Make build-many-glibcs.py track component versions requested and used.
This patch makes build-many-glibcs.py record the component versions
checked out, and whether those versions were explicitly requested or
defaults, in a file versions.json in the source directory.

The preferred version of a component is the first of: one explicitly
specified on the command line; one explicitly specified on the command
line in a previous run of build-many-glibcs.py; a default version for
that component.  Thus, once you've run build-many-glibcs.py checkout
once with the updated script (making sure to specify the right
versions of any components previously checked out with a non-default
version), in future you can just run it without version specifiers and
it will know when a default-version component has changed its default
version and so should be checked out again.

Because you might have local changes and not want a default-version
component checkout replaced, you need to pass the --replace-sources
option to allow the script to delete and replace a component source
directory automatically; otherwise, it will give an error if a version
has changed.  The script does not try to change branches of git or SVN
checkouts without checking out from scratch; if the version number
requested has changed and --replace-sources is used, the relevant
source directory will be removed completely and a new one checked out
from scratch.

Apart from allowing automatic updates of components with default
versions, this also facilitates bots reporting on the versions used in
a given build.  versions.json contains not just information on the
version number and whether that was requested explicitly, but also git
or SVN revision information intended to be used in email reports from
bots.

	* scripts/build-many-glibcs.py: Import json module.
	(Context.__init__): Take replace_sources argument.  Load
	versions.json.
	(Context.load_versions_json): New function.
	(Context.store_json): Likewise.
	(Context.store_versions_json): Likewise.
	(Context.set_component_version): Likewise.
	(Context.checkout): Update versions.json.  Check for and handle
	changes of version.  Prefer previously explicitly specified
	version to default version.
	(Context.checkout_vcs): Return a revision identifier.
	(Context.git_checkout): Likewise.
	(Context.gcc_checkout): Likewise.
	(get_parser): Add --replace-sources option.
	(main): Pass replace_sources argument to Context call.
2016-11-24 22:25:58 +00:00
Chris Metcalf c650aa612f build-many-glibcs: Revert -fno-isolate-erroneous-paths options for tilepro
TILEPro now has a __builtin_trap instruction in gcc tip and gcc 6.
2016-11-23 12:25:23 -05:00
Joseph Myers 5ee1a4443a Make build-many-glibcs.py use -fno-isolate-erroneous-paths options for tilepro.
My most recent build-many-glibcs.py build with GCC mainline showed
build failures for tilepro with the symptoms (multiple definitions of
symbols building ld.so, see the build log referenced in the GCC bug
referenced in the comment for an example) that correspond to the
isolate-erroneous-paths optimization not being suitable for building
glibc unless the GCC port provides a trap pattern (so __builtin_trap
expands to an inline instruction rather than a call to abort).  Since
tilepro indeed lacks such as pattern in GCC, this patch duly arranges
for this optimization to be disabled when building for tilepro, as it
is for sh.

Tested (compilation only) for tilepro.

	* scripts/build-many-glibcs.py (Context.add_all_configs): Also use
	-fno-isolate-erroneous-paths options for tilepro.
2016-11-22 01:58:26 +00:00
Joseph Myers 8885f97909 Quote shell commands in logs from build-many-glibcs.py.
As requested in
<https://sourceware.org/ml/libc-alpha/2016-11/msg00664.html>, this
patch makes the commands recorded in build-many-glibcs.py quote words
so they can be cut-and-pasted back into a shell.  (Note that these
logs are generated by the wrapper script generated to run commands
with logs, hence the needs for quoting logic to be implemented in that
shell script.)

	* scripts/build-many-glibcs.py (Context.write_files): Make wrapper
	script quote words in command output to log suitably for input to
	the shell.
2016-11-18 18:22:09 +00:00
Joseph Myers c440d5d58d Actually use newly built host libraries in build-many-glibcs.py.
This patch adds the missing GCC configure options required to make use
of the newly built host libraries in build-many-glibcs.py.

	* scripts/build-many-glibcs.py (Config.build_gcc): Configure with
	newly built gmp, mpfr and mpc.
2016-11-17 17:45:41 +00:00
Joseph Myers 0c95f51d84 Fix build-many-glibcs.py style issues.
* scripts/build-many-glibcs.py (os.path): Do not import.
	(Context): Inherit explicitly from object.  Remove blank line
	between class and docstring.
	(Config): Likewise.
	(Glibc): Likewise.
	(Command): Likewise.
	(CommandList): Likewise.
	(Context.write_files): Store chmod mode in a variable.
2016-11-14 23:48:50 +00:00
Joseph Myers 14f95a4203 Add script to build many glibc configurations.
This patch adds a Python (3.5 or later) script to build many different
configurations of glibc, including building the required cross
compilers first.  It's not intended to change any patch testing
requirements, although some people may wish to use it for high-risk
patches such as adding warning options (and it can also be used to
test building, including compiling tests, for an individual
configuration, if e.g. you wish to do such a compilation test of a
patch for an architecture it touches).

The configurations include all the GNU/Linux ABI variants in
<https://sourceware.org/glibc/wiki/ABIList> (although some do not yet
build cleanly) and it would be desirable to cover enough other
variants e.g. for CPUs using different sysdeps directories to test
building each piece of code in glibc at least once.  It would also be
desirable to extend it to cover Hurd and NaCl, which might best be
done by people familiar with those configurations.

You call the script as

build-many-glibcs.py /some/where thing-to-do <other-arguments>

where /some/where is a working directory for the script.  It will
create and use subdirectories build, install, logs therein.  You can
use it with thing-to-do being "checkout" to create a subdirectory src
therein, with subdirectories binutils, gcc, glibc, gmp, linux, mpc,
mpfr with the sources of those components, or create those directories
manually (all except glibc can be symlinks to sources elsewhere).  In
the checkout case, by default it checks out GCC 6 branch, binutils
2.27 branch, glibc mainline and releases of other components.  You can
specify <component>-<version> to choose a version to check out, where
<version> is "vcs-mainline" or "vcs-<branch>" to check out from
version control (only supported for gcc, binutils, glibc) and
otherwise a release version number to download and use a tarball;
components not specified on the command line have default versions
checked out.  If you rerun "checkout" (with the same version
specifications) it will update checkouts from version control, but
will not detect cases where the location something is expected to be
checked out from has changed.

Other than "checkout", thing-to-do is one of host-libraries,
compilers, glibcs.  So you run, in that order:

build-many-glibcs.py /some/where host-libraries
build-many-glibcs.py /some/where compilers
build-many-glibcs.py /some/where glibcs

host-libraries is run once and then those libraries are used for all
the compilers.  compilers can be run once and then used many times for
testing different glibc versions (so a bot only needs to update glibc
and rerun the glibcs task, if using stable GCC / binutils; if testing
the latest versions of the whole toolchain together including mainline
GCC, it would probably want to update everything and rerun both
compilers and glibcs).  You can also name particular variants after
"compilers" or "glibcs" to build just those variants (the possible
variants are hardcoded in the script).

I may add support for allowing the set of configurations to depend on
the GCC version (to get cleaner default results), and optionally
looping over architecture-independent glibc variants of CFLAGS and
configure options as well, for every glibc configuration listed
(e.g. -Os).

GCC versions before 4.9 are not expected to work (the code uses
--with-glibc-version to get the bootstrap GCC appropriately
configured).  There are various problems for particular configurations
as well.

Command-line options to the script: -jN to run N jobs in parallel
(default the number of CPU cores reported by the system); --keep=all
or --keep=failed to control keeping around build directories (default
--keep=none).

	* scripts/build-many-glibcs.py: New file.
2016-11-11 21:07:08 +00:00
Joseph Myers 06b9e94f79 Make check-installed-headers.sh ignore sys/sysctl.h for x32.
check-installed-headers tests were failing for x32 because of the x86
bits/sysctl.h containing a #error for x32.  This patch makes the tests
ignore sys/sysctl.h for x32, similar to the other special-casing of
particular headers.

Tested for x86_64 (full testing for -m64, compile-only for x32).

	* scripts/check-installed-headers.sh: Ignore sys/sysctl.h for x32.
2016-11-07 23:32:17 +00:00
Zack Weinberg 8d3bd94748 Minor corrections to scripts/check-installed-headers.sh.
* scripts/check-installed-headers.sh: Generalize treatment of
	sys/elf.h to all target architectures.
2016-09-28 07:28:00 -04:00
Zack Weinberg 4775578486 Installed header hygiene (BZ#20366): Test of installed headers.
This adds a test to ensure that the problems fixed in the last several
patches do not recur.  Each directory checks the headers that it
installs for two properties: first, each header must be compilable in
isolation, as both C and C++, under a representative combination of
language and library conformance levels; second, there is a blacklist
of identifiers that may not appear in any installed header, currently
consisting of the legacy BSD typedefs.  (There is an exemption for the
headers that define those typedefs, and for the RPC headers.  It may be
necessary to make this more sophisticated if we add more stuff to the
blacklist in the future.)

In order for this test to work correctly, every wrapper header
that actually defines something must guard those definitions with
 #ifndef _ISOMAC.  This is the existing mechanism used by the conform/
tests to tell wrapper headers not to define anything that the public
header wouldn't, and not to use anything from libc-symbols.h.  conform/
only cares for headers that we need to check for standards conformance,
whereas this test applies to *every* header.  (Headers in include/ that
are either installed directly, or are internal-use-only and do *not*
correspond to any installed header, are not affected.)

	* scripts/check-installed-headers.sh: New script.
	* Rules: In each directory that defines header files to be installed,
	run check-installed-headers.sh on them as a special test.
	* Makefile: Likewise for the headers installed at top level.

	* include/aliases.h, include/alloca.h, include/argz.h
	* include/arpa/nameser.h, include/arpa/nameser_compat.h
	* include/elf.h, include/envz.h, include/err.h
	* include/execinfo.h, include/fpu_control.h, include/getopt.h
	* include/gshadow.h, include/ifaddrs.h, include/libintl.h
	* include/link.h, include/malloc.h, include/mcheck.h
	* include/mntent.h, include/netinet/ether.h
	* include/nss.h, include/obstack.h, include/printf.h
	* include/pty.h, include/resolv.h, include/rpc/auth.h
	* include/rpc/auth_des.h, include/rpc/auth_unix.h
	* include/rpc/clnt.h, include/rpc/des_crypt.h
	* include/rpc/key_prot.h, include/rpc/netdb.h
	* include/rpc/pmap_clnt.h, include/rpc/pmap_prot.h
	* include/rpc/pmap_rmt.h, include/rpc/rpc.h
	* include/rpc/rpc_msg.h, include/rpc/svc.h
	* include/rpc/svc_auth.h, include/rpc/xdr.h
	* include/rpcsvc/nis_callback.h, include/rpcsvc/nislib.h
	* include/rpcsvc/yp.h, include/rpcsvc/ypclnt.h
	* include/rpcsvc/ypupd.h, include/shadow.h
	* include/stdio_ext.h, include/sys/epoll.h
	* include/sys/file.h, include/sys/gmon.h, include/sys/ioctl.h
	* include/sys/prctl.h, include/sys/profil.h
	* include/sys/statfs.h, include/sys/sysctl.h
	* include/sys/sysinfo.h, include/ttyent.h, include/utmp.h
	* sysdeps/arm/nacl/include/bits/setjmp.h
	* sysdeps/mips/include/sys/asm.h
	* sysdeps/unix/sysv/linux/include/sys/sysinfo.h
	* sysdeps/unix/sysv/linux/include/sys/timex.h
	* sysdeps/x86/fpu/include/bits/fenv.h:
	Add #ifndef _ISOMAC guard around internal declarations.
	Add multiple-inclusion guard if not already present.
2016-09-23 08:43:56 -04:00
Florian Weimer e68c8d2757 sysd-rules: Cut down the number of rtld-% pattern rules
rtld only needs shared objects, so the other patterns are pointless and
significantly increase the work make has to perform while identifying
which pattern rule to apply.
2016-09-20 10:41:05 +02:00
Samuel Thibault 66abf9bfbe mach: Add more allowed external headers
* scripts/check-local-headers.sh (exclude): Add mach_debug/.
2016-08-21 03:24:55 +02:00
Siddhesh Poyarekar c10f90dcef Revert "Add pretty printers for the NPTL lock types"
This reverts commit 62ce266b0b.

The change is not mature enough because it needs the following fixes:

1. Redirect test output to a file like other tests

2. Eliminate the need to use a .gdbinit because distributions will
   break without it.  I should have caught that but I was in too much
   of a hurry to get the patch in :/

3. Feature checking during configure to determine things like minimum
   required gdb version, python-pexpect version, etc. to make sure
   that tests work correctly.
2016-07-11 20:32:12 +05:30
Martin Galvan 62ce266b0b Add pretty printers for the NPTL lock types
This patch adds pretty printers for the following NPTL types:

- pthread_mutex_t
- pthread_mutexattr_t
- pthread_cond_t
- pthread_condattr_t
- pthread_rwlock_t
- pthread_rwlockattr_t

To load the pretty printers into your gdb session, do the following:

python
import sys
sys.path.insert(0, '/path/to/glibc/build/nptl/pretty-printers')
end

source /path/to/glibc/source/pretty-printers/nptl-printers.py

You can check which printers are registered and enabled by issuing the
'info pretty-printer' gdb command. Printers should trigger automatically when
trying to print a variable of one of the types mentioned above.

The printers are architecture-independent, and were manually tested on both
the gdb CLI and Eclipse CDT.

In order to work, the printers need to know the values of various flags that
are scattered throughout pthread.h and pthreadP.h as enums and #defines. Since
replicating these constants in the printers file itself would create a
maintenance burden, I wrote a script called gen-py-const.awk that Makerules uses
to extract the constants. This script is pretty much the same as gen-as-const.awk,
except it doesn't cast the constant values to 'long' and is thorougly documented.
The constants need only to be enumerated in a .pysym file, which is then referenced
by a Make variable called gen-py-const-headers.

As for the install directory, I discussed this with Mike Frysinger and Siddhesh
Poyarekar, and we agreed that it can be handled in a separate patch, and it shouldn't
block merging of this one.

In addition, I've written a series of test cases for the pretty printers.
Each lock type (mutex, condvar and rwlock) has two test programs, one for itself
and other for its related 'attributes' object. Each test program in turn has a
PExpect-based Python script that drives gdb and compares its output to the
expected printer's. The tests run on the glibc host, which is assumed to have
both gdb and PExpect; if either is absent the tests will fail with code 77
(UNSUPPORTED). For cross-testing you should use cross-test-ssh.sh as test-wrapper.
I've tested the printers on both a native build and a cross build using a Beaglebone
Black, with the build system's filesystem shared with the board through NFS.

Finally, I've written a README that explains all this and more.

Hopefully this should be good to go in now. Thanks.

ChangeLog:

2016-07-04  Martin Galvan  <martin.galvan@tallertechnologies.com>

	* Makeconfig (build-hardcoded-path-in-tests): Set to 'yes' for shared builds
	if tests-need-hardcoded-path is defined.
	(all-subdirs): Add pretty-printers.
	* Makerules ($(py-const)): New rule.
	* Rules (others): Add $(py-const), if defined.
	* nptl/Makefile (gen-py-const-headers): Define.
	* nptl/nptl-printers.py: New file.
	* nptl/nptl_lock_constants.pysym: Likewise.
	* pretty-printers/Makefile: Likewise.
	* pretty-printers/README: Likewise.
	* pretty-printers/test-condvar-attributes.c: Likewise.
	* pretty-printers/test-condvar-attributes.p: Likewise.
	* pretty-printers/test-condvar-printer.c: Likewise.
	* pretty-printers/test-condvar-printer.py: Likewise.
	* pretty-printers/test-mutex-attributes.c: Likewise.
	* pretty-printers/test-mutex-attributes.py: Likewise.
	* pretty-printers/test-mutex-printer.c: Likewise.
	* pretty-printers/test-mutex-printer.py: Likewise.
	* pretty-printers/test-rwlock-attributes.c: Likewise.
	* pretty-printers/test-rwlock-attributes.py: Likewise.
	* pretty-printers/test-rwlock-printer.c: Likewise.
	* pretty-printers/test-rwlock-printer.py: Likewise.
	* pretty-printers/test_common.py: Likewise.
	* scripts/gen-py-const.awk: Likewise.
2016-07-08 20:03:05 +05:30
Samuel Thibault 94d211c879 Add more hurd exception to local headers list
* scripts/check-local-headers.sh (exclude): Add hurd/ihash.h, and
	include .*-.*/ in addition to .*-.*-.*/ (i.e. i386-gnu in addition to
	i386-linux-gnu).
2016-05-30 19:03:01 +02:00
Nick Alcock 5057feffcc Allow overriding of CFLAGS as well as CPPFLAGS for rtld.
We need this to pass -fno-stack-protector to all the pieces of rtld in
non-elf/ directories.
2016-04-09 23:48:32 -04:00
Marko Myllynen bc49a7afd3 Make shebang interpreter directives consistent
Undo changes in files maintained elsewhere.
2016-01-08 14:09:06 -05:00
Marko Myllynen 48d0341cdd Make shebang interpreter directives consistent 2016-01-07 04:03:21 -05:00
Joseph Myers 084e7d57bb Update miscellaneous files from upstream sources.
This patch updates texinfo.tex, config.guess, config.sub and
move-if-change from their respective upstream sources.

	* manual/texinfo.tex: Update to version 2016-01-04.21 with
	trailing whitespace removed.
	* scripts/config.guess: Update to version 2016-01-01.
	* scripts/config.sub: Update to version 2016-01-01.
	* scripts/move-if-change: Update from gnulib.
2016-01-05 18:04:06 +00:00
Joseph Myers 1979f3c1ad Update copyright dates not handled by scripts/update-copyrights.
I've updated copyright dates in glibc for 2016.  This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.

	* NEWS: Update copyright dates.
	* catgets/gencat.c (print_version): Likewise.
	* csu/version.c (banner): Likewise.
	* debug/catchsegv.sh: Likewise.
	* debug/pcprofiledump.c (print_version): Likewise.
	* debug/xtrace.sh (do_version): Likewise.
	* elf/ldconfig.c (print_version): Likewise.
	* elf/ldd.bash.in: Likewise.
	* elf/pldd.c (print_version): Likewise.
	* elf/sotruss.sh: Likewise.
	* elf/sprof.c (print_version): Likewise.
	* iconv/iconv_prog.c (print_version): Likewise.
	* iconv/iconvconfig.c (print_version): Likewise.
	* locale/programs/locale.c (print_version): Likewise.
	* locale/programs/localedef.c (print_version): Likewise.
	* login/programs/pt_chown.c (print_version): Likewise.
	* malloc/memusage.sh (do_version): Likewise.
	* malloc/memusagestat.c (print_version): Likewise.
	* malloc/mtrace.pl: Likewise.
	* manual/libc.texinfo: Likewise.
	* nptl/version.c (banner): Likewise.
	* nscd/nscd.c (print_version): Likewise.
	* nss/getent.c (print_version): Likewise.
	* nss/makedb.c (print_version): Likewise.
	* posix/getconf.c (main): Likewise.
	* scripts/test-installation.pl: Likewise.
	* sysdeps/unix/sysv/linux/lddlibc4.c (main): Likewise.
2016-01-04 16:26:30 +00:00
Joseph Myers f7a9f785e5 Update copyright dates with scripts/update-copyrights. 2016-01-04 16:05:18 +00:00
Mike Frysinger d108b75561 list-fixed-bugs: use argparse for the commandline
This makes the interface more friendly to users.
2015-12-29 13:34:51 -05:00
Ludovic Courtès db340c904d Use shell's builtin pwd.
Insisting on /bin/pwd is unnecessary nowadays.  Autoconf-generated
scripts have been using the shell's built-in "pwd" for a long time.`
2015-12-02 23:58:15 -05:00
Florian Weimer c100dca32a Implement "make update-all-abi"
* scripts/update-abilist.sh: New file.
	* Makefile (+subdir_targets): Add subdir_update-all-abi.
	* Makerules (update-all-abi-%, update-all-abi)
	(subdir_update-all-abi): New targets.
	* elf/Makefile (update-all-abi): New target.
2015-11-24 17:10:19 +01:00
Mike Frysinger 1a44f1433e pylintrc: disable reports
I've never found these useful, nor found anyone else who likes them.
Turn them off by default.
2015-11-11 13:41:57 -05:00
Florian Weimer 8c77b6ad40 Simplify the abilist format
The new format lists the version on each line, as in:

	VERSION SYMBOL TYPE [VALUE]

This makes it easier to process the files with line-oriented tools.

The abilist files were converted with this awk script:

/^[^ ]/ { version = $1 }
/^ / { print version, substr($0, 2) }

And sorted under the "C" locale with sort.
2015-11-06 13:58:53 +01:00
Joseph Myers f268ab5f69 Add script to list fixed bugs for the NEWS file.
This patch adds a script to list fixed bugs for the NEWS file, as
proposed in
<https://sourceware.org/ml/libc-alpha/2015-10/msg01043.html>, with the
bugs listed in the format proposed in
<https://sourceware.org/ml/libc-alpha/2015-11/msg00088.html>.

	* scripts/list-fixed-bugs.py: New file.
2015-11-05 22:18:32 +00:00
H.J. Lu d3d9c95aef Support PLT and GOT references in local PIC check
Linker in binutils 2.26 and newer generate GOT references instead
PLT references when -z now is passed to linker.  We need to extend
scripts/localplt.awk to allow PLT or GOT references.

	[BZ #19007]
	* scripts/localplt.awk: Also allow GOT references.
	* sysdeps/unix/sysv/linux/i386/localplt.data: Mark
	_Unwind_Find_FDE, calloc, memalign, realloc and __libc_memalign
	with "+ REL R_386_GLOB_DAT".
	* sysdeps/x86_64/localplt.data: Mark calloc, memalign, realloc
	and __libc_memalign with "+ RELA R_X86_64_GLOB_DAT".
2015-10-14 06:00:02 -07:00
Joseph Myers 5c34f1b5bb Remove scripts/rpm2dynsym.sh.
The file scripts/rpm2dynsym.sh appears to be unused anywhere in glibc.
This patch removes this script.

Tested for x86_64 and x86 (testsuite, and that installed shared
libraries are unchanged by the patch).

	* scripts/rpm2dynsym.sh: Remove file.
2015-10-06 20:45:47 +00:00
Andrew Senkevich a08e80d114 [BZ #18796]
* scripts/test-installation.pl: Don't add -lmvec to build options if
    libmvec wasn't built.
2015-08-19 16:54:28 +03:00
Mike Frysinger d6d20de8b7 gawk: fix gensub usage
The third arg can either be a string with "g" or "G", or it is a number.
The empty string elicits a warning with newer versions like so:
gawk: scripts/sysd-rules.awk:56: warning: gensub: third argument `' treated as 1
2015-08-09 04:19:17 -04:00
H.J. Lu 9637d8a253 Extend local PLT reference check
On x86, linker in binutils 2.26 and newer consolidates R_*_JUMP_SLOT with
R_*_GLOB_DAT relocation against the same symbol.  This patch extends
local PLT reference check to support alternate relocations.

	[BZ #18078]
	* scripts/check-localplt.awk: Support alternate relocations.
	* scripts/localplt.awk: Also check relocations in DT_RELA/DT_REL
	sections.
	* sysdeps/unix/sysv/linux/i386/localplt.data: Mark free and
	malloc entries with + REL R_386_GLOB_DAT.
	* sysdeps/x86_64/localplt.data: New file.
2015-07-29 11:58:06 -07:00
Samuel Thibault 9e70234cbd Add more exception to local headers list
* scripts/check-local-headers.sh (exclude): Add device/,
	hurd/hurd_types.h, hurd/ioctl_types.h, hurd/paths.h, hurd/ioctls.defs,
	cthreads.h.
2015-03-23 00:31:02 +01:00
Roland McGrath 44a6213c8e Let tests result in UNSUPPORTED; use that for unbuildable C++ cases 2015-03-10 15:13:14 -07:00
Alan Modra a0af371c25 Fix localplt test breakage with new readelf
Since 2014-11-24 binutils git commit bb4d2ac2, readelf has appended
the symbol version to symbols shown in reloc dumps.

	[BZ #16512]
	* scripts/localplt.awk: Strip off symbol version.
	* NEWS: Mention bug fix.
2015-03-03 23:43:18 +10:30
Joseph Myers 9706dc5f53 Update copyright dates not handled by scripts/update-copyrights.
I've updated copyright dates in glibc for 2015.  This is the patch for
the changes not generated by scripts/update-copyrights and subsequent
build / regeneration of generated files.  Apart from the files updated
last time (of which sotruss.ksh had moved to sotruss.sh during the
year) this also updates nptl/version.c (missed from 2006 until
October) and sysdeps/unix/sysv/linux/lddlibc4.c (missed since 2009).

	* NEWS: Update copyright dates.
	* catgets/gencat.c (print_version): Likewise.
	* csu/version.c (banner): Likewise.
	* debug/catchsegv.sh: Likewise.
	* debug/pcprofiledump.c (print_version): Likewise.
	* debug/xtrace.sh (do_version): Likewise.
	* elf/ldconfig.c (print_version): Likewise.
	* elf/ldd.bash.in: Likewise.
	* elf/pldd.c (print_version): Likewise.
	* elf/sotruss.sh: Likewise.
	* elf/sprof.c (print_version): Likewise.
	* iconv/iconv_prog.c (print_version): Likewise.
	* iconv/iconvconfig.c (print_version): Likewise.
	* locale/programs/locale.c (print_version): Likewise.
	* locale/programs/localedef.c (print_version): Likewise.
	* login/programs/pt_chown.c (print_version): Likewise.
	* malloc/memusage.sh (do_version): Likewise.
	* malloc/memusagestat.c (print_version): Likewise.
	* malloc/mtrace.pl: Likewise.
	* manual/libc.texinfo: Likewise.
	* nptl/version.c (banner): Likewise.
	* nscd/nscd.c (print_version): Likewise.
	* nss/getent.c (print_version): Likewise.
	* nss/makedb.c (print_version): Likewise.
	* posix/getconf.c (main): Likewise.
	* scripts/test-installation.pl: Likewise.
	* sysdeps/unix/sysv/linux/lddlibc4.c (main): Likewise.
2015-01-02 16:54:45 +00:00
Joseph Myers b168057aaa Update copyright dates with scripts/update-copyrights. 2015-01-02 16:29:47 +00:00
Siddhesh Poyarekar b217c15fc3 Remove uses of sprintf in gen-posix-conf-vars.awk
Simply some code by replacing sprintf in the awk script.
2015-01-02 11:16:35 +05:30
Siddhesh Poyarekar 195f3afcbe Use one-dimension arrays in gen-posix-conf-vars.awk
True multi-dimensional arrays were introduced in awk 4.0 and we
support awk versions as early as 3.12.  Use a single subscript of the
form prefix_conf instead of two dimensions to work around this
limitation.  We also need one additional array of just the conf names
subscripted by the prefix_conf to print the names for the
specifications.

	* scripts/gen-posix-conf-vars.awk: Don't use multi-dimensional
	arrays.
2014-12-31 13:22:58 +05:30
Siddhesh Poyarekar 7910c2ae73 Make type for spec variable size as size_t 2014-12-29 19:56:52 +05:30
Siddhesh Poyarekar 50cbbaa935 Use posix-conf-vars.list to generate spec array
This patch adds support to generate the spec array in getconf from the
conf.list.  The generated code is mostly unchanged.  the only changes
are due to the change in layout of the spec and val arrays in the ELF.

The val array can also be auto-generated from posix-conf-vars.list
once the remaining macros are added to it.

	* posix/posix-conf-vars.list (SPEC:XBS5): Add sysconf prefix.
	* posix/confstr.c: Define NEED_SPEC_ARRAY to 0.
	* posix/posix-envs.def: Likewise.
	* sysdeps/posix/sysconf.c: Likewise.
	* posix/getconf.c: Define NEED_SPEC_ARRAY to 1.
	(specs): Remove array.
	* scripts/gen-posix-conf-vars.awk: Support generation of specs
	array.
2014-12-29 19:56:27 +05:30
Siddhesh Poyarekar 4a6aca7bf8 Remove Wundef warnings for specification macros
This patch adds a file posix-conf-vars.list that is used to generate
macros to determine if a macro is defined as set, unset or not
defined.  gen-posix-conf-vars.awk processes this file and generates a
header (posix-conf-vars-def.h) with these macros.  A new header
posix-conf-vars.h includes this generated header and defines accessor
macros for the generated macros.

Tested on x86_64.

	* posix/Makefile (before-compile): Add posix-conf-vars-def.h.
	($(objpfx)posix-conf-vars-def.h): New target.
	* posix/posix-conf-vars.list: New file.
	* posix/posix-conf-vars.h: New file.
	* posix/confstr.c: Include posix-conf-vars.h.
	(confstr): Use CONF_IS_* macros.
	* posix/posix-envs.def: Include posix-conf-vars.h.  Use
	CONF_IS_* macros.
	* scripts/gen-posix-conf-vars.awk: New file.
2014-12-29 17:37:54 +05:30
Siddhesh Poyarekar 130ac68ca2 Auto-generate libc-modules.h
Remove libc-modules.h from the tree and auto-generate it from
soversions.i and the list of modules in the built-modules variable
defined in Makeconfig.  Macros generated have increasing numbered
values, with built-modules having lower values starting from 1,
following which a separator value LIBS_BEGIN is added and then finally
the library names from soversions.i are appended to the list.  This
allows us to conveniently differentiate between the versioned
libraries and other built modules, which is needed in errno.h and
netdb.h to decide whether to use an internal symbol or an external
one.

Verified that generated code remains unchanged on x86_64.

	* Makeconfig (built-modules): List non-library modules to be
	built.
	(module-cppflags): Include libc-modules.h for
	everything except shlib-versions.v.i.
	(CPPFLAGS): Use it.
	(before-compile): Add libc-modules.h.
	($(common-objpfx)libc-modules.h,
	$(common-objpfx)libc-modules.stmp): New targets.
	(common-generated): Add libc-modules.h and libc-modules.stmp.
	($(common-objpfx)Versions.v.i): Depend on libc-modules.h.
	* include/libc-symbols.h: Don't include libc-modules.h.
	* include/libc-modules.h: Remove file.
	* scripts/gen-libc-modules.awk: New script to generate
	libc-modules.h.
	* sysdeps/unix/Makefile ($(common-objpfx)sysd-syscalls):
	Depend on libc-modules.stmp.
2014-11-19 12:16:00 +05:30
Joseph Myers 93ae1ebaa6 Clean up gnu/lib-names.h generation (bug 14171).
This patch eliminates the mixture of SONAME information in
shlib-versions files and SONAME information used to generate
gnu/lib-names.h in makefiles, with the information in the makefiles
being removed so all this information comes from the shlib-versions
files.

So that gnu/lib-names.h supports multiple ABIs, it is changed to be
generated on the same basis as gnu/stubs.h: when there are multiple
ABIs, gnu/lib-names.h is a wrapper header (the same header installed
whatever ABI is being built) and separate headers such as
gnu/lib-names-64.h contain the substantive contents (only one such
header being installed by any glibc build).

The rules for building gnu/lib-names.h were moved from Makeconfig to
Makerules because they need to come after sysdeps makefiles are
included (now that "ifndef abi-variants" is a toplevel conditional on
the rules rather than $(abi-variants) being evaluated later inside the
commands for a rule).

Tested for x86_64 and x86 that the installed shared libraries are
unchanged by this patch, and examined the installed gnu/lib-names*.h
headers by hand.  Also tested the case of a single ABI (where there is
just a single header installed, again like stubs.h) by hacking
abi-variants to empty for x86_64.

	[BZ #14171]
	* Makeconfig [$(build-shared) = yes]
	($(common-objpfx)soversions.mk): Don't handle SONAMEs specified in
	makefiles.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	($(common-objpfx)gnu/lib-names.h): Remove rule.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	($(common-objpfx)gnu/lib-names.stmp): Likewise.  Split and moved
	to Makerules.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	(before-compile): Don't append $(common-objpfx)gnu/lib-names.h
	here.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	(common-generated): Don't append gnu/lib-names.h and
	gnu/lib-names.stmp here.
	* Makerules [$(build-shared) = yes && $(soversions.mk-done) = t]
	(lib-names-h-abi): New variable.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	(lib-names-stmp-abi): Likewise.
	[$(build-shared) = yes && $(soversions.mk-done) = t &&
	abi-variants] (before-compile): Append
	$(common-objpfx)$(lib-names-h-abi).
	[$(build-shared) = yes && $(soversions.mk-done) = t &&
	abi-variants] (common-generated): Append gnu/lib-names.h.
	[$(build-shared) = yes && $(soversions.mk-done) = t &&
	abi-variants] (install-others-nosubdir): Depend on
	$(inst_includedir)/$(lib-names-h-abi).
	[$(build-shared) = yes && $(soversions.mk-done) = t &&
	abi-variants] ($(common-objpfx)gnu/lib-names.h): New rule.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	($(common-objpfx)$(lib-names-h-abi)): New rule.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	($(common-objpfx)$(lib-names-stmp-abi)): Likewise.
	[$(build-shared) = yes && $(soversions.mk-done) = t]
	(common-generated): Append $(lib-names-h-abi) and
	$(lib-names-stmp-abi).
	* scripts/lib-names.awk: Do not handle multi being set.
	* sysdeps/unix/sysv/linux/aarch64/Makefile (abi-lp64-ld-soname):
	Remove variable.
	(abi-lp64_be-ld-soname): Likewise.
	* sysdeps/unix/sysv/linux/arm/Makefile (abi-soft-ld-soname):
	Likewise.
	(abi-hard-ld-soname): Likewise.
	* sysdeps/unix/sysv/linux/i386/shlib-versions: New file.
	* sysdeps/unix/sysv/linux/mips/Makefile (abi-o32_soft-ld-soname):
	Remove variable.
	(abi-o32_hard-ld-soname): Likewise.
	(abi-o32_soft_2008-ld-soname): Likewise.
	(abi-o32_hard_2008-ld-soname): Likewise.
	(abi-n32_soft-ld-soname): Likewise.
	(abi-n32_hard-ld-soname): Likewise.
	(abi-n32_soft_2008-ld-soname): Likewise.
	(abi-n32_hard_2008-ld-soname): Likewise.
	(abi-n64_soft-ld-soname): Likewise.
	(abi-n64_hard-ld-soname): Likewise.
	(abi-n64_soft_2008-ld-soname): Likewise.
	(abi-n64_hard_2008-ld-soname): Likewise.
	* sysdeps/unix/sysv/linux/powerpc/Makefile (abi-64-v1-ld-soname):
	Likewise.
	(abi-64-v2-ld-soname): Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/shlib-versions: Add
	ld.so entries.
	* sysdeps/unix/sysv/linux/s390/Makefile (abi-64-ld-soname): Remove
	variable.
	* sysdeps/unix/sysv/linux/s390/s390-64/shlib-versions: Add ld.so
	entry.
	* sysdeps/unix/sysv/linux/x86/Makefile (abi-32-ld-soname): Remove
	variable.
	(abi-64-ld-soname): Likewise.
	(abi-x32-ld-soname): Likewise.
	* sysdeps/unix/sysv/linux/x86_64/64/shlib-versions: Add ld.so
	entry.
	* sysdeps/unix/sysv/linux/x86_64/x32/shlib-versions: Likewise.
2014-09-26 17:33:04 +00:00
Joseph Myers af296fcdab Remove bitrotten --enable-oldest-abi (bug 6652).
This patch removes the --enable-oldest-abi configure option, which has
long been bitrotten (as reported in bug 6652).  The principle of
removing this option was agreed in the thread starting at
<https://sourceware.org/ml/libc-alpha/2013-07/msg00174.html>.

Tested for x86_64 and x86 that the installed shared libraries other
than libc.so are unchanged by this patch and that libc.so disassembly
and symbol versions are unchanged (debug info changes because of
changed line numbers in csu/version.c).

	[BZ #6652]
	* Makeconfig (soversions-default-setname): Remove variable.
	($(common-objpfx)soversions.i): Don't pass default_setname to
	soversions.awk.
	* Makerules ($(common-objpfx)abi-versions.h): Don't pass
	oldest_abi to abi-versions.awk.
	* config.h.in (GLIBC_OLDEST_ABI): Remove macro undefine.
	* config.make.in (oldest-abi): Remove variable.
	* configure.ac (--enable-oldest-abi): Remove configure option.
	* configure: Regenerated.
	* csu/version.c (banner) [GLIBC_OLDEST_ABI]: Remove conditional
	text.
	* scripts/abi-versions.awk: Do not handle oldest_abi variable.
	* scripts/soversions.awk: Do not handle default_setname variable.
	* sysdeps/mach/hurd/configure.ac: Do not handle oldest_abi
	variable.
	* sysdeps/mach/hurd/configure: Regenerated.
	* sysdeps/unix/sysv/linux/configure.ac: Do not handle oldest_abi
	variable.
	* sysdeps/unix/sysv/linux/configure: Regenerated.
2014-09-16 17:45:03 +00:00
Joseph Myers ba90e05052 Remove configuration name patterns from shlib-versions.
This patch removes the first column (patterns matching configuration
names) from shlib-versions, leaving shlib-versions entry selection
based purely on sysdeps directories.

An implication of this removal is that the default for any non-Linux
ports using NPTL will be the same SONAMEs for NPTL libraries as for
Linux (as those defaults, previously limited to .*-.*-linux.*, are
left in nptl/shlib-versions and nptl_db/shlib-versions).

Special host_os handling in configure.ac that was purely for
shlib-versions is removed.  (The host_os setting is still used for
libc-abis - see
<https://sourceware.org/ml/libc-alpha/2014-01/msg00375.html> regarding
that - but no entries there are affected by this change.)

Tested on x86_64 and x86 that the installed shared libraries are
unchanged by this patch.

	* scripts/soversions.awk: Do not handle configuration names.
	* Makeconfig ($(common-objpfx)soversions.i): Do not pass cpu,
	vendor and os variables to soversions.awk.
	* configure.ac: Do not modify gnu-* host_os.
	* configure: Regenerated
	* shlib-versions: Remove first column with configuration names.
	* nptl/shlib-versions: Likewise.
	* nptl_db/shlib-versions: Likewise.
	* sysdeps/hppa/shlib-versions: Likewise.
	* sysdeps/m68k/shlib-versions: Likewise.
	* sysdeps/mach/hurd/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/aarch64/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/alpha/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/arm/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/hppa/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/ia64/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/m68k/coldfire/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/microblaze/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/mips/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/powerpc/powerpc64/shlib-versions:
	Likewise.
	* sysdeps/unix/sysv/linux/s390/s390-64/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/sh/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc32/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/sparc/sparc64/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/tile/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/64/shlib-versions: Likewise.
	* sysdeps/unix/sysv/linux/x86_64/x32/shlib-versions: Likewise.

libidn/ChangeLog:
	* shlib-versions: Remove first column with configuration names.
2014-09-12 12:28:47 +00:00
Joseph Myers f1eafb41fa Remove shlib-versions ABI names support.
shlib-versions files can contain ABI lines that map triplets to a
canonical ABI name.  This name was once used for various purposes
where test baseline files for different ABIs went in a single
directory; now these purposes use sysdeps files, generation of headers
which have per-ABI variants uses abi-variants and related Makefile
variables and the shlib-versions ABI names are unused.  This patch
duly removes those lines and associated build system support for them.

Tested for x86_64 (both a full testsuite run and confirming the
installed shared libraries are unchanged by the patch).

	* Makeconfig ($(common-objpfx)soversions.mk): Do not generate
	abi-name definition.
	* scripts/soversions.awk: Do not handle or generate ABI lines.
	* shlib-versions: Remove ABI entries.
	* sysdeps/powerpc/nofpu/shlib-versions: Remove file.
	* sysdeps/x86_64/x32/shlib-versions: Remove ABI entry.
2014-06-27 20:24:23 +00:00