Commit Graph

2466 Commits

Author SHA1 Message Date
Nick Alcock 791915db42 libctf: handle nonrepresentable types at link time
GCC can emit references to type 0 to indicate that this type is one that
is not representable in the version of CTF it emits (for instance,
version 3 cannot encode vector types).  Type 0 is already used in the
function section to indicate padding inserted to skip functions we do
not want to encode the type of, so using zero in this way is a good
extension of the format: but libctf reports such types as ECTF_BADID,
which is indistinguishable from file corruption via links to truly
nonexistent types with IDs like 0xDEADBEEF etc, which we really do want
to stop for.

In particular, this stops all traversals of types dead at this point,
preventing us from even dumping CTF files containing unrepresentable
types to see what's going on!

So add a new error, ECTF_NONREPRESENTABLE, which is returned by
recursive type resolution when a reference to a zero type is found.  (No
zero type is ever emitted into the CTF file by GCC, only references to
one).  We can't do much with types that are ultimately nonrepresentable,
but we can do enough to keep functioning.

Adjust ctf_add_type to ensure that top-level types of type zero and
structure and union members of ultimate type zero are simply skipped
without reporting an error, so we can copy structures and unions that
contain nonrepresentable members (skipping them and leaving a hole where
they would be, so no consumers downstream of the linker need to worry
about this): adjust the dumper so that we dump members of
nonrepresentable types in a simple form that indicates
nonrepresentability rather than terminating the dump, and do not falsely
assume all errors to be -ENOMEM: adjust the linker so that types that
fail to get added are simply skipped, so that both nonrepresentable
types and outright errors do not terminate the type addition, which
could skip many valid types and cause further errors when variables of
those types are added.

In future, when we gain the ability to call back to the linker to report
link-time type resolution errors, we should report failures to add all
but nonrepresentable types.  But we can't do that yet.

v5: Fix tabdamage.

include/
	* ctf-api.h (ECTF_NONREPRESENTABLE): New.
libctf/
	* ctf-types.c (ctf_type_resolve): Return ECTF_NONREPRESENTABLE on
	type zero.
	* ctf-create.c (ctf_add_type): Detect and skip nonrepresentable
	members and types.
	(ctf_add_variable): Likewise for variables pointing to them.
	* ctf-link.c (ctf_link_one_type): Do not warn for nonrepresentable
	type link failure, but do warn for others.
	* ctf-dump.c (ctf_dump_format_type): Likewise.  Do not assume all
	errors to be ENOMEM.
	(ctf_dump_member): Likewise.
	(ctf_dump_type): Likewise.
	(ctf_dump_header_strfield): Do not assume all errors to be ENOMEM.
	(ctf_dump_header_sectfield): Do not assume all errors to be ENOMEM.
	(ctf_dump_header): Likewise.
	(ctf_dump_label): likewise.
	(ctf_dump_objts): likewise.
	(ctf_dump_funcs): likewise.
	(ctf_dump_var): likewise.
	(ctf_dump_str): Likewise.
2019-10-03 17:04:56 +01:00
Nick Alcock 87279e3cef libctf: installable libctf as a shared library
This lets other programs read and write CTF-format data.

Two versioned shared libraries are created: libctf.so and
libctf-nobfd.so.  They contain identical content except that
libctf-nobfd.so contains no references to libbfd and does not implement
ctf_open, ctf_fdopen, ctf_bfdopen or ctf_bfdopen_ctfsect, so it can be
used by programs that cannot use BFD, like readelf.

The soname major version is presently .0 until the linker API
stabilizes, when it will flip to .1 and hopefully never change again.

New in v3.
v4: libtoolize and turn into a pair of shared libraries.  Drop
    --enable-install-ctf: now controlled by --enable-shared and
    --enable-install-libbfd, like everything else.
v5: Add ../bfd to ACLOCAL_AMFLAGS and AC_CONFIG_MACRO_DIR.  Fix tabdamage.

	* Makefile.def (host_modules): libctf is no longer no_install.
	* Makefile.in: Regenerated.
libctf/
	* configure.ac (AC_DISABLE_SHARED): New, like opcodes/.
	(LT_INIT): Likewise.
	(AM_INSTALL_LIBBFD): Likewise.
	(dlopen): Note why this is necessary in a comment.
	(SHARED_LIBADD): Initialize for possibly-PIC libiberty: derived from
	opcodes/.
	(SHARED_LDFLAGS): Likewise.
	(BFD_LIBADD): Likewise, for libbfd.
	(BFD_DEPENDENCIES): Likewise.
	(VERSION_FLAGS): Initialize, using a version script if ld supports
	one, or libtool -export-symbols-regex otherwise.
	(AC_CONFIG_MACRO_DIR): Add ../BFD.
	* Makefile.am (ACLOCAL_AMFLAGS): Likewise.
	(INCDIR): New.
	(AM_CPPFLAGS): Use $(srcdir), not $(top_srcdir).
	(noinst_LIBRARIES): Replace with...
	[INSTALL_LIBBFD] (lib_LTLIBRARIES): This, or...
	[!INSTALL_LIBBFD] (noinst_LTLIBRARIES): ... this, mentioning new
	libctf-nobfd.la as well.
	[INSTALL_LIBCTF] (include_HEADERS): Add the CTF headers.
	[!INSTALL_LIBCTF] (include_HEADERS): New, empty.
	(libctf_a_SOURCES): Rename to...
	(libctf_nobfd_la_SOURCES): ... this, all of libctf other than
	ctf-open-bfd.c.
	(libctf_la_SOURCES): Now derived from libctf_nobfd_la_SOURCES,
	with ctf-open-bfd.c added.
	(libctf_nobfd_la_LIBADD): New, using @SHARED_LIBADD@.
	(libctf_la_LIBADD): New, using @BFD_LIBADD@ as well.
	(libctf_la_DEPENDENCIES): New, using @BFD_DEPENDENCIES@.
	* Makefile.am [INSTALL_LIBCTF]: Use it.
	* aclocal.m4: Add ../bfd/acinclude.m4, ../config/acx.m4, and the
	libtool macros.
	* libctf.ver: New, everything is version LIBCTF_1.0 currently (even
	the unstable components).
	* Makefile.in: Regenerated.
	* config.h.in: Likewise.
	* configure: Likewise.
binutils/
	* Makefile.am (LIBCTF): Mention the .la file.
	(LIBCTF_NOBFD): New.
	(readelf_DEPENDENCIES): Use it.
	(readelf_LDADD): Likewise.
	* Makefile.in: Regenerated.
ld/
	* configure.ac (TESTCTFLIB): Set to the .so or .a, like TESTBFDLIB.
	* Makefile.am (TESTCTFLIB): Use it.
	(LIBCTF): Use the .la file.
	(check-DEJAGNU): Use it.
	* Makefile.in: Regenerated.
	* configure: Likewise.
include/
	* ctf-api.h: Note the instability of the ctf_link interfaces.
2019-10-03 17:04:56 +01:00
Nick Alcock 1ff6de0312 bfd, ld: add CTF section linking
This is quite complicated because the CTF section's contents depend on
the final contents of the symtab and strtab, because it has two sections
whose contents are shuffled to be in 1:1 correspondence with the symtab,
and an internal strtab that gets deduplicated against the ELF strtab
(with offsets adjusted to point into the ELF strtab instead).  It is
also compressed if large enough, so its size depends on its contents!

So we cannot construct it as early as most sections: we cannot even
*begin* construction until after the symtab and strtab are finalized.
Thankfully there is already one section treated similarly: compressed
debugging sections: the only differences are that compressed debugging
sections have extra handling to deal with their changing name if
compressed (CTF sections are always called ".ctf" for now, though we
have reserved ".ctf.*" against future use), and that compressed
debugging sections have previously-uncompressed content which has to be
stashed away for later compression, while CTF sections have no content
at all until we generate it (very late).

BFD also cannot do the link itself: libctf knows how to do it, and BFD
cannot call libctf directly because libctf already depends on bfd for
file I/O.  So we have to use a pair of callbacks, one, examine_strtab,
which allows a caller to examine the symtab and strtab after
finalization (called from elf_link_swap_symbols_out(), right before the
symtabs are written, and after the strtab has been finalized), and one
which actually does the emission (called emit_ctf simply because it is
grouped with a bunch of section-specific late-emission function calls at
the bottom of bfd_elf_final_link, and a section-specific name seems best
for that).  emit_ctf is actually called *twice*: once from lang_process
if the emulation suggests that this bfd target does not examine the
symtab or strtab, and once via a bfd callback if it does.  (This means
that non-ELF targets still get CTF emitted, even though the late CTF
emission stage is never called for them).

v2: merged with non-ELF support patch: slight commit message
    adjustments.
v3: do not spend time merging CTF, or crash, if the CTF section is
    explicitly discarded.  Do not try to merge or compress CTF unless
    linking.
v4: add CTF_COMPRESSION_THRESHOLD.  Annul the freed input ctf_file_t's
    after writeout: set SEC_IN_MEMORY on the output contents so a future
    bfd enhancement knows it could free it.  Add SEC_LINKER_CREATED |
    SEC_KEEP to avoid having to add .ctf to the linker script.  Drop
    now-unnecessary ldlang.h-level elf-bfd.h include and hackery around
    it.  Adapt to elf32.em->elf.em and elf-generic.em->ldelf*.c
    changes.
v5: fix tabdamage.  Drop #inclusions in .h files: include in .c files,
    .em files, and use struct forwards instead.  Use bfd_section_is_ctf
    inline function rather than SECTION_IS_CTF macro.  Move a few
    comments.

	* Makefile.def (dependencies): all-ld depends on all-libctf.
	* Makefile.in: Regenerated.

include/
	* bfdlink.h (elf_strtab_hash): New forward.
	(elf_sym_strtab): Likewise.
	(struct bfd_link_callbacks <examine_strtab>): New.
	(struct bfd_link_callbacks <emit_ctf>): Likewise.

bfd/
	* elf-bfd.h (bfd_section_is_ctf): New inline function.
	* elf.c (special_sections_c): Add ".ctf".
	(assign_file_positions_for_non_load_sections): Note that
	compressed debugging sections etc are not assigned here.  Treat
	CTF sections like SEC_ELF_COMPRESS sections when is_linker_output:
	sh_offset -1.
	(assign_file_positions_except_relocs): Likewise.
	(find_section_in_list): Note that debugging and CTF sections, as
	well as reloc sections, are assigned later.
	(_bfd_elf_assign_file_positions_for_non_load): CTF sections get
	their size and contents updated.
	(_bfd_elf_set_section_contents): Skip CTF sections: unlike
	compressed sections, they have no uncompressed content to copy at
	this stage.
	* elflink.c (elf_link_swap_symbols_out): Call the examine_strtab
	callback right before the strtab is written out.
	(bfd_elf_final_link): Don't cache the section contents of CTF
	sections: they are not populated yet.  Call the emit_ctf callback
	right at the end, after all the symbols and strings are flushed
	out.

ld/
	* ldlang.h: (struct lang_input_statement_struct): Add the_ctf.
	(struct elf_sym_strtab): Add forward.
	(struct elf_strtab_hash): Likewise.
	(ldlang_ctf_apply_strsym): Declare.
	(ldlang_write_ctf_late): Likewise.
	* ldemul.h (ldemul_emit_ctf_early): New.
	(ldemul_examine_strtab_for_ctf): Likewise.
	(ld_emulation_xfer_type) <emit_ctf_early>: Likewise.
	(ld_emulation_xfer_type) <examine_strtab_for_ctf>: Likewise.
	* ldemul.c (ldemul_emit_ctf_early): New.
	(ldemul_examine_strtab_for_ctf): Likewise.
	* ldlang.c: Include ctf-api.h.
	(CTF_COMPRESSION_THRESHOLD): New.
	(ctf_output): New. Initialized in...
	(ldlang_open_ctf): ... this new function.  Open all the CTF
	sections in the input files: mark them non-loaded and empty
	so as not to copy their contents to the output, but linker-created
	so the section gets created in the target.
	(ldlang_merge_ctf): New, merge types via ctf_link_add_ctf and
	ctf_link.
	(ldlang_ctf_apply_strsym): New, an examine_strtab callback: wrap
	ldemul_examine_strtab_for_ctf.
	(lang_write_ctf): New, write out the CTF section.
	(ldlang_write_ctf_late): New, late call via bfd's emit_ctf hook.
	(lang_process): Call ldlang_open_ctf, ldlang_merge_ctf, and
	lang_write_ctf.
	* ldmain.c (link_callbacks): Add ldlang_ctf_apply_strsym,
	ldlang_write_ctf_late.
	* emultempl/aix.em: Add ctf-api.h.
	* emultempl/armcoff.em: Likewise.
	* emultempl/beos.em: Likewise.
	* emultempl/elf.em: Likewise.
	* emultempl/generic.em: Likewise.
	* emultempl/linux.em: Likewise.
	* emultempl/msp430.em: Likewise.
	* emultempl/pe.em: Likewise.
	* emultempl/pep.em: Likewise.
	* emultempl/ticoff.em: Likewise.
	* emultempl/vanilla.em: Likewise.
	* ldcref.c: Likewise.
	* ldctor.c: Likewise.
	* ldelf.c: Likewise.
	* ldelfgen.c: Likewise.
	* ldemul.c: Likewise.
	* ldexp.c: Likewise.
	* ldfile.c: Likewise.
	* ldgram.c: Likewise.
	* ldlex.l: Likewise.
	* ldmain.c: Likewise.
	* ldmisc.c: Likewise.
	* ldver.c: Likewise.
	* ldwrite.c: Likewise.
	* lexsup.c: Likewise.
	* mri.c: Likewise.
	* pe-dll.c: Likewise.
	* plugin.c: Likewise.

	* ldelfgen.c (ldelf_emit_ctf_early): New.
	(ldelf_examine_strtab_for_ctf): tell libctf about the symtab and
	strtab.
	(struct ctf_strsym_iter_cb_arg): New, state to do so.
	(ldelf_ctf_strtab_iter_cb): New: tell libctf about
	each string in the strtab in turn.
	(ldelf_ctf_symbols_iter_cb): New, tell libctf
	about each symbol in the symtab in turn.
	* ldelfgen.h (struct elf_sym_strtab): Add forward.
	(struct elf_strtab_hash): Likewise.
	(struct ctf_file): Likewise.
	(ldelf_emit_ctf_early): Declare.
	(ldelf_examine_strtab_for_ctf): Likewise.
	* emultempl/elf-generic.em (LDEMUL_EMIT_CTF_EARLY): Set it.
	(LDEMUL_EXAMINE_STRTAB_FOR_CTF): Likewise.
	* emultempl/aix.em (ld_${EMULATION_NAME}_emulation): Add
	emit_ctf_early and examine_strtab_for_ctf, NULL by default.
	* emultempl/armcoff.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/beos.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/elf.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/generic.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/linux.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/msp430.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/pe.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/pep.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/ticoff.em (ld_${EMULATION_NAME}_emulation): Likewise.
	* emultempl/vanilla.em (ld_vanilla_emulation): Likewise.

	* Makefile.am: Pull in libctf (and zlib, a transitive requirement
	for compressed CTF section emission).  Pass it on to DejaGNU.
	* configure.ac: Add AM_ZLIB.
	* aclocal.m4: Added zlib.m4.
	* Makefile.in: Regenerated.
	* testsuite/ld-bootstrap/bootstrap.exp: Use it when relinking ld.
2019-10-03 17:04:56 +01:00
Nick Alcock 7e97445a5a libctf: get rid of a disruptive public include of <sys/param.h>
This hoary old header defines things like MAX that users of libctf might
perfectly reasonably define themselves.

The CTF headers do not need it: move it into libctf/ctf-impl.h instead.

include/
	* ctf-api.h (includes): No longer include <sys/param.h>.
libctf/
	* ctf-impl.h (includes): Include <sys/param.h> here.
2019-10-03 17:04:55 +01:00
Nick Alcock 49ea9b450b libctf: add CU-mapping machinery
Once the deduplicator is capable of actually detecting conflicting types
with the same name (i.e., not yet) we will place such conflicting types,
and types that depend on them, into CTF dictionaries that are the child
of the main dictionary we usually emit: currently, this will lead to the
.ctf section becoming a CTF archive rather than a single dictionary,
with the default-named archive member (_CTF_SECTION, or NULL) being the
main shared dictionary with most of the types in it.

By default, the sections are named after the compilation unit they come
from (complete path and all), with the cuname field in the CTF header
providing further evidence of the name without requiring the caller to
engage in tiresome parsing.  But some callers may not wish the mapping
from input CU to output sub-dictionary to be purely CU-based.

The machinery here allows this to be freely changed, in two ways:

 - callers can call ctf_link_add_cu_mapping to specify that a single
   input compilation unit should have its types placed in some other CU
   if they conflict: the CU will always be created, even if empty, so
   the consuming program can depend on its existence.  You can map
   multiple input CUs to one output CU to force all their types to be
   merged together: if some of *those* types conflict, the behaviour is
   currently unspecified (the new deduplicator will specify it).

 - callers can call ctf_link_set_memb_name_changer to provide a function
   which is passed every CTF sub-dictionary name in turn (including
   _CTF_SECTION) and can return a new name, or NULL if no change is
   desired.  The mapping from input to output names should not map two
   input names to the same output name: if this happens, the two are not
   merged but will result in an archive with two members with the same
   name (technically valid, but it's hard to access the second
   same-named member: you have to do an iteration over archive members).

This is used by the kernel's ctfarchive machinery (not yet upstream) to
encode CTF under member names like {module name}.ctf rather than
.ctf.CU, but it is anticipated that other large projects may wish to
have their own storage for CTF outside of .ctf sections and may wish to
have new naming schemes that suit their special-purpose consumers.

New in v3.
v4: check for strdup failure.
v5: fix tabdamage.

include/
	* ctf-api.h (ctf_link_add_cu_mapping): New.
	(ctf_link_memb_name_changer_f): New.
	(ctf_link_set_memb_name_changer): New.

libctf/
	* ctf-impl.h (ctf_file_t) <ctf_link_cu_mappping>: New.
	<ctf_link_memb_name_changer>: Likewise.
	<ctf_link_memb_name_changer_arg>: Likewise.
	* ctf-create.c (ctf_update): Update accordingly.
	* ctf-open.c (ctf_file_close): Likewise.
	* ctf-link.c (ctf_create_per_cu): Apply the cu mapping.
	(ctf_link_add_cu_mapping): New.
	(ctf_link_set_memb_name_changer): Likewise.
	(ctf_change_parent_name): New.
	(ctf_name_list_accum_cb_arg_t) <dynames>: New, storage for names
	allocated by the caller's ctf_link_memb_name_changer.
	<ndynames>: Likewise.
	(ctf_accumulate_archive_names): Call the ctf_link_memb_name_changer.
	(ctf_link_write): Likewise (for _CTF_SECTION only): also call
	ctf_change_parent_name.  Free any resulting names.
2019-10-03 17:04:55 +01:00
Nick Alcock eabb7154df libctf: add linking of the variable section
The compiler describes the name and type of all file-scope variables in
this section.  Merging it at link time requires using the type mapping
added in the previous commit to determine the appropriate type for the
variable in the output, given its type in the input: we check the shared
container first, and if the type doesn't exist there, it must be a
conflicted type in the per-CU child, and the variable should go there
too.  We also put the variable in the per-CU child if a variable with
the same name but a different type already exists in the parent: we
ignore any such conflict in the child because CTF cannot represent such
things, nor can they happen unless a third-party linking program has
overridden the mapping of CU to CTF archive member name (using machinery
added in a later commit).

v3: rewritten using an algorithm that actually works in the case of
    conflicting names.  Some code motion from the next commit.  Set
    the per-CU parent name.
v4: check for strdup failure.
v5: fix tabdamage.

include/
	* ctf-api.h (ECTF_INTERNAL): New.

libctf/
	* ctf-link.c (ctf_create_per_cu): New, refactored out of...
	(ctf_link_one_type): ... here, with parent-name setting added.
	(check_variable): New.
	(ctf_link_one_variable): Likewise.
	(ctf_link_one_input_archive_member): Call it.
	* ctf-error.c (_ctf_errlist): Updated with new errors.
2019-10-03 17:04:55 +01:00
Nick Alcock 72c83edd92 libctf: add the ctf_link machinery
This is the start of work on the core of the linking mechanism for CTF
sections.  This commit handles the type and string sections.

The linker calls these functions in sequence:

ctf_link_add_ctf: to add each CTF section in the input in turn to a
  newly-created ctf_file_t (which will appear in the output, and which
  itself will become the shared parent that contains types that all
  TUs have in common (in all link modes) and all types that do not
  have conflicting definitions between types (by default).  Input files
  that are themselves products of ld -r are supported, though this is
  not heavily tested yet.

ctf_link: called once all input files are added to merge the types in
  all the input containers into the output container, eliminating
  duplicates.

ctf_link_add_strtab: called once the ELF string table is finalized and
  all its offsets are known, this calls a callback provided by the
  linker which returns the string content and offset of every string in
  the ELF strtab in turn: all these strings which appear in the input
  CTF strtab are eliminated from it in favour of the ELF strtab:
  equally, any strings that only appear in the input strtab will
  reappear in the internal CTF strtab of the output.

ctf_link_shuffle_syms (not yet implemented): called once the ELF symtab
  is finalized, this calls a callback provided by the linker which
  returns information on every symbol in turn as a ctf_link_sym_t.  This
  is then used to shuffle the function info and data object sections in
  the CTF section into symbol table order, eliminating the index
  sections which map those sections to symbol names before that point.
  Currently just returns ECTF_NOTYET.

ctf_link_write: Returns a buffer containing either a serialized
  ctf_file_t (if there are no types with conflicting definitions in the
  object files in the link) or a ctf_archive_t containing a large
  ctf_file_t (the common types) and a bunch of small ones named after
  individual CUs in which conflicting types are found (containing the
  conflicting types, and all types that reference them).  A threshold
  size above which compression takes place is passed as one parameter.
  (Currently, only gzip compression is supported, but I hope to add lzma
  as well.)

Lifetime rules for this are simple: don't close the input CTF files
until you've called ctf_link for the last time.  We do not assume
that symbols or strings passed in by the callback outlast the
call to ctf_link_add_strtab or ctf_link_shuffle_syms.

Right now, the duplicate elimination mechanism is the one already
present as part of the ctf_add_type function, and is not particularly
good: it misses numerous actual duplicates, and the conflicting-types
detection hardly ever reports that types conflict, even when they do
(one of them just tends to get silently dropped): it is also very slow.
This will all be fixed in the next few weeks, but the fix hardly touches
any of this code, and the linker does work without it, just not as
well as it otherwise might.  (And when no CTF section is present,
there is no effect on performance, of course.  So only people using
a trunk GCC with not-yet-committed patches will even notice.  By the
time it gets upstream, things should be better.)

v3: Fix error handling.
v4: check for strdup failure.
v5: fix tabdamage.

include/
	* ctf-api.h (struct ctf_link_sym): New, a symbol in flight to the
	libctf linking machinery.
	(CTF_LINK_SHARE_UNCONFLICTED): New.
	(CTF_LINK_SHARE_DUPLICATED): New.
	(ECTF_LINKADDEDLATE): New, replacing ECTF_UNUSED.
	(ECTF_NOTYET): New, a 'not yet implemented' message.
	(ctf_link_add_ctf): New, add an input file's CTF to the link.
	(ctf_link): New, merge the type and string sections.
	(ctf_link_strtab_string_f): New, callback for feeding strtab info.
	(ctf_link_iter_symbol_f): New, callback for feeding symtab info.
	(ctf_link_add_strtab): New, tell the CTF linker about the ELF
	strtab's strings.
	(ctf_link_shuffle_syms): New, ask the CTF linker to shuffle its
	symbols into symtab order.
	(ctf_link_write): New, ask the CTF linker to write the CTF out.

libctf/
	* ctf-link.c: New file, linking of the string and type sections.
	* Makefile.am (libctf_a_SOURCES): Add it.
	* Makefile.in: Regenerate.

	* ctf-impl.h (ctf_file_t): New fields ctf_link_inputs,
	ctf_link_outputs.
	* ctf-create.c (ctf_update): Update accordingly.
	* ctf-open.c (ctf_file_close): Likewise.
	* ctf-error.c (_ctf_errlist): Updated with new errors.
2019-10-03 17:04:55 +01:00
Nick Alcock 5537f9b9a3 libctf: write CTF files to memory, and CTF archives to fds
Before now, we've been able to write CTF files to gzFile descriptors or
fds, and CTF archives to named files only.

Make this a bit less irregular by allowing CTF archives to be written
to fds with the new function ctf_arc_write_fd: also allow CTF
files to be written to a new memory buffer via ctf_write_mem.

(It would be nice to complete things by adding a new function to write
CTF archives to memory, but this is too difficult to do given the short
time the linker is expected to be writing them out: we will transition
to a better format in format v4, though we will always support reading
CTF archives that are stored in .ctf sections.)

include/
	* ctf-api.h (ctf_arc_write_fd): New.
	(ctf_write_mem): Likewise.
	(ctf_gzwrite): Spacing fix.

libctf/
	* ctf-archive.c (ctf_arc_write): Split off, and reimplement in terms
	of...
	(ctf_arc_write_fd): ... this new function.
	* ctf-create.c (ctf_write_mem): New.
2019-10-03 17:04:55 +01:00
Nick Alcock d851ecd373 libctf: support getting strings from the ELF strtab
The CTF file format has always supported "external strtabs", which
internally are strtab offsets with their MSB on: such refs
get their strings from the strtab passed in at CTF file open time:
this is usually intended to be the ELF strtab, and that's what this
implementation is meant to support, though in theory the external
strtab could come from anywhere.

This commit adds support for these external strings in the ctf-string.c
strtab tracking layer.  It's quite easy: we just add a field csa_offset
to the atoms table that tracks all strings: this field tracks the offset
of the string in the ELF strtab (with its MSB already on, courtesy of a
new macro CTF_SET_STID), and adds a new function that sets the
csa_offset to the specified offset (plus MSB).  Then we just need to
avoid writing out strings to the internal strtab if they have csa_offset
set, and note that the internal strtab is shorter than it might
otherwise be.

(We could in theory save a little more time here by eschewing sorting
such strings, since we never actually write the strings out anywhere,
but that would mean storing them separately and it's just not worth the
complexity cost until profiling shows it's worth doing.)

We also have to go through a bit of extra effort at variable-sorting
time.  This was previously using direct references to the internal
strtab: it couldn't use ctf_strptr or ctf_strraw because the new strtab
is not yet ready to put in its usual field (in a ctf_file_t that hasn't
even been allocated yet at this stage): but now we're using the external
strtab, this will no longer do because it'll be looking things up in the
wrong strtab, with disastrous results.  Instead, pass the new internal
strtab in to a new ctf_strraw_explicit function which is just like
ctf_strraw except you can specify a ne winternal strtab to use.

But even now that it is using a new internal strtab, this is not quite
enough: it can't look up strings in the external strtab because ld
hasn't written it out yet, and when it does will write it straight to
disk.  Instead, when we write the internal strtab, note all the offset
-> string mappings that we have noted belong in the *external* strtab to
a new "synthetic external strtab" dynhash, ctf_syn_ext_strtab, and look
in there at ctf_strraw time if it is set.  This uses minimal extra
memory (because only strings in the external strtab that we actually use
are stored, and even those come straight out of the atoms table), but
let both variable sorting and name interning when ctf_bufopen is next
called work fine.  (This also means that we don't need to filter out
spurious ECTF_STRTAB warnings from ctf_bufopen but can pass them back to
the caller, once we wrap ctf_bufopen so that we have a new internal
variant of ctf_bufopen etc that we can pass the synthetic external
strtab to. That error has been filtered out since the days of Solaris
libctf, which didn't try to handle the problem of getting external
strtabs right at construction time at all.)

v3: add the synthetic strtab and all associated machinery.
v5: fix tabdamage.

include/
	* ctf.h (CTF_SET_STID): New.

libctf/
	* ctf-impl.h (ctf_str_atom_t) <csa_offset>: New field.
	(ctf_file_t) <ctf_syn_ext_strtab>: Likewise.
	(ctf_str_add_ref): Name the last arg.
	(ctf_str_add_external) New.
	(ctf_str_add_strraw_explicit): Likewise.
	(ctf_simple_open_internal): Likewise.
	(ctf_bufopen_internal): Likewise.

	* ctf-string.c (ctf_strraw_explicit): Split from...
	(ctf_strraw): ... here, with new support for ctf_syn_ext_strtab.
	(ctf_str_add_ref_internal): Return the atom, not the
	string.
	(ctf_str_add): Adjust accordingly.
	(ctf_str_add_ref): Likewise.  Move up in the file.
	(ctf_str_add_external): New: update the csa_offset.
	(ctf_str_count_strtab): Only account for strings with no csa_offset
	in the internal strtab length.
	(ctf_str_write_strtab): If the csa_offset is set, update the
	string's refs without writing the string out, and update the
	ctf_syn_ext_strtab.  Make OOM handling less ugly.
	* ctf-create.c (struct ctf_sort_var_arg_cb): New.
	(ctf_update): Handle failure to populate the strtab.  Pass in the
	new ctf_sort_var arg.  Adjust for ctf_syn_ext_strtab addition.
	Call ctf_simple_open_internal, not ctf_simple_open.
	(ctf_sort_var): Call ctf_strraw_explicit rather than looking up
	strings by hand.
	* ctf-hash.c (ctf_hash_insert_type): Likewise (but using
	ctf_strraw).  Adjust to diagnose ECTF_STRTAB nonetheless.
	* ctf-open.c (init_types): No longer filter out ECTF_STRTAB.
	(ctf_file_close): Destroy the ctf_syn_ext_strtab.
	(ctf_simple_open): Rename to, and reimplement as a wrapper around...
	(ctf_simple_open_internal): ... this new function, which calls
	ctf_bufopen_internal.
	(ctf_bufopen): Rename to, and reimplement as a wrapper around...
	(ctf_bufopen_internal): ... this new function, which sets
	ctf_syn_ext_strtab.
2019-10-03 17:04:55 +01:00
Nick Alcock 0ac6231298 libctf: Add iteration over non-root types
The existing function ctf_type_iter lets you iterate over root-visible
types (types you can look up by name).  There is no way to iterate over
non-root-visible types, which is troublesome because both the linker
and dumper want to do that.

So add a new function that can do it: the callback it takes accepts
an extra parameter which indicates whether the type is root-visible
or not.

include/
	* ctf-api.h (ctf_type_all_f): New.
	(ctf_type_iter_all): New.

libctf/
	* ctf_types.c (ctf_type_iter_all): New.
2019-10-03 17:04:55 +01:00
Nick Alcock 2db912ba1a libctf: add the object index and function index sections
No code handles these yet, but our latest GCC patches are generating
them, so we have to be ready for them or erroneously conclude that we
have file corruption.

(This simultaneously fixes a longstanding bug, concealed because nothing
was generating anything in the object or function info sections, where
the end of the section was being tested against the wrong thing: it
would have walked over the entire contents of the variable section and
treated them as part of the function info section.  This had to change
now anyway because the new sections have landed in between.)

include/
	* ctf.h: Add object index and function index sections.  Describe
	them. Improve the description of the variable section and clarify
	the constraints on backward-pointing type nodes.
	(ctf_header): Add cth_objtidxoff, cth_funcidxoff.

libctf/
	* ctf-open.c (init_symtab): Check for overflow against the right
	section.
	(upgrade_header): Set cth_objtidxoff, cth_funcidxoff to zero-length.
	(upgrade_types_v1): Note that these sections are not checked.
	(flip_header): Endian-swap the header fields.
	(flip_ctf): Endian-swap the sections.
	(flip_objts): Update comment.
	(ctf_bufopen): Check header offsets and alignment for validity.
2019-10-03 17:04:55 +01:00
Nick Alcock fd55eae84d libctf: allow the header to change between versions
libctf supports dynamic upgrading of the type table as file format
versions change, but before now has not supported changes to the CTF
header.  Doing this is complicated by the baroque storage method used:
the CTF header is kept prepended to the rest of the CTF data, just as
when read from the file, and written out from there, and is
endian-flipped in place.

This makes accessing it needlessly hard and makes it almost impossible
to make the header larger if we add fields.  The general storage
machinery around the malloced ctf pointer (the 'ctf_base') is also
overcomplicated: the pointer is sometimes malloced locally and sometimes
assigned from a parameter, so freeing it requires checking to see if
that parameter was used, needlessly coupling ctf_bufopen and
ctf_file_close together.

So split the header out into a new ctf_file_t.ctf_header, which is
written out explicitly: squeeze it out of the CTF buffer whenever we
reallocate it, and use ctf_file_t.ctf_buf to skip past the header when
we do not need to reallocate (when no upgrading or endian-flipping is
required).  We now track whether the CTF base can be freed explicitly
via a new ctf_dynbase pointer which is non-NULL only when freeing is
possible.

With all this done, we can upgrade the header on the fly and add new
fields as desired, via a new upgrade_header function in ctf-open.
As with other forms of upgrading, libctf upgrades older headers
automatically to the latest supported version at open time.

For a first use of this field, we add a new string field cth_cuname, and
a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this
is used by debuggers to determine whether a CTF section's types relate
to a single compilation unit, or to all compilation units in the
program.  (Types with ambiguous definitions in different CUs have only
one of these types placed in the top-level shared .ctf container: the
rest are placed in much smaller per-CU containers, which have the shared
container as their parent.  Since CTF must be useful in the absence of
DWARF, we store the names of the relevant CUs ourselves, so the debugger
can look them up.)

v5: fix tabdamage.

include/
	* ctf-api.h (ctf_cuname): New function.
	(ctf_cuname_set): Likewise.
	* ctf.h: Improve comment around upgrading, no longer
	implying that v2 is the target of upgrades (it is v3 now).
	(ctf_header_v2_t): New, old-format header for backward
	compatibility.
	(ctf_header_t): Add cth_cuname: this is the first of several
	header changes in format v3.
libctf/
	* ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase,
	ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const.
	* ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and
	ctf_base: do not assume that it is always sizeof (ctf_header_t).
	Print out ctf_cuname: only print out ctf_parname if set.
	(ctf_free_base): Removed, ctf_base is no longer freed: free
	ctf_dynbase instead.
	(ctf_set_version): Fix spacing.
	(upgrade_header): New, in-place header upgrading.
	(upgrade_types): Rename to...
	(upgrade_types_v1): ... this.  Free ctf_dynbase, not ctf_base.  No
	longer track old and new headers separately.  No longer allow for
	header sizes explicitly: squeeze the headers out on upgrade (they
	are preserved in fp->ctf_header).  Set ctf_dynbase, ctf_base and
	ctf_buf explicitly.  Use ctf_free, not ctf_free_base.
	(upgrade_types): New, also handle ctf_parmax updating.
	(flip_header): Flip ctf_cuname.
	(flip_types): Flip BUF explicitly rather than deriving BUF from
	BASE.
	(ctf_bufopen): Store the header in fp->ctf_header.  Correct minimum
	required alignment of objtoff and funcoff.  No longer store it in
	the ctf_buf unless that buf is derived unmodified from the input.
	Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals
	that duplicate fields in ctf_file: move allocation of ctf_file
	further up instead.  Call upgrade_header as needed.  Move
	version-specific ctf_parmax initialization into upgrade_types.  More
	concise error handling.
	(ctf_file_close): No longer test for null pointers before freeing.
	Free ctf_dyncuname, ctf_dynbase, and ctf_header.  Do not call
	ctf_free_base.
	(ctf_cuname): New.
	(ctf_cuname_set): New.
	* ctf-create.c (ctf_update): Populate ctf_cuname.
	(ctf_gzwrite): Write out the header explicitly.  Remove obsolescent
	comment.
	(ctf_write): Likewise.
	(ctf_compress_write): Get the header from ctf_header, not ctf_base.
	Fix the compression length: fp->ctf_size never counted the CTF
	header.  Simplify the compress call accordingly.
2019-10-03 17:04:55 +01:00
Nick Alcock 083114f8ba libctf, include: ChangeLog format fixes
Double-spaces before email addresses were consistently missing.
2019-10-03 17:04:55 +01:00
Alan Modra 640197ac27 linker bfd.h tidy
bfd/
	* bfd-in.h (bfd_symbol, bfd_section_already_linked),
	(bfd_elf_version_tree): Delete forward declarations.
	Move other forward decls and remaining elf function decl later.
	(bfd_section_already_linked_table_init),
	(bfd_section_already_linked_table_free),
	(_bfd_handle_already_linked, _bfd_nearby_section),
	(_bfd_fix_excluded_sec_syms): Move to bfdlink.h.
include/
	* bfdlink.h (struct bfd_section_already_linked): Forward declare.
	(bfd_section_already_linked_table_init),
	(bfd_section_already_linked_table_free),
	(_bfd_handle_already_linked, _bfd_nearby_section),
	(_bfd_fix_excluded_sec_syms): Declare.
2019-09-23 10:27:22 +09:30
Alan Modra 0b4453c791 elf bfd.h tidy
bfd/
	* bfd-in.h (enum notice_asneeded_action): Move to bfdlink.h.
	Move most other elf declarations..
	* elf-bfd.h: ..to here.
	* bfd-in2.h: Regenerate.
include/
	* bfdlink.h (enum notice_asneeded_action): Define.
ld/
	* deffilep.y: Include bfdlink.h.
	* ldelf.c: Likewise.
	* ldelfgen.c: Likewise.
	* ldver.c: Likewise.
	* mri.c: Likewise.
	* emultempl/irix.em: Don't include ld.h, ldmain.h, libiberty.h.
	Comment.
2019-09-23 10:27:22 +09:30
Jim Wilson 7e9ad3a35c RISC-V: Gate opcode tables by enum rather than string.
Generalize opcode arch dependencies so that we can support the
overlapping B extension Zb* subsets.

	2019-09-17  Maxim Blinov  <maxim.blinov@embecosm.com>
	gas/
	* config/tc-riscv.c (riscv_multi_subset_supports): Handle
	insn_class enum rather than subset char string.
	(riscv_ip): Update call to riscv_multi_subset_supports.
	include/
	* opcode/riscv.h (riscv_insn_class): New enum.
	* opcode/riscv.h (struct riscv_opcode): Change
	subset field to insn_class field.
	opcodes/
	* riscv-opc.c (riscv_opcodes): Change subset field
	to insn_class field for all instructions.
	(riscv_insn_types): Likewise.
2019-09-17 17:59:08 -07:00
Phil Blundell 60391a255b Add markers for 2.33 branch to NEWS and ChangeLog files. 2019-09-09 10:27:40 +01:00
Claudiu Zissulescu cd18a823d1 [ARC] [COMMITTED] Fix FASTMATH field.
Move FASTMATH to the right enum.

2019-08-30  Claudiu Zissulescu <claziss@gmail.com>

        * opcode/arc.h (FASTMATH): Move it from insn_class_t to
        insn_subclass_t enum.
2019-08-30 11:31:19 +03:00
Alan Modra baf46cd780 ARM CMSE symbols
This patch removes use of st_target_internal to cache the result of
comparing symbol names against CMSE_PREFIX.  The problem with setting
a bit in st_target_internal in swap_symbol_in is that calling
bfd_elf_sym_name from swap_symbol_in requires symtab_hdr, and you
don't know for sure whether swap_symbol_in is operating on dynsyms
(and thus elf_tdata (abfd)->dynsymtab_hdr should be used) or on the
normal symtab (thus elf_tdata (abfd)->symtab_hdr).  You can make an
educated guess based on abfd->flags & DYNAMIC but that relies on
knowing a lot about calls to bfd_elf_get_elf_syms, and is fragile in
the face of possible future changes.

include/
	* elf/arm.h (ARM_GET_SYM_CMSE_SPCL, ARM_SET_SYM_CMSE_SPCL): Delete.
bfd/
	* elf32-arm.c (cmse_scan): Don't use ARM_GET_SYM_CMSE_SPCL,
	instead recognize CMSE_PREFIX in symbol name.
	(elf32_arm_gc_mark_extra_sections): Likewise.
	(elf32_arm_filter_cmse_symbols): Don't test ARM_GET_SYM_CMSE_SPCL.
	(elf32_arm_swap_symbol_in): Don't invoke ARM_SET_SYM_CMSE_SPCL.
2019-08-22 14:14:23 +09:30
Mihailo Stojanovic f16a9783c5 Add support for a MIPS specific .MIPS.xhash section.
This patch is a reimplementation of [1] which was submitted in 2015 by
Neil Schellenberger. Copyright issue was sorted out [2] last year.
It proposed a new section (.gnu.xhash) and related dynamic tag
(DT_GNU_XHASH). The new section would be virtually identical to the
existing .gnu.hash except for the translation table (xlat) which would
contain correct MIPS .dynsym indexes corresponding to the hashvals in
chains. This is because MIPS ABI imposes a different ordering on the
dynsyms than the one expected by the .gnu.hash section. Another addition
would be a leading word (ngnusyms) which would contain the number of
entries in the translation table.

  In this patch, the new section name and dynamic tag are changed to
reflect the fact that the section should be treated as MIPS-specific
(.MIPS.xhash and DT_MIPS_XHASH).

  This patch addresses the alignment issue as reported in [3], which is
caused by the leading word added to the .MIPS.xhash section. Leading word
is removed in this patch, and the number of entries in the translation
table is now calculated using DT_MIPS_SYMTABNO dynamic tag (this is
addressed by the corresponding glibc patch).

  Suggestions on coding style in [4] were taken into account. Existing
GNU hash testcase was covered, and another one was added in the MIPS
part of the testsuite.

  The other major change is reserving MIPS ABI version 5 for .MIPS.xhash,
marking the need of support for .MIPS.xhash in the dynamic linker (again,
addressed in the corresponding glibc patch). This is something which I
am not sure of, especially after reading [5]. I am confused on whether
this ABI version is reserved for IFUNC, or it can be used for this
purpose.

Already mentioned glibc patch is submitted at:
https://sourceware.org/ml/libc-alpha/2019-06/msg00456.html

[1] https://sourceware.org/ml/binutils/2015-10/msg00057.html
[2] https://sourceware.org/ml/binutils/2018-03/msg00025.html
[3] https://sourceware.org/ml/binutils/2016-01/msg00006.html
[4] https://sourceware.org/ml/binutils/2016-02/msg00097.html
[5] https://sourceware.org/ml/libc-alpha/2016-12/msg00853.html

ld      * emulparams/elf32bmip.sh: Add .MIPS.xhash section.
        * emulparams/elf32bmipn32-defs.sh: Add .MIPS.xhash section.
        * emulparams/elf64bmip-defs.sh: Add .MIPS.xhash section.
        * emultempl/mipself.em: Remove mips_after_parse function.
        * testsuite/ld-elf/hash.d: Update comment.
        * testsuite/ld-mips-elf/hash1.d: New test.
        * testsuite/ld-mips-elf/hash1.s: Ditto.
        * testsuite/ld-mips-elf/hash1a.d: Remove.
        * testsuite/ld-mips-elf/hash1b.d: Ditto.
        * testsuite/ld-mips-elf/hash1c.d: Ditto
        * testsuite/ld-mips-elf/hash2.d: New test.
        * testsuite/ld-mips-elf/mips-elf.exp: New tests.
        * testsuite/ld-mips-elf/start.s: New test.

bfd     * elf-bfd.h (struct elf_backend_data): New members.
        * elflink.c (_bfd_elf_link_create_dynamic_sections): Create
        .gnu.hash section if necessary.
        (struct collect_gnu_hash_codes): New member.
        (elf_gnu_hash_process_symidx): New function name.
        (elf_renumber_gnu_hash_syms): Ignore local and undefined
        symbols. Record xlat location for every symbol which should have
        a .MIPS.xhash entry.
        (bfd_elf_size_dynamic_sections): Add DT_GNU_HASH dynamic tag to
        dynamic section if necessary.
        (GNU_HASH_SECTION_NAME): New define.
        (bfd_elf_size_dynsym_hash_dynstr): Get .MIPS.xhash section.
        Update the section size info.
        * elfxx-mips.c (struct mips_elf_hash_sort_data): New members.
        (struct mips_elf_link_hash_entry): New member.
        (mips_elf_link_hash_newfunc): Initialize .MIPS.xhash translation
        table location.
        (mips_elf_sort_hash_table): Initialize the pointer to the
        .MIPS.xhash section.
        (mips_elf_sort_hash_table_f): Populate the .MIPS.xhash
        translation table entry with the symbol dynindx.
        (_bfd_mips_elf_section_from_shdr): Add SHT_MIPS_XHASH.
        (_bfd_mips_elf_fake_sections): Initialize .MIPS.xhash section
        info.
        (_bfd_mips_elf_create_dynamic_sections): Create .MIPS.xhash
        section.
        (_bfd_mips_elf_size_dynamic_sections): Add DT_MIPS_XHASH tag to
        dynamic section.
        (_bfd_mips_elf_finish_synamic_sections): Add DT_MIPS_XHASH.
        (_bfd_mips_elf_final_write_processing): Set .MIPS.xhash section
        sh_link info.
        (_bfd_mips_elf_get_target_dtag): Get DT_MIPS_XHASH tag.
        (MIPS_LIBC_ABI_XHASH): New ABI version enum value.
        (_bfd_mips_post_process_headers): Mark the ABI version as
        MIPS_LIBC_ABI_XHASH if there exists a .MIPS.xhash section,
        but not a .hash section.
        (_bfd_mips_elf_record_xhash_symbol): New function. Record a
        position in the translation table, associated with the hash
        entry.
        * elfxx-mips.h (literal_reloc_p): Define
        elf_backend_record_xhash_symbol backend hook.
        * elfxx-target.h: Initialize elf_backend_record_xhash_symbol
        backend hook.

include * elf/mips.h (SHT_GNU_XHASH): New define.
        (DT_GNU_XHASH): New define.

binutils * readelf.c (get_mips_dynamic_type): Return MIPS_XHASH dynamic type.
        (get_mips_section_type_name): Return MI{S_XHASH name string.
        (dynamic_section_mips_val): Initialize the .MIPS.xhash dynamic
        info.
        (process_symbol_table): Initialize the .MIPS.xhash section
        pointer. Adjust the readelf output to support the new section.
        (process_object): Set the .MIPS.xhash dynamic info to zero.
2019-08-09 11:06:37 +01:00
Yoshinori Sato 7b1dd70105 Update the handling of shift rotate and load/store multiple instructions in the H8300 assembler.
incldue	* opcode/h8300.h (EXPAND_UNOP_EXTENDED_B): Add MODEL.
	(EXPAND_UNOP_EXTENDED_W): Likewise.
	(EXPAND_UNOP_EXTENDED_L): Likewise.
	(struct h8_opcode h8_opcodes): Likewise.
	(struct h8_opcode h8_opcodes): tas / ldm / stm supported H8S or
	later.
2019-08-08 13:59:17 +01:00
Claudiu Zissulescu 7e126ba31a [ARC] Update ARC opcode table
Update ARC opcode table by cleaning up invalid instructions, and fixing wrong encodings.

opcodes/
xxxx-xx-xx  Claudiu Zissulescu  <claziss@synopsys.com>

	* arc-ext-tbl.h (bspeek): Remove it, added to main table.
	(bspop): Likewise.
	(modapp): Likewise.
	* arc-opc.c (RAD_CHK): Add.
	* arc-tbl.h: Regenerate.

include/
xxxx-xx-xx  Claudiu Zissulescu  <claziss@synopsys.com>

	* include/opcode/arc.h (FASTMATH): Add.
	(SWITCH): Likewise.
2019-07-24 16:46:01 +03:00
Alan Modra 7137a1ed0a PT_GNU_MBIND section mapping
* elf/internal.h (ELF_SECTION_IN_SEGMENT_1): Exclude non-alloc
	sections in GNU_MBIND segments.
2019-07-24 16:12:01 +09:30
Omar Majid 1dd1bc4daf Allow objdump to recognise the System.Runtime.dll files that get shipped with .NET Core 2.1.
include	* coff/i386.h (IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE): Define.
	(IMAGE_FILE_MACHINE_NATIVE_FREEBSD_OVERRIDE): Define.
	(IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE): Define.
	(IMAGE_FILE_MACHINE_NATIVE_NETBSD_OVERRIDE): Define.
	(I386_APPLE_MAGIC): Define.
	(I386_FREEBSD_MAGIC): Define.
	(I386_LINUX_MAGIC): Define.
	(I386_NETBSD_MAGIC): Define.
	(I386BADMAG): Extend macro to allow new magic numbers.
	* coff/x86_64.h (IMAGE_FILE_MACHINE_NATIVE_APPLE_OVERRIDE): Define.
	(IMAGE_FILE_MACHINE_NATIVE_FREEBSD_OVERRIDE): Define.
	(IMAGE_FILE_MACHINE_NATIVE_LINUX_OVERRIDE): Define.
	(IMAGE_FILE_MACHINE_NATIVE_NETBSD_OVERRIDE): Define.
	(AMD64_APPLE_MAGIC): Define.
	(AMD64_FREEBSD_MAGIC): Define.
	(AMD64_LINUX_MAGIC): Define.
	(AMD64_NETBSD_MAGIC): Define.
	(AMD64BADMAG): Extend macro to allow new magic numbers.

bfd	* coffcode.h (coff_set_arch_mach_hook): Handle I386_APPLE_MAGIC,
	I386_FREEBSD_MAGIC, I386_LINUX_MAGIC, I386_NETBSD_MAGIC,
	AMD64_APPLE_MAGIC, AMD64_FREEBSD_MAGIC, AMD64_LINUX_MAGIC,
	AMD64_NETBSD_MAGIC.
	* peXXigen.c: Add comment about source of .NET magic numbers.

binutils* Makefile.am (AUTOMAKE_OPTIONS): Add subdir-objects
	(GENTESTDLLSPROG): Define.
	(TEST_PROGS): Add GENTESTDLLSPROG.
	* Makefile.in: Regenerate.
	* testsuite/binutils-all/objdump.exp
	(test_objdump_dotnet_assemblies): New proc.
	Run the new proc.
	* testsuite/gentestdlls.c: New source file.
2019-07-23 09:44:57 +01:00
Alan Modra c213164ad2 [PowerPC64] pc-relative TLS relocations
This patch supports using pcrel instructions in TLS code sequences.  A
number of new relocations are needed, gas operand modifiers to
generate those relocations, and new TLS optimisation.  For
optimisation it turns out that the new pcrel GD and LD sequences can
be distinguished from the non-pcrel GD and LD sequences by there being
different relocations on the new sequence.  The final "add ra,rb,13"
on IE sequences similarly needs a new relocation, or as I chose, a
modification of R_PPC64_TLS.  On pcrel IE code, the R_PPC64_TLS points
one byte into the "add" instruction rather than being on the
instruction boundary.

GD:
 pla 3,z@got@tlsgd@pcrel	  # R_PPC64_GOT_TLSGD34
 bl __tls_get_addr@notoc(z@tlsgd) # R_PPC64_TLSGD and R_PPC64_REL24_NOTOC
  edited to IE
   pld 3,z@got@tprel@pcrel
   add 3,3,13
  edited to LE
   paddi 3,13,z@tprel
   nop

LD:
 pla 3,z@got@tlsld@pcrel	  # R_PPC64_GOT_TLSLD34
 bl __tls_get_addr@notoc(z@tlsld) # R_PPC64_TLSLD and R_PPC64_REL24_NOTOC
 ..
 paddi 9,3,z2@dtprel
 pld 10,z3@got@dtprel@pcrel
 add 10,10,3
  edited to LE
   paddi 3,13,0x1000
   nop

IE:
 pld 9,z@got@tprel@pcrel	  # R_PPC64_GOT_TPREL34
 add 3,9,z@tls@pcrel		  # R_PPC64_TLS at insn+1
 ldx 4,9,z@tls@pcrel
 lwax 5,9,z@tls@pcrel
 stdx 5,9,z@tls@pcrel
  edited to LE
  paddi 9,13,z@tprel
  nop
  ld 4,0(9)
  lwa 5,0(9)
  std 5,0(9)

LE:
 paddi 10,13,z@tprel

include/
	* elf/ppc64.h (R_PPC64_TPREL34, R_PPC64_DTPREL34),
	(R_PPC64_GOT_TLSGD34, R_PPC64_GOT_TLSLD34),
	(R_PPC64_GOT_TPREL34, R_PPC64_GOT_DTPREL34): Define.
	(IS_PPC64_TLS_RELOC): Include new tls relocs.
bfd/
	* reloc.c (BFD_RELOC_PPC64_TPREL34, BFD_RELOC_PPC64_DTPREL34),
	(BFD_RELOC_PPC64_GOT_TLSGD34, BFD_RELOC_PPC64_GOT_TLSLD34),
	(BFD_RELOC_PPC64_GOT_TPREL34, BFD_RELOC_PPC64_GOT_DTPREL34),
	(BFD_RELOC_PPC64_TLS_PCREL): New pcrel tls relocs.
	* elf64-ppc.c (ppc64_elf_howto_raw): Add howtos for pcrel tls relocs.
	(ppc64_elf_reloc_type_lookup): Translate pcrel tls relocs.
	(must_be_dyn_reloc, dec_dynrel_count): Add R_PPC64_TPREL64.
	(ppc64_elf_check_relocs): Support pcrel tls relocs.
	(ppc64_elf_tls_optimize, ppc64_elf_relocate_section): Likewise.
	* bfd-in2.h: Regenerate.
	* libbfd.h: Regenerate.
gas/
	* config/tc-ppc.c (ppc_elf_suffix): Map "tls@pcrel", "got@tlsgd@pcrel",
	"got@tlsld@pcrel", "got@tprel@pcrel", and "got@dtprel@pcrel".
	(fixup_size, md_assemble): Handle pcrel tls relocs.
	(ppc_force_relocation, ppc_fix_adjustable): Likewise.
	(md_apply_fix, tc_gen_reloc): Likewise.
ld/
	* testsuite/ld-powerpc/tlsgd.d,
	* testsuite/ld-powerpc/tlsgd.s,
	* testsuite/ld-powerpc/tlsie.d,
	* testsuite/ld-powerpc/tlsie.s,
	* testsuite/ld-powerpc/tlsld.d,
	* testsuite/ld-powerpc/tlsld.s: New tests.
	* testsuite/ld-powerpc/powerpc.exp: Run them.
2019-07-19 18:01:25 +09:30
Nick Alcock 12a0b67d28 libctf: introduce ctf_func_type_{info,args}, ctf_type_aname_raw
The first two of these allow you to get function type info and args out
of the types section give a type ID: astonishingly, this was missing
from libctf before now: so even though types of kind CTF_K_FUNCTION were
supported, you couldn't find out anything about them.  (The existing
ctf_func_info and ctf_func_args only allow you to get info about
functions in the function section, i.e. given symbol table indexes, not
type IDs.)

The second of these allows you to get the raw undecorated name out of
the CTF section (strdupped for safety) without traversing subtypes to
build a full C identifier out of it.  It's useful for things that are
already tracking the type kind etc and just need an unadorned name.

include/
	* ctf-api.h (ECTF_NOTFUNC): Fix description.
	(ctf_func_type_info): New.
	(ctf_func_type_args): Likewise.
libctf/
	* ctf-types.c (ctf_type_aname_raw): New.
	(ctf_func_type_info): Likewise.
	(ctf_func_type_args): Likewise.
	* ctf-error.c (_ctf_errlist): Fix description.
2019-07-18 20:53:57 +01:00
Jan Beulich 21df382b91 x86: fold SReg{2,3}
They're the only exception to there generally being no mix of register
kinds possible in an insn operand template, and there being two bits per
operand for their representation is also quite wasteful, considering the
low number of uses.  Fold both bits and deal with the little bit of
fallout.

Also take the liberty and drop dead code trying to set REX_B: No segment
register has RegRex set on it.

Additionally I was quite surprised that PUSH/POP with the permitted
segment registers is not covered by the test cases.  Add the missing
pieces.
2019-07-16 09:30:29 +02:00
Matthew Malcomson 41be57caf3 [gas][aarch64][SVE2] Fix pmull{t,b} requirement on SVE2-AES
I had mistakenly given all variants of the new SVE2 instructions
pmull{t,b} a dependency on the feature +sve2-aes.

Only the variant specifying .Q -> .D  sizes should have that
restriction.

This patch fixes that mistake and updates the testsuite to have extra
tests (matching the given set of tests per line in aarch64-tbl.h that
the rest of the SVE2 tests follow).

We also add a line in the documentation of the command line to clarify
how to enable `pmull{t,b}` of this larger size.  This is needed because
all other instructions gated under the `sve2-aes` architecture extension
are marked in the instruction documentation by an `HaveSVE2AES` check
while pmull{t,b} is gated under the `HaveSVE2PMULL128` check.

Regtested targeting aarch64-linux.

gas/ChangeLog:

2019-07-01  Matthew Malcomson  <matthew.malcomson@arm.com>

	* testsuite/gas/aarch64/illegal-sve2-aes.d: Update tests.
	* testsuite/gas/aarch64/illegal-sve2.l: Update tests.
	* doc/c-aarch64.texi: Add special note of pmull{t,b}
	instructions under the sve2-aes architecture extension.
	* testsuite/gas/aarch64/illegal-sve2.s: Add small size
	pmull{t,b} instructions.
	* testsuite/gas/aarch64/sve2.d: Add small size pmull{t,b}
	disassembly.
	* testsuite/gas/aarch64/sve2.s: Add small size pmull{t,b}
	instructions.

include/ChangeLog:

2019-07-01  Matthew Malcomson  <matthew.malcomson@arm.com>

	* opcode/aarch64.h (enum aarch64_insn_class): sve_size_013
	renamed to sve_size_13.

opcodes/ChangeLog:

2019-07-01  Matthew Malcomson  <matthew.malcomson@arm.com>

	* aarch64-asm.c (aarch64_encode_variant_using_iclass): Use new
	sve_size_13 icode to account for variant behaviour of
	pmull{t,b}.
	* aarch64-dis-2.c: Regenerate.
	* aarch64-dis.c (aarch64_decode_variant_using_iclass): Use new
	sve_size_13 icode to account for variant behaviour of
	pmull{t,b}.
	* aarch64-tbl.h (OP_SVE_VVV_HD_BS): Add new qualifier.
	(OP_SVE_VVV_Q_D): Add new qualifier.
	(OP_SVE_VVV_QHD_DBS): Remove now unused qualifier.
	(struct aarch64_opcode): Split pmull{t,b} into those requiring
	AES and those not.
2019-07-01 15:17:22 +01:00
Nick Alcock 7cee18263c libctf: endianness fixes
Testing of the first code to generate CTF_K_SLICEs on big-endian
revealed a bunch of new problems in this area.  Most importantly, the
trick we did earlier to avoid wasting two bytes on padding in the
ctf_slice_t is best avoided: because it leads to the whole file after
that point no longer being naturally aligned, all multibyte accesses
from then on must use memmove() to avoid unaligned access on platforms
where that is fatal.  In future, this is planned, but for now we are
still doing direct access in many places, so we must revert to making
ctf_slice_t properly aligned for storage in an array.

Rather than wasting bytes on padding, we boost the size of cts_offset
and cts_bits.  This is still a waste of space (we cannot have offsets or
bits in bitfields > 256) but it cannot be avoided for now, and slices
are not so common that this will be a serious problem.

A possibly-worse endianness problem fixed at the same time involves
a codepath used only for foreign-endian, uncompressed CTF files, where
we were not copying the actual CTF data into the buffer, leading to
libctf reading only zeroes (or, possibly, uninitialized garbage).

Finally, when we read in a CTF file, we copy the header and work from
the copy.  We were flipping the endianness of the header copy, and of
the body of the file buffer, but not of the header in the file buffer
itself: so if we write the file back out again we end up with an
unreadable frankenfile with header and body of different endiannesses.
Fix by flipping both copies of the header.

include/
	* ctf.h (ctf_slice_t): Make cts_offset and cts_bits unsigned
	short, so following structures are properly aligned.

libctf/
	* ctf-open.c (get_vbytes_common): Return the new slice size.
	(ctf_bufopen): Flip the endianness of the CTF-section header copy.
	Remember to copy in the CTF data when opening an uncompressed
	foreign-endian CTF file.  Prune useless variable manipulation.
2019-06-21 13:04:02 +01:00
Szabolcs Nagy e30d1fa1bf Add R_AARCH64_P32_MOVW_PREL_* ELF32 relocs
These ilp32 relocations were missing for some reason.

bfd/ChangeLog:

	* elfnn-aarch64.c: Enable MOVW_PREL relocs for ELF32.

include/ChangeLog:

	* elf/aarch64.h (R_AARCH64_P32_MOVW_PREL_G0): Define.
	(R_AARCH64_P32_MOVW_PREL_G0_NC): Define.
	(R_AARCH64_P32_MOVW_PREL_G1): Define.

ld/ChangeLog:

	* testsuite/ld-aarch64/aarch64-elf.exp: Add emit-relocs-22 and -23.
	* testsuite/ld-aarch64/emit-relocs-22.d: New test.
	* testsuite/ld-aarch64/emit-relocs-22.s: New test.
	* testsuite/ld-aarch64/emit-relocs-23.d: New test.
	* testsuite/ld-aarch64/emit-relocs-23.s: New test.
2019-06-14 12:02:19 +01:00
Nick Alcock a610aa4f9c libctf: fix the type of ctf_enum.cte_value
This stops the file format from depending on the size of the host int.
(It does mean that we cannot encode enums with a value > 2^32 on
platforms with an int > 2^32: this will be fixed in the next format
revision.)

include/
	* ctf.h (ctf_enum.cte_value): Fix type to int32_t.
2019-06-04 17:05:08 +01:00
Jose E. Marchesi a0486bac41 libctf: fix a number of build problems found on Solaris and NetBSD
- Use of nonportable <endian.h>
- Use of qsort_r
- Use of zlib without appropriate magic to pull in the binutils zlib
- Use of off64_t without checking (fixed by dropping the unused fields
  that need off64_t entirely)
- signedness problems due to long being too short a type on 32-bit
  platforms: ctf_id_t is now 'unsigned long', and CTF_ERR must be
  used only for functions that return ctf_id_t
- One lingering use of bzero() and of <sys/errno.h>

All fixed, using code from gnulib where possible.

Relatedly, set cts_size in a couple of places it was missed
(string table and symbol table loading upon ctf_bfdopen()).

binutils/
	* objdump.c (make_ctfsect): Drop cts_type, cts_flags, and
	cts_offset.
	* readelf.c (shdr_to_ctf_sect): Likewise.
include/
	* ctf-api.h (ctf_sect_t): Drop cts_type, cts_flags, and cts_offset.
	(ctf_id_t): This is now an unsigned type.
	(CTF_ERR): Cast it to ctf_id_t.  Note that it should only be used
	for ctf_id_t-returning functions.
libctf/
	* Makefile.am (ZLIB): New.
	(ZLIBINC): Likewise.
	(AM_CFLAGS): Use them.
	(libctf_a_LIBADD): New, for LIBOBJS.
	* configure.ac: Check for zlib, endian.h, and qsort_r.
	* ctf-endian.h: New, providing htole64 and le64toh.
	* swap.h: Code style fixes.
	(bswap_identity_64): New.
	* qsort_r.c: New, from gnulib (with one added #include).
	* ctf-decls.h: New, providing a conditional qsort_r declaration,
	and unconditional definitions of MIN and MAX.
	* ctf-impl.h: Use it.  Do not use <sys/errno.h>.
	(ctf_set_errno): Now returns unsigned long.
	* ctf-util.c (ctf_set_errno): Adjust here too.
	* ctf-archive.c: Use ctf-endian.h.
	(ctf_arc_open_by_offset): Use memset, not bzero.  Drop cts_type,
	cts_flags and cts_offset.
	(ctf_arc_write): Drop debugging dependent on the size of off_t.
	* ctf-create.c: Provide a definition of roundup if not defined.
	(ctf_create): Drop cts_type, cts_flags and cts_offset.
	(ctf_add_reftype): Do not check if type IDs are below zero.
	(ctf_add_slice): Likewise.
	(ctf_add_typedef): Likewise.
	(ctf_add_member_offset): Cast error-returning ssize_t's to size_t
	when known error-free.  Drop CTF_ERR usage for functions returning
	int.
	(ctf_add_member_encoded): Drop CTF_ERR usage for functions returning
	int.
	(ctf_add_variable): Likewise.
	(enumcmp): Likewise.
	(enumadd): Likewise.
	(membcmp): Likewise.
	(ctf_add_type): Likewise.  Cast error-returning ssize_t's to size_t
	when known error-free.
	* ctf-dump.c (ctf_is_slice): Drop CTF_ERR usage for functions
	returning int: use CTF_ERR for functions returning ctf_type_id.
	(ctf_dump_label): Likewise.
	(ctf_dump_objts): Likewise.
	* ctf-labels.c (ctf_label_topmost): Likewise.
	(ctf_label_iter): Likewise.
	(ctf_label_info): Likewise.
	* ctf-lookup.c (ctf_func_args): Likewise.
	* ctf-open.c (upgrade_types): Cast to size_t where appropriate.
	(ctf_bufopen): Likewise.  Use zlib types as needed.
	* ctf-types.c (ctf_member_iter): Drop CTF_ERR usage for functions
	returning int.
	(ctf_enum_iter): Likewise.
	(ctf_type_size): Likewise.
	(ctf_type_align): Likewise.  Cast to size_t where appropriate.
	(ctf_type_kind_unsliced): Likewise.
	(ctf_type_kind): Likewise.
	(ctf_type_encoding): Likewise.
	(ctf_member_info): Likewise.
	(ctf_array_info): Likewise.
	(ctf_enum_value): Likewise.
	(ctf_type_rvisit): Likewise.
	* ctf-open-bfd.c (ctf_bfdopen): Drop cts_type, cts_flags and
	cts_offset.
	(ctf_simple_open): Likewise.
	(ctf_bfdopen_ctfsect): Likewise.  Set cts_size properly.
	* Makefile.in: Regenerate.
	* aclocal.m4: Likewise.
	* config.h: Likewise.
	* configure: Likewise.
2019-05-31 11:10:51 +02:00
Nick Alcock a30b3e182a libctf: debug dumping
This introduces ctf_dump(), an iterator which returns a series of
strings, each representing a debugging dump of one item from a given
section in the CTF file.  The items may be multiline: a callback is
provided to allow the caller to decorate each line as they desire before
the line is returned.

libctf/
	* ctf-dump.c: New.

include/
	* ctf-api.h (ctf_dump_decorate_f): New.
	(ctf_dump_state_t): new.
	(ctf_dump): New.
2019-05-28 17:09:37 +01:00
Nick Alcock 6dbf2b7340 libctf: labels
This facility allows you to associate regions of type IDs with *labels*,
a labelled tiling of the type ID space. You can use these to define
CTF containers with distinct parents for distinct ranges of the ID
space, or to assist with parallelization of CTF processing, or for any
other purpose you can think of.

Notably absent from here (though declared in the API header) is any way
to define new labels: this will probably be introduced soon, as part of
the linker deduplication work.  (One existed in the past, but was deeply
tied to the Solaris CTF file generator and had to be torn out.)

libctf/
	* ctf-labels.c: New.
include/
	* ctf-api.h (ctf_label_f): New.
	(ctf_label_set): New.
	(ctf_label_get): New.
	(ctf_label_topmost): New.
	(ctf_label_info): New.
	(ctf_label_iter): New.
2019-05-28 17:09:30 +01:00
Nick Alcock 6c33b742ce libctf: library version enforcement
This old Solaris standard allows callers to specify that they are
expecting one particular API and/or CTF file format from the library.

libctf/
	* ctf-impl.h (_libctf_version): New declaration.
	* ctf-subr.c (_libctf_version): Define it.
	(ctf_version): New.

include/
	* ctf-api.h (ctf_version): New.
2019-05-28 17:08:29 +01:00
Nick Alcock b437bfe0f4 libctf: lookups by name and symbol
These functions allow you to look up types given a name in a simple
subset of C declarator syntax (no function pointers), to look up the
types of variables given a name, and to look up the types of data
objects and the type signatures of functions given symbol table offsets.

(Despite its name, one function in this commit, ctf_lookup_symbol_name(),
is for the internal use of libctf only, and does not appear in any
public header files.)

libctf/
	* ctf-lookup.c (isqualifier): New.
	(ctf_lookup_by_name): Likewise.
	(struct ctf_lookup_var_key): Likewise.
	(ctf_lookup_var): Likewise.
	(ctf_lookup_variable): Likewise.
	(ctf_lookup_symbol_name): Likewise.
	(ctf_lookup_by_symbol): Likewise.
	(ctf_func_info): Likewise.
	(ctf_func_args): Likewise.

include/
	* ctf-api.h (ctf_func_info): New.
	(ctf_func_args): Likewise.
	(ctf_lookup_by_symbol): Likewise.
	(ctf_lookup_by_symbol): Likewise.
	(ctf_lookup_variable): Likewise.
2019-05-28 17:08:19 +01:00
Nick Alcock 316afdb130 libctf: core type lookup
Finally we get to the functions used to actually look up and enumerate
properties of types in a container (names, sizes, members, what type a
pointer or cv-qual references, determination of whether two types are
assignment-compatible, etc).

With a very few exceptions these do not work for types newly added via
ctf_add_*(): they only work on types in read-only containers, or types
added before the most recent call to ctf_update().

This also adds support for lookup of "variables" (string -> type ID
mappings) and for generation of C type names corresponding to a type ID.

libctf/
	* ctf-decl.c: New file.
	* ctf-types.c: Likewise.
	* ctf-impl.h: New declarations.

include/
	* ctf-api.h (ctf_visit_f): New definition.
	(ctf_member_f): Likewise.
	(ctf_enum_f): Likewise.
	(ctf_variable_f): Likewise.
	(ctf_type_f): Likewise.
	(ctf_type_isparent): Likewise.
	(ctf_type_ischild): Likewise.
	(ctf_type_resolve): Likewise.
	(ctf_type_aname): Likewise.
	(ctf_type_lname): Likewise.
	(ctf_type_name): Likewise.
	(ctf_type_sizee): Likewise.
	(ctf_type_align): Likewise.
	(ctf_type_kind): Likewise.
	(ctf_type_reference): Likewise.
	(ctf_type_pointer): Likewise.
	(ctf_type_encoding): Likewise.
	(ctf_type_visit): Likewise.
	(ctf_type_cmp): Likewise.
	(ctf_type_compat): Likewise.
	(ctf_member_info): Likewise.
	(ctf_array_info): Likewise.
	(ctf_enum_name): Likewise.
	(ctf_enum_value): Likewise.
	(ctf_member_iter): Likewise.
	(ctf_enum_iter): Likewise.
	(ctf_type_iter): Likewise.
	(ctf_variable_iter): Likewise.
2019-05-28 17:08:14 +01:00
Nick Alcock 143dce8481 libctf: ELF file opening via BFD
These functions let you open an ELF file with a customarily-named CTF
section in it, automatically opening the CTF file or archive and
associating the symbol and string tables in the ELF file with the CTF
container, so that you can look up the types of symbols in the ELF file
via ctf_lookup_by_symbol(), and so that strings can be shared between
the ELF file and CTF container, to save space.

It uses BFD machinery to do so.  This has now been lightly tested and
seems to work.  In particular, if you already have a bfd you can pass
it in to ctf_bfdopen(), and if you want a bfd made for you you can
call ctf_open() or ctf_fdopen(), optionally specifying a target (or
try once without a target and then again with one if you get
ECTF_BFD_AMBIGUOUS back).

We use a forward declaration for the struct bfd in ctf-api.h, so that
ctf-api.h users are not required to pull in <bfd.h>.  (This is mostly
for the sake of readelf.)

libctf/
	* ctf-open-bfd.c: New file.
	* ctf-open.c (ctf_close): New.
	* ctf-impl.h: Include bfd.h.
	(ctf_file): New members ctf_data_mmapped, ctf_data_mmapped_len.
	(ctf_archive_internal): New members ctfi_abfd, ctfi_data,
	ctfi_bfd_close.
	(ctf_bfdopen_ctfsect): New declaration.
	(_CTF_SECTION): likewise.

include/
	* ctf-api.h (struct bfd): New forward.
	(ctf_fdopen): New.
	(ctf_bfdopen): Likewise.
	(ctf_open): Likewise.
	(ctf_arc_open): Likewise.
2019-05-28 17:08:08 +01:00
Nick Alcock 9402cc593f libctf: mmappable archives
If you need to store a large number of CTF containers somewhere, this
provides a dedicated facility for doing so: an mmappable archive format
like a very simple tar or ar without all the system-dependent format
horrors or need for heavy file copying, with built-in compression of
files above a particular size threshold.

libctf automatically mmap()s uncompressed elements of these archives, or
uncompresses them, as needed.  (If the platform does not support mmap(),
copying into dynamically-allocated buffers is used.)

Archive iteration operations are partitioned into raw and non-raw
forms. Raw operations pass thhe raw archive contents to the callback:
non-raw forms open each member with ctf_bufopen() and pass the resulting
ctf_file_t to the iterator instead.  This lets you manipulate the raw
data in the archive, or the contents interpreted as a CTF file, as
needed.

It is not yet known whether we will store CTF archives in a linked ELF
object in one of these (akin to debugdata) or whether they'll get one
section per TU plus one parent container for types shared between them.
(In the case of ELF objects with very large numbers of TUs, an archive
of all of them would seem preferable, so we might just use an archive,
and add lzma support so you can assume that .gnu_debugdata and .ctf are
compressed using the same algorithm if both are present.)

To make usage easier, the ctf_archive_t is not the on-disk
representation but an abstraction over both ctf_file_t's and archives of
many ctf_file_t's: users see both CTF archives and raw CTF files as
ctf_archive_t's upon opening, the only difference being that a raw CTF
file has only a single "archive member", named ".ctf" (the default if a
null pointer is passed in as the name).  The next commit will make use
of this facility, in addition to providing the public interface to
actually open archives.  (In the future, it should be possible to have
all CTF sections in an ELF file appear as an "archive" in the same
fashion.)

This machinery is also used to allow library-internal creators of
ctf_archive_t's (such as the next commit) to stash away an ELF string
and symbol table, so that all opens of members in a given archive will
use them.  This lets CTF archives exploit the ELF string and symbol
table just like raw CTF files can.

(All this leads to somewhat confusing type naming.  The ctf_archive_t is
a typedef for the opaque internal type, struct ctf_archive_internal: the
non-internal "struct ctf_archive" is the on-disk structure meant for
other libraries manipulating CTF files.  It is probably clearest to use
the struct name for struct ctf_archive_internal inside the program, and
the typedef names outside.)

libctf/
	* ctf-archive.c: New.
	* ctf-impl.h (ctf_archive_internal): New type.
	(ctf_arc_open_internal): New declaration.
	(ctf_arc_bufopen): Likewise.
	(ctf_arc_close_internal): Likewise.
include/
	* ctf.h (CTFA_MAGIC): New.
	(struct ctf_archive): New.
	(struct ctf_archive_modent): Likewise.
	* ctf-api.h (ctf_archive_member_f): New.
	(ctf_archive_raw_member_f): Likewise.
	(ctf_arc_write): Likewise.
	(ctf_arc_close): Likewise.
	(ctf_arc_open_by_name): Likewise.
	(ctf_archive_iter): Likewise.
	(ctf_archive_raw_iter): Likewise.
	(ctf_get_arc): Likewise.
2019-05-28 17:07:55 +01:00
Nick Alcock 72f3392127 libctf: opening
This fills in the other half of the opening/creation puzzle: opening of
already-existing CTF files.  Such files are always read-only: if you
want to add to a CTF file opened with one of the opening functions in
this file, use ctf_add_type(), in a later commit, to copy appropriate
types into a newly ctf_create()d, writable container.

The lowest-level opening functions are in here: ctf_bufopen(), which
takes ctf_sect_t structures akin to ELF section headers, and
ctf_simple_open(), which can be used if you don't have an entire ELF
section header to work from.  Both will malloc() new space for the
buffers only if necessary, will mmap() directly from the file if
requested, and will mprotect() it afterwards to prevent accidental
corruption of the types. These functions are also used by ctf_update()
when converting types in a writable container into read-only types that
can be looked up using the lookup functions (in later commits).

The files are always of the native endianness of the system that created
them: at read time, the endianness of the header magic number is used to
determine whether or not the file needs byte-swapping, and the entire
thing is aggressively byte-swapped.

The agggressive nature of this swapping avoids complicating the rest of
the code with endianness conversions, while the native endianness
introduces no byte-swapping overhead in the common case. (The
endianness-independence code is also much newer than everything else in
this file, and deserves closer scrutiny.)

The accessors at the top of the file are there to transparently support
older versions of the CTF file format, allowing translation from older
formats that have different sizes for the structures in ctf.h:
currently, these older formats are intermingled with the newer ones in
ctf.h: they will probably migrate to a compatibility header in time, to
ease readability.  The ctf_set_base() function is split out for the same
reason: when conversion code to a newer format is written, it would need
to malloc() new storage for the entire ctf_file_t if a file format
change causes it to grow, and for that we need ctf_set_base() to be a
separate function.

One pair of linked data structures supported by this file has no
creation code in libctf yet: the data and function object sections read
by init_symtab(). These will probably arrive soon, when the linker comes
to need them. (init_symtab() has hardly been changed since 2009, but if
any code in libctf has rotted over time, this will.)

A few simple accessors are also present that can even be called on
read-only containers because they don't actually modify them, since the
relevant things are not stored in the container but merely change its
operation: ctf_setmodel(), which lets you specify whether a container is
LP64 or not (used to statically determine the sizes of a few types),
ctf_import(), which is the only way to associate a parent container with
a child container, and ctf_setspecific(), which lets the caller
associate an arbitrary pointer with the CTF container for any use. If
the user doesn't call these functions correctly, libctf will misbehave:
this is particularly important for ctf_import(), since a container built
against a given parent container will not be able to resolve types that
depend on types in the parent unless it is ctf_import()ed with a parent
container with the same set of types at the same IDs, or a superset.

Possible future extensions (also noted in the ctf-hash.c file) include
storing a count of things so that we don't need to do one pass over the
CTF file counting everything, and computing a perfect hash at CTF
creation time in some compact form, storing it in the CTF file, and
using it to hash things so we don't need to do a second pass over the
entire CTF file to set up the hashes used to go from names to type IDs.
(There are multiple such hashes, one for each C type namespace: types,
enums, structs, and unions.)

libctf/
	* ctf-open.c: New file.
	* swap.h: Likewise.
include/
	* ctf-api.h (ctf_file_close): New declaration.
	(ctf_getdatasect): Likewise.
	(ctf_parent_file): Likewise.
	(ctf_parent_name): Likewise.
	(ctf_parent_name_set): Likewise.
	(ctf_import): Likewise.
	(ctf_setmodel): Likewise.
	(ctf_getmodel): Likewise.
	(ctf_setspecific): Likewise.
	(ctf_getspecific): Likewise.
2019-05-28 17:07:46 +01:00
Nick Alcock 47d546f427 libctf: creation functions
The CTF creation process looks roughly like (error handling elided):

int err;
ctf_file_t *foo = ctf_create (&err);

ctf_id_t type = ctf_add_THING (foo, ...);
ctf_update (foo);
ctf_*write (...);

Some ctf_add_THING functions accept other type IDs as arguments,
depending on the type: cv-quals, pointers, and structure and union
members all take other types as arguments.  So do 'slices', which
let you take an existing integral type and recast it as a type
with a different bitness or offset within a byte, for bitfields.
One class of THING is not a type: "variables", which are mappings
of names (in the internal string table) to types.  These are mostly
useful when encoding variables that do not appear in a symbol table
but which some external user has some other way to figure out the
address of at runtime (dynamic symbol lookup or querying a VM
interpreter or something).

You can snapshot the creation process at any point: rolling back to a
snapshot deletes all types and variables added since that point.

You can make arbitrary type queries on the CTF container during the
creation process, but you must call ctf_update() first, which
translates the growing dynamic container into a static one (this uses
the CTF opening machinery, added in a later commit), which is quite
expensive.  This function must also be called after adding types
and before writing the container out.

Because addition of types involves looking up existing types, we add a
little of the type lookup machinery here, as well: only enough to
look up types in dynamic containers under construction.

libctf/
	* ctf-create.c: New file.
	* ctf-lookup.c: New file.

include/
	* ctf-api.h (zlib.h): New include.
	(ctf_sect_t): New.
	(ctf_sect_names_t): Likewise.
	(ctf_encoding_t): Likewise.
	(ctf_membinfo_t): Likewise.
	(ctf_arinfo_t): Likewise.
	(ctf_funcinfo_t): Likewise.
	(ctf_lblinfo_t): Likewise.
	(ctf_snapshot_id_t): Likewise.
	(CTF_FUNC_VARARG): Likewise.
	(ctf_simple_open): Likewise.
	(ctf_bufopen): Likewise.
	(ctf_create): Likewise.
	(ctf_add_array): Likewise.
	(ctf_add_const): Likewise.
	(ctf_add_enum_encoded): Likewise.
	(ctf_add_enum): Likewise.
	(ctf_add_float): Likewise.
	(ctf_add_forward): Likewise.
	(ctf_add_function): Likewise.
	(ctf_add_integer): Likewise.
	(ctf_add_slice): Likewise.
	(ctf_add_pointer): Likewise.
	(ctf_add_type): Likewise.
	(ctf_add_typedef): Likewise.
	(ctf_add_restrict): Likewise.
	(ctf_add_struct): Likewise.
	(ctf_add_union): Likewise.
	(ctf_add_struct_sized): Likewise.
	(ctf_add_union_sized): Likewise.
	(ctf_add_volatile): Likewise.
	(ctf_add_enumerator): Likewise.
	(ctf_add_member): Likewise.
	(ctf_add_member_offset): Likewise.
	(ctf_add_member_encoded): Likewise.
	(ctf_add_variable): Likewise.
	(ctf_set_array): Likewise.
	(ctf_update): Likewise.
	(ctf_snapshot): Likewise.
	(ctf_rollback): Likewise.
	(ctf_discard): Likewise.
	(ctf_write): Likewise.
	(ctf_gzwrite): Likewise.
	(ctf_compress_write): Likewise.
2019-05-28 17:07:40 +01:00
Nick Alcock 479604f44f libctf: error handling
CTF functions return zero on success or an extended errno value which
can be translated into a string via the functions in this commit.

The errno numbers start at -CTF_BASE.

libctf/
	* ctf-error.c: New file.

include/
	* ctf-api.h (ctf_errno): New declaration.
	(ctf_errmsg): Likewise.
2019-05-28 17:07:24 +01:00
Nick Alcock 60da9d9559 libctf: lowest-level memory allocation and debug-dumping wrappers
The memory-allocation wrappers are simple things to allow malloc
interposition: they are only used inconsistently at present, usually
where malloc debugging was required in the past.

These provide a default implementation that is environment-variable
triggered (initialized on the first call to the libctf creation and
file-opening functions, the first functions people will use), and
a ctf_setdebug()/ctf_getdebug() pair that allows the caller to
explicitly turn debugging off and on.  If ctf_setdebug() is called,
the automatic setting from an environment variable is skipped.

libctf/
	* ctf-impl.h: New file.
	* ctf-subr.c: New file.

include/
	* ctf-api.h (ctf_setdebug): New.
	(ctf_getdebug): Likewise.
2019-05-28 17:07:15 +01:00
Nick Alcock 2e94b05630 include: new header ctf-api.h
This non-installed header is the means by which libctf consumers
communicate with libctf.

This header will be extended in subsequent commits.

include/
	* ctf-api.h: New file.
2019-05-28 17:07:11 +01:00
Nick Alcock fceac76e64 include: new header ctf.h: file format description
The data structures and macros in this header can be used, if desired,
to access or create CTF files directly, without going through libctf,
though this should rarely be necessary in practice.

libctf relies on this header as its description of the CTF file format.

include/
	* ctf.h: New file.
2019-05-28 17:06:55 +01:00
Szabolcs Nagy 2301ed1c9a aarch64: add STO_AARCH64_VARIANT_PCS and DT_AARCH64_VARIANT_PCS
The bottom 2 bits of st_other are used for visibility, the top 6 bits are
de facto reserved for processor specific use.  This patch defines a
bits to mark function symbols that follow a variant procedure call standard
with different register usage convention.

A dynamic tag is also defined that marks modules with R_<CLS>_JUMP_SLOT
relocations referencing symbols marked with STO_AARCH64_VARIANT_PCS.
This can be used by dynamic linkers that support lazy binding to decide
what registers need to be preserved during symbol resolution.

binutils/ChangeLog:

	* readelf.c (get_aarch64_dynamic_type): Handle DT_AARCH64_VARIANT_PCS.
	(get_aarch64_symbol_other): New, handles STO_AARCH64_VARIANT_PCS.
	(get_symbol_other): Call get_aarch64_symbol_other.

include/ChangeLog:

	* elf/aarch64.h (DT_AARCH64_VARIANT_PCS): Define.
	(STO_AARCH64_VARIANT_PCS): Define.
2019-05-24 15:05:57 +01:00
Alan Modra 5663e32184 PowerPC relocations for prefix insns
include/
	* elf/ppc64.h (R_PPC64_PLTSEQ_NOTOC, R_PPC64_PLTCALL_NOTOC),
	(R_PPC64_PCREL_OPT, R_PPC64_D34, R_PPC64_D34_LO, R_PPC64_D34_HI30),
	(R_PPC64_D34_HA30, R_PPC64_PCREL34, R_PPC64_GOT_PCREL34),
	(R_PPC64_PLT_PCREL34, R_PPC64_PLT_PCREL34_NOTOC),
	(R_PPC64_ADDR16_HIGHER34, R_PPC64_ADDR16_HIGHERA34),
	(R_PPC64_ADDR16_HIGHEST34, R_PPC64_ADDR16_HIGHESTA34),
	(R_PPC64_REL16_HIGHER34, R_PPC64_REL16_HIGHERA34),
	(R_PPC64_REL16_HIGHEST34, R_PPC64_REL16_HIGHESTA34),
	(R_PPC64_D28, R_PPC64_PCREL28): Define.
bfd/
	* reloc.c (BFD_RELOC_PPC64_D34, BFD_RELOC_PPC64_D34_LO),
	(BFD_RELOC_PPC64_D34_HI30, BFD_RELOC_PPC64_D34_HA30),
	(BFD_RELOC_PPC64_PCREL34, BFD_RELOC_PPC64_GOT_PCREL34),
	(BFD_RELOC_PPC64_PLT_PCREL34),
	(BFD_RELOC_PPC64_ADDR16_HIGHER34, BFD_RELOC_PPC64_ADDR16_HIGHERA34),
	(BFD_RELOC_PPC64_ADDR16_HIGHEST34, BFD_RELOC_PPC64_ADDR16_HIGHESTA34),
	(BFD_RELOC_PPC64_REL16_HIGHER34, BFD_RELOC_PPC64_REL16_HIGHERA34),
	(BFD_RELOC_PPC64_REL16_HIGHEST34, BFD_RELOC_PPC64_REL16_HIGHESTA34),
	(BFD_RELOC_PPC64_D28, BFD_RELOC_PPC64_PCREL28): New reloc enums.
	* elf64-ppc.c (PNOP): Define.
	(ppc64_elf_howto_raw): Add reloc howtos for new relocations.
	(ppc64_elf_reloc_type_lookup): Translate new bfd reloc numbers.
	(ppc64_elf_ha_reloc): Adjust addend for highera34 and highesta34
	relocs.
	(ppc64_elf_prefix_reloc): New function.
	(struct ppc_link_hash_table): Add notoc_plt.
	(is_branch_reloc): Add R_PPC64_PLTCALL_NOTOC.
	(is_plt_seq_reloc): Add R_PPC64_PLT_PCREL34,
	R_PPC64_PLT_PCREL34_NOTOC, and R_PPC64_PLTSEQ_NOTOC.
	(ppc64_elf_check_relocs): Handle pcrel got and plt relocs.  Set
	has_pltcall for section on seeing R_PPC64_PLTCALL_NOTOC.  Handle
	possible need for dynamic relocs on non-pcrel powerxx relocs.
	(dec_dynrel_count): Handle non-pcrel powerxx relocs.
	(ppc64_elf_inline_plt): Handle R_PPC64_PLTCALL_NOTOC.
	(toc_adjusting_stub_needed): Likewise.
	(ppc64_elf_tls_optimize): Handle R_PPC64_PLTSEQ_NOTOC.
	(ppc64_elf_relocate_section): Handle new powerxx relocs.
	* bfd-in2.h: Regenerate.
	* libbfd.h: Regenerate.
gas/
	* config/tc-ppc.c (ppc_elf_suffix): Support @pcrel, @got@pcrel,
	@plt@pcrel, @higher34, @highera34, @highest34, and @highesta34.
	(fixup_size): Handle new powerxx relocs.
	(md_assemble): Warn for @pcrel on non-prefix insns.
	Accept @l, @h and @ha on prefix insns, and infer reloc without
	any @ suffix.  Translate powerxx relocs to suit DQ and DS field
	instructions.  Include operand tests as well as opcode test to
	translate BFD_RELOC_HI16_S to BFD_RELOC_PPC_16DX_HA.
	(ppc_fix_adjustable): Return false for pcrel GOT and PLT relocs.
	(md_apply_fix): Handle new powerxx relocs.
	* config/tc-ppc.h (TC_FORCE_RELOCATION_SUB_LOCAL): Accept
	BFD_RELOC_PPC64_ADDR16_HIGHER34, BFD_RELOC_PPC64_ADDR16_HIGHERA34,
	BFD_RELOC_PPC64_ADDR16_HIGHEST34, BFD_RELOC_PPC64_ADDR16_HIGHESTA34,
	BFD_RELOC_PPC64_D34, and BFD_RELOC_PPC64_D28.
	* testsuite/gas/ppc/prefix-reloc.d,
	* testsuite/gas/ppc/prefix-reloc.s: New test.
	* testsuite/gas/ppc/ppc.exp: Run it.
2019-05-24 10:27:49 +09:30
Peter Bergner dd7efa7915 PowerPC add initial -mfuture instruction support
This patch adds initial 64-bit insn assembler/disassembler support.
The only instruction added is "pnop" along with the automatic aligning
of prefix instruction so they do not cross 64-byte boundaries.

include/
	* dis-asm.h (WIDE_OUTPUT): Define.
	* opcode/ppc.h (prefix_opcodes, prefix_num_opcodes): Declare.
	(PPC_OPCODE_POWERXX, PPC_GET_PREFIX, PPC_GET_SUFFIX),
	(PPC_PREFIX_P, PPC_PREFIX_SEG): Define.
opcodes/
	* ppc-dis.c (ppc_opts): Add "future" entry.
	(PREFIX_OPCD_SEGS): Define.
	(prefix_opcd_indices): New array.
	(disassemble_init_powerpc): Initialize prefix_opcd_indices.
	(lookup_prefix): New function.
	(print_insn_powerpc): Handle 64-bit prefix instructions.
	* ppc-opc.c (PREFIX_OP, PREFIX_FORM, SUFFIX_MASK, PREFIX_MASK),
	(PMRR, POWERXX): Define.
	(prefix_opcodes): New instruction table.
	(prefix_num_opcodes): New constant.
binutils/
	* objdump.c (disassemble_bytes): Set WIDE_OUTPUT in flags.
gas/
	* config/tc-ppc.c (ppc_setup_opcodes): Handle prefix_opcodes.
	(struct insn_label_list): New.
	(insn_labels, free_insn_labels): New variables.
	(ppc_record_label, ppc_clear_labels, ppc_start_line_hook): New funcs.
	(ppc_frob_label, ppc_new_dot_label): Move functions earlier in file
	and call ppc_record_label.
	(md_assemble): Handle 64-bit prefix instructions.  Align labels
	that are on the same line as a prefix instruction.
	* config/tc-ppc.h (tc_frob_label, ppc_frob_label): Move to
	later in the file.
	(md_start_line_hook): Define.
	(ppc_start_line_hook): Declare.
	* testsuite/gas/ppc/prefix-align.d,
	* testsuite/gas/ppc/prefix-align.s: New test.
	* testsuite/gas/ppc/ppc.exp: Run new test.
2019-05-24 10:24:45 +09:30
Jose E. Marchesi 8ebe621240 include: add elf/bpf.h
This patch adds a header file with BPF-specific ELF definitions.  In
particular, the architecture relocations.

include/ChangeLog:

2019-05-23  Jose E. Marchesi  <jose.marchesi@oracle.com>

	* elf/bpf.h: New file.
2019-05-23 19:32:58 +02:00
Andre Vieira a7ad558c37 [PATCH 1/57][Arm][GAS]: Add support for +mve and +mve.fp
bfd/ChangeLog:
2019-05-16  Andre Vieira  <andre.simoesdiasvieira@arm.com>

	* elf32-arm.c (elf32_arm_merge_eabi_attributes): Add case for Tag_MVE_arch.

binutils/ChangeLog:
2019-05-16  Andre Vieira  <andre.simoesdiasvieira@arm.com>

	* readelf.c (arm_attr_tag_MVE_arch): New array for Tag_MVE_arch values.
	(arm_attr_public_tag arm_attr_public_tags): Add case for Tag_MVE_arch.

elfcpp/ChangeLog:
2019-05-16  Andre Vieira  <andre.simoesdiasvieira@arm.com>

	* arm.h (Tag_MVE_arch): Define new enum value.

gas/ChangeLog:
2019-05-16  Andre Vieira  <andre.simoesdiasvieira@arm.com>

	* config/tc-arm.c (mve_ext, mve_fp_ext): New features.
	(armv8_1m_main_ext_table): Add new extensions.
	(aeabi_set_public_attributes): Translate new features to new build attributes.
	(arm_convert_symbolic_attribute): Add Tag_MVE_arch.
	* doc/c-arm.texi: Document new extensions and new build attribute.

include/ChangeLog:
2019-05-16  Andre Vieira  <andre.simoesdiasvieira@arm.com>

	* elf/arm.h (Tag_MVE_arch): Define new enum value.
	* opcode/arm.h (FPU_MVE, FPU_MVE_FP): New MACROs for new features.
2019-05-16 16:17:21 +01:00