2019-04-23 19:55:27 +02:00
|
|
|
/* Implementation header.
|
|
|
|
Copyright (C) 2019 Free Software Foundation, Inc.
|
|
|
|
|
|
|
|
This file is part of libctf.
|
|
|
|
|
|
|
|
libctf is free software; you can redistribute it and/or modify it under
|
|
|
|
the terms of the GNU General Public License as published by the Free
|
|
|
|
Software Foundation; either version 3, or (at your option) any later
|
|
|
|
version.
|
|
|
|
|
|
|
|
This program is distributed in the hope that it will be useful, but
|
|
|
|
WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
|
|
|
|
See the GNU General Public License for more details.
|
|
|
|
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
|
|
along with this program; see the file COPYING. If not see
|
|
|
|
<http://www.gnu.org/licenses/>. */
|
|
|
|
|
|
|
|
#ifndef _CTF_IMPL_H
|
|
|
|
#define _CTF_IMPL_H
|
|
|
|
|
|
|
|
#include "config.h"
|
libctf: fix a number of build problems found on Solaris and NetBSD
- Use of nonportable <endian.h>
- Use of qsort_r
- Use of zlib without appropriate magic to pull in the binutils zlib
- Use of off64_t without checking (fixed by dropping the unused fields
that need off64_t entirely)
- signedness problems due to long being too short a type on 32-bit
platforms: ctf_id_t is now 'unsigned long', and CTF_ERR must be
used only for functions that return ctf_id_t
- One lingering use of bzero() and of <sys/errno.h>
All fixed, using code from gnulib where possible.
Relatedly, set cts_size in a couple of places it was missed
(string table and symbol table loading upon ctf_bfdopen()).
binutils/
* objdump.c (make_ctfsect): Drop cts_type, cts_flags, and
cts_offset.
* readelf.c (shdr_to_ctf_sect): Likewise.
include/
* ctf-api.h (ctf_sect_t): Drop cts_type, cts_flags, and cts_offset.
(ctf_id_t): This is now an unsigned type.
(CTF_ERR): Cast it to ctf_id_t. Note that it should only be used
for ctf_id_t-returning functions.
libctf/
* Makefile.am (ZLIB): New.
(ZLIBINC): Likewise.
(AM_CFLAGS): Use them.
(libctf_a_LIBADD): New, for LIBOBJS.
* configure.ac: Check for zlib, endian.h, and qsort_r.
* ctf-endian.h: New, providing htole64 and le64toh.
* swap.h: Code style fixes.
(bswap_identity_64): New.
* qsort_r.c: New, from gnulib (with one added #include).
* ctf-decls.h: New, providing a conditional qsort_r declaration,
and unconditional definitions of MIN and MAX.
* ctf-impl.h: Use it. Do not use <sys/errno.h>.
(ctf_set_errno): Now returns unsigned long.
* ctf-util.c (ctf_set_errno): Adjust here too.
* ctf-archive.c: Use ctf-endian.h.
(ctf_arc_open_by_offset): Use memset, not bzero. Drop cts_type,
cts_flags and cts_offset.
(ctf_arc_write): Drop debugging dependent on the size of off_t.
* ctf-create.c: Provide a definition of roundup if not defined.
(ctf_create): Drop cts_type, cts_flags and cts_offset.
(ctf_add_reftype): Do not check if type IDs are below zero.
(ctf_add_slice): Likewise.
(ctf_add_typedef): Likewise.
(ctf_add_member_offset): Cast error-returning ssize_t's to size_t
when known error-free. Drop CTF_ERR usage for functions returning
int.
(ctf_add_member_encoded): Drop CTF_ERR usage for functions returning
int.
(ctf_add_variable): Likewise.
(enumcmp): Likewise.
(enumadd): Likewise.
(membcmp): Likewise.
(ctf_add_type): Likewise. Cast error-returning ssize_t's to size_t
when known error-free.
* ctf-dump.c (ctf_is_slice): Drop CTF_ERR usage for functions
returning int: use CTF_ERR for functions returning ctf_type_id.
(ctf_dump_label): Likewise.
(ctf_dump_objts): Likewise.
* ctf-labels.c (ctf_label_topmost): Likewise.
(ctf_label_iter): Likewise.
(ctf_label_info): Likewise.
* ctf-lookup.c (ctf_func_args): Likewise.
* ctf-open.c (upgrade_types): Cast to size_t where appropriate.
(ctf_bufopen): Likewise. Use zlib types as needed.
* ctf-types.c (ctf_member_iter): Drop CTF_ERR usage for functions
returning int.
(ctf_enum_iter): Likewise.
(ctf_type_size): Likewise.
(ctf_type_align): Likewise. Cast to size_t where appropriate.
(ctf_type_kind_unsliced): Likewise.
(ctf_type_kind): Likewise.
(ctf_type_encoding): Likewise.
(ctf_member_info): Likewise.
(ctf_array_info): Likewise.
(ctf_enum_value): Likewise.
(ctf_type_rvisit): Likewise.
* ctf-open-bfd.c (ctf_bfdopen): Drop cts_type, cts_flags and
cts_offset.
(ctf_simple_open): Likewise.
(ctf_bfdopen_ctfsect): Likewise. Set cts_size properly.
* Makefile.in: Regenerate.
* aclocal.m4: Likewise.
* config.h: Likewise.
* configure: Likewise.
2019-05-31 11:10:51 +02:00
|
|
|
#include <errno.h>
|
|
|
|
#include "ctf-decls.h"
|
2019-04-23 19:55:27 +02:00
|
|
|
#include <ctf-api.h>
|
|
|
|
#include <sys/types.h>
|
2019-04-23 22:45:30 +02:00
|
|
|
#include <stdlib.h>
|
|
|
|
#include <stdarg.h>
|
|
|
|
#include <stdio.h>
|
|
|
|
#include <stdint.h>
|
|
|
|
#include <limits.h>
|
|
|
|
#include <ctype.h>
|
|
|
|
#include <elf.h>
|
libctf: ELF file opening via BFD
These functions let you open an ELF file with a customarily-named CTF
section in it, automatically opening the CTF file or archive and
associating the symbol and string tables in the ELF file with the CTF
container, so that you can look up the types of symbols in the ELF file
via ctf_lookup_by_symbol(), and so that strings can be shared between
the ELF file and CTF container, to save space.
It uses BFD machinery to do so. This has now been lightly tested and
seems to work. In particular, if you already have a bfd you can pass
it in to ctf_bfdopen(), and if you want a bfd made for you you can
call ctf_open() or ctf_fdopen(), optionally specifying a target (or
try once without a target and then again with one if you get
ECTF_BFD_AMBIGUOUS back).
We use a forward declaration for the struct bfd in ctf-api.h, so that
ctf-api.h users are not required to pull in <bfd.h>. (This is mostly
for the sake of readelf.)
libctf/
* ctf-open-bfd.c: New file.
* ctf-open.c (ctf_close): New.
* ctf-impl.h: Include bfd.h.
(ctf_file): New members ctf_data_mmapped, ctf_data_mmapped_len.
(ctf_archive_internal): New members ctfi_abfd, ctfi_data,
ctfi_bfd_close.
(ctf_bfdopen_ctfsect): New declaration.
(_CTF_SECTION): likewise.
include/
* ctf-api.h (struct bfd): New forward.
(ctf_fdopen): New.
(ctf_bfdopen): Likewise.
(ctf_open): Likewise.
(ctf_arc_open): Likewise.
2019-04-24 11:46:39 +02:00
|
|
|
#include <bfd.h>
|
2019-04-23 19:55:27 +02:00
|
|
|
|
|
|
|
#ifdef __cplusplus
|
|
|
|
extern "C"
|
|
|
|
{
|
|
|
|
#endif
|
|
|
|
|
|
|
|
/* Compiler attributes. */
|
|
|
|
|
|
|
|
#if defined (__GNUC__)
|
|
|
|
|
|
|
|
/* GCC. We assume that all compilers claiming to be GCC support sufficiently
|
|
|
|
many GCC attributes that the code below works. If some non-GCC compilers
|
|
|
|
masquerading as GCC in fact do not implement these attributes, version checks
|
|
|
|
may be required. */
|
|
|
|
|
|
|
|
/* We use the _libctf_*_ pattern to avoid clashes with any future attribute
|
|
|
|
macros glibc may introduce, which have names of the pattern
|
|
|
|
__attribute_blah__. */
|
|
|
|
|
|
|
|
#define _libctf_printflike_(string_index,first_to_check) \
|
|
|
|
__attribute__ ((__format__ (__printf__, (string_index), (first_to_check))))
|
|
|
|
#define _libctf_unlikely_(x) __builtin_expect ((x), 0)
|
|
|
|
#define _libctf_unused_ __attribute__ ((__unused__))
|
|
|
|
#define _libctf_malloc_ __attribute__((__malloc__))
|
|
|
|
|
|
|
|
#endif
|
|
|
|
|
2019-04-23 23:12:16 +02:00
|
|
|
/* libctf in-memory state. */
|
|
|
|
|
|
|
|
typedef struct ctf_fixed_hash ctf_hash_t; /* Private to ctf-hash.c. */
|
|
|
|
typedef struct ctf_dynhash ctf_dynhash_t; /* Private to ctf-hash.c. */
|
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
typedef struct ctf_strs
|
|
|
|
{
|
|
|
|
const char *cts_strs; /* Base address of string table. */
|
|
|
|
size_t cts_len; /* Size of string table in bytes. */
|
|
|
|
} ctf_strs_t;
|
|
|
|
|
libctf: deduplicate and sort the string table
ctf.h states:
> [...] the CTF string table does not contain any duplicated strings.
Unfortunately this is entirely untrue: libctf has before now made no
attempt whatsoever to deduplicate the string table. It computes the
string table's length on the fly as it adds new strings to the dynamic
CTF file, and ctf_update() just writes each string to the table and
notes the current write position as it traverses the dynamic CTF file's
data structures and builds the final CTF buffer. There is no global
view of the strings and no deduplication.
Fix this by erasing the ctf_dtvstrlen dead-reckoning length, and adding
a new dynhash table ctf_str_atoms that maps unique strings to a list
of references to those strings: a reference is a simple uint32_t * to
some value somewhere in the under-construction CTF buffer that needs
updating to note the string offset when the strtab is laid out.
Adding a string is now a simple matter of calling ctf_str_add_ref(),
which adds a new atom to the atoms table, if one doesn't already exist,
and adding the location of the reference to this atom to the refs list
attached to the atom: this works reliably as long as one takes care to
only call ctf_str_add_ref() once the final location of the offset is
known (so you can't call it on a temporary structure and then memcpy()
that structure into place in the CTF buffer, because the ref will still
point to the old location: ctf_update() changes accordingly).
Generating the CTF string table is a matter of calling
ctf_str_write_strtab(), which counts the length and number of elements
in the atoms table using the ctf_dynhash_iter() function we just added,
populating an array of pointers into the atoms table and sorting it into
order (to help compressors), then traversing this table and emitting it,
updating the refs to each atom as we go. The only complexity here is
arranging to keep the null string at offset zero, since a lot of code in
libctf depends on being able to leave strtab references at 0 to indicate
'no name'. Once the table is constructed and the refs updated, we know
how long it is, so we can realloc() the partial CTF buffer we allocated
earlier and can copy the table on to the end of it (and purge the refs
because they're not needed any more and have been invalidated by the
realloc() call in any case).
The net effect of all this is a reduction in uncompressed strtab sizes
of about 30% (perhaps a quarter to a half of all strings across the
Linux kernel are eliminated as duplicates). Of course, duplicated
strings are highly redundant, so the space saving after compression is
only about 20%: when the other non-strtab sections are factored in, CTF
sizes shrink by about 10%.
No change in externally-visible API or file format (other than the
reduction in pointless redundancy).
libctf/
* ctf-impl.h: (struct ctf_strs_writable): New, non-const version of
struct ctf_strs.
(struct ctf_dtdef): Note that dtd_data.ctt_name is unpopulated.
(struct ctf_str_atom): New, disambiguated single string.
(struct ctf_str_atom_ref): New, points to some other location that
references this string's offset.
(struct ctf_file): New members ctf_str_atoms and ctf_str_num_refs.
Remove member ctf_dtvstrlen: we no longer track the total strlen
as we add strings.
(ctf_str_create_atoms): Declare new function in ctf-string.c.
(ctf_str_free_atoms): Likewise.
(ctf_str_add): Likewise.
(ctf_str_add_ref): Likewise.
(ctf_str_purge_refs): Likewise.
(ctf_str_write_strtab): Likewise.
(ctf_realloc): Declare new function in ctf-util.c.
* ctf-open.c (ctf_bufopen): Create the atoms table.
(ctf_file_close): Destroy it.
* ctf-create.c (ctf_update): Copy-and-free it on update. No longer
special-case the position of the parname string. Construct the
strtab by calling ctf_str_add_ref and ctf_str_write_strtab after the
rest of each buffer element is constructed, not via open-coding:
realloc the CTF buffer and append the strtab to it. No longer
maintain ctf_dtvstrlen. Sort the variable entry table later, after
strtab construction.
(ctf_copy_membnames): Remove: integrated into ctf_copy_{s,l,e}members.
(ctf_copy_smembers): Drop the string offset: call ctf_str_add_ref
after buffer element construction instead.
(ctf_copy_lmembers): Likewise.
(ctf_copy_emembers): Likewise.
(ctf_create): No longer maintain the ctf_dtvstrlen.
(ctf_dtd_delete): Likewise.
(ctf_dvd_delete): Likewise.
(ctf_add_generic): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(membadd): Likewise.
* ctf-util.c (ctf_realloc): New, wrapper around realloc that aborts
if there are active ctf_str_num_refs.
(ctf_strraw): Move to ctf-string.c.
(ctf_strptr): Likewise.
* ctf-string.c: New file, strtab manipulation.
* Makefile.am (libctf_a_SOURCES): Add it.
* Makefile.in: Regenerate.
2019-06-27 14:51:10 +02:00
|
|
|
typedef struct ctf_strs_writable
|
|
|
|
{
|
|
|
|
char *cts_strs; /* Base address of string table. */
|
|
|
|
size_t cts_len; /* Size of string table in bytes. */
|
|
|
|
} ctf_strs_writable_t;
|
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
typedef struct ctf_dmodel
|
|
|
|
{
|
|
|
|
const char *ctd_name; /* Data model name. */
|
|
|
|
int ctd_code; /* Data model code. */
|
|
|
|
size_t ctd_pointer; /* Size of void * in bytes. */
|
|
|
|
size_t ctd_char; /* Size of char in bytes. */
|
|
|
|
size_t ctd_short; /* Size of short in bytes. */
|
|
|
|
size_t ctd_int; /* Size of int in bytes. */
|
|
|
|
size_t ctd_long; /* Size of long in bytes. */
|
|
|
|
} ctf_dmodel_t;
|
|
|
|
|
|
|
|
typedef struct ctf_lookup
|
|
|
|
{
|
|
|
|
const char *ctl_prefix; /* String prefix for this lookup. */
|
|
|
|
size_t ctl_len; /* Length of prefix string in bytes. */
|
|
|
|
ctf_hash_t *ctl_hash; /* Pointer to hash table for lookup. */
|
|
|
|
} ctf_lookup_t;
|
|
|
|
|
|
|
|
typedef struct ctf_fileops
|
|
|
|
{
|
|
|
|
uint32_t (*ctfo_get_kind) (uint32_t);
|
|
|
|
uint32_t (*ctfo_get_root) (uint32_t);
|
|
|
|
uint32_t (*ctfo_get_vlen) (uint32_t);
|
|
|
|
ssize_t (*ctfo_get_ctt_size) (const ctf_file_t *, const ctf_type_t *,
|
|
|
|
ssize_t *, ssize_t *);
|
|
|
|
ssize_t (*ctfo_get_vbytes) (unsigned short, ssize_t, size_t);
|
|
|
|
} ctf_fileops_t;
|
|
|
|
|
2019-04-23 22:45:30 +02:00
|
|
|
typedef struct ctf_list
|
|
|
|
{
|
|
|
|
struct ctf_list *l_prev; /* Previous pointer or tail pointer. */
|
|
|
|
struct ctf_list *l_next; /* Next pointer or head pointer. */
|
|
|
|
} ctf_list_t;
|
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
typedef enum
|
|
|
|
{
|
|
|
|
CTF_PREC_BASE,
|
|
|
|
CTF_PREC_POINTER,
|
|
|
|
CTF_PREC_ARRAY,
|
|
|
|
CTF_PREC_FUNCTION,
|
|
|
|
CTF_PREC_MAX
|
|
|
|
} ctf_decl_prec_t;
|
|
|
|
|
|
|
|
typedef struct ctf_decl_node
|
|
|
|
{
|
|
|
|
ctf_list_t cd_list; /* Linked list pointers. */
|
|
|
|
ctf_id_t cd_type; /* Type identifier. */
|
|
|
|
uint32_t cd_kind; /* Type kind. */
|
|
|
|
uint32_t cd_n; /* Type dimension if array. */
|
|
|
|
} ctf_decl_node_t;
|
|
|
|
|
|
|
|
typedef struct ctf_decl
|
|
|
|
{
|
|
|
|
ctf_list_t cd_nodes[CTF_PREC_MAX]; /* Declaration node stacks. */
|
|
|
|
int cd_order[CTF_PREC_MAX]; /* Storage order of decls. */
|
|
|
|
ctf_decl_prec_t cd_qualp; /* Qualifier precision. */
|
|
|
|
ctf_decl_prec_t cd_ordp; /* Ordered precision. */
|
|
|
|
char *cd_buf; /* Buffer for output. */
|
|
|
|
int cd_err; /* Saved error value. */
|
|
|
|
int cd_enomem; /* Nonzero if OOM during printing. */
|
|
|
|
} ctf_decl_t;
|
|
|
|
|
|
|
|
typedef struct ctf_dmdef
|
|
|
|
{
|
|
|
|
ctf_list_t dmd_list; /* List forward/back pointers. */
|
|
|
|
char *dmd_name; /* Name of this member. */
|
|
|
|
ctf_id_t dmd_type; /* Type of this member (for sou). */
|
|
|
|
unsigned long dmd_offset; /* Offset of this member in bits (for sou). */
|
|
|
|
int dmd_value; /* Value of this member (for enum). */
|
|
|
|
} ctf_dmdef_t;
|
|
|
|
|
|
|
|
typedef struct ctf_dtdef
|
|
|
|
{
|
|
|
|
ctf_list_t dtd_list; /* List forward/back pointers. */
|
|
|
|
char *dtd_name; /* Name associated with definition (if any). */
|
|
|
|
ctf_id_t dtd_type; /* Type identifier for this definition. */
|
libctf: deduplicate and sort the string table
ctf.h states:
> [...] the CTF string table does not contain any duplicated strings.
Unfortunately this is entirely untrue: libctf has before now made no
attempt whatsoever to deduplicate the string table. It computes the
string table's length on the fly as it adds new strings to the dynamic
CTF file, and ctf_update() just writes each string to the table and
notes the current write position as it traverses the dynamic CTF file's
data structures and builds the final CTF buffer. There is no global
view of the strings and no deduplication.
Fix this by erasing the ctf_dtvstrlen dead-reckoning length, and adding
a new dynhash table ctf_str_atoms that maps unique strings to a list
of references to those strings: a reference is a simple uint32_t * to
some value somewhere in the under-construction CTF buffer that needs
updating to note the string offset when the strtab is laid out.
Adding a string is now a simple matter of calling ctf_str_add_ref(),
which adds a new atom to the atoms table, if one doesn't already exist,
and adding the location of the reference to this atom to the refs list
attached to the atom: this works reliably as long as one takes care to
only call ctf_str_add_ref() once the final location of the offset is
known (so you can't call it on a temporary structure and then memcpy()
that structure into place in the CTF buffer, because the ref will still
point to the old location: ctf_update() changes accordingly).
Generating the CTF string table is a matter of calling
ctf_str_write_strtab(), which counts the length and number of elements
in the atoms table using the ctf_dynhash_iter() function we just added,
populating an array of pointers into the atoms table and sorting it into
order (to help compressors), then traversing this table and emitting it,
updating the refs to each atom as we go. The only complexity here is
arranging to keep the null string at offset zero, since a lot of code in
libctf depends on being able to leave strtab references at 0 to indicate
'no name'. Once the table is constructed and the refs updated, we know
how long it is, so we can realloc() the partial CTF buffer we allocated
earlier and can copy the table on to the end of it (and purge the refs
because they're not needed any more and have been invalidated by the
realloc() call in any case).
The net effect of all this is a reduction in uncompressed strtab sizes
of about 30% (perhaps a quarter to a half of all strings across the
Linux kernel are eliminated as duplicates). Of course, duplicated
strings are highly redundant, so the space saving after compression is
only about 20%: when the other non-strtab sections are factored in, CTF
sizes shrink by about 10%.
No change in externally-visible API or file format (other than the
reduction in pointless redundancy).
libctf/
* ctf-impl.h: (struct ctf_strs_writable): New, non-const version of
struct ctf_strs.
(struct ctf_dtdef): Note that dtd_data.ctt_name is unpopulated.
(struct ctf_str_atom): New, disambiguated single string.
(struct ctf_str_atom_ref): New, points to some other location that
references this string's offset.
(struct ctf_file): New members ctf_str_atoms and ctf_str_num_refs.
Remove member ctf_dtvstrlen: we no longer track the total strlen
as we add strings.
(ctf_str_create_atoms): Declare new function in ctf-string.c.
(ctf_str_free_atoms): Likewise.
(ctf_str_add): Likewise.
(ctf_str_add_ref): Likewise.
(ctf_str_purge_refs): Likewise.
(ctf_str_write_strtab): Likewise.
(ctf_realloc): Declare new function in ctf-util.c.
* ctf-open.c (ctf_bufopen): Create the atoms table.
(ctf_file_close): Destroy it.
* ctf-create.c (ctf_update): Copy-and-free it on update. No longer
special-case the position of the parname string. Construct the
strtab by calling ctf_str_add_ref and ctf_str_write_strtab after the
rest of each buffer element is constructed, not via open-coding:
realloc the CTF buffer and append the strtab to it. No longer
maintain ctf_dtvstrlen. Sort the variable entry table later, after
strtab construction.
(ctf_copy_membnames): Remove: integrated into ctf_copy_{s,l,e}members.
(ctf_copy_smembers): Drop the string offset: call ctf_str_add_ref
after buffer element construction instead.
(ctf_copy_lmembers): Likewise.
(ctf_copy_emembers): Likewise.
(ctf_create): No longer maintain the ctf_dtvstrlen.
(ctf_dtd_delete): Likewise.
(ctf_dvd_delete): Likewise.
(ctf_add_generic): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(membadd): Likewise.
* ctf-util.c (ctf_realloc): New, wrapper around realloc that aborts
if there are active ctf_str_num_refs.
(ctf_strraw): Move to ctf-string.c.
(ctf_strptr): Likewise.
* ctf-string.c: New file, strtab manipulation.
* Makefile.am (libctf_a_SOURCES): Add it.
* Makefile.in: Regenerate.
2019-06-27 14:51:10 +02:00
|
|
|
ctf_type_t dtd_data; /* Type node: name left unpopulated. */
|
2019-04-23 23:24:13 +02:00
|
|
|
union
|
|
|
|
{
|
|
|
|
ctf_list_t dtu_members; /* struct, union, or enum */
|
|
|
|
ctf_arinfo_t dtu_arr; /* array */
|
|
|
|
ctf_encoding_t dtu_enc; /* integer or float */
|
|
|
|
ctf_id_t *dtu_argv; /* function */
|
|
|
|
ctf_slice_t dtu_slice; /* slice */
|
|
|
|
} dtd_u;
|
|
|
|
} ctf_dtdef_t;
|
|
|
|
|
|
|
|
typedef struct ctf_dvdef
|
|
|
|
{
|
|
|
|
ctf_list_t dvd_list; /* List forward/back pointers. */
|
|
|
|
char *dvd_name; /* Name associated with variable. */
|
|
|
|
ctf_id_t dvd_type; /* Type of variable. */
|
|
|
|
unsigned long dvd_snapshots; /* Snapshot count when inserted. */
|
|
|
|
} ctf_dvdef_t;
|
|
|
|
|
|
|
|
typedef struct ctf_bundle
|
|
|
|
{
|
|
|
|
ctf_file_t *ctb_file; /* CTF container handle. */
|
|
|
|
ctf_id_t ctb_type; /* CTF type identifier. */
|
|
|
|
ctf_dtdef_t *ctb_dtd; /* CTF dynamic type definition (if any). */
|
|
|
|
} ctf_bundle_t;
|
|
|
|
|
libctf: deduplicate and sort the string table
ctf.h states:
> [...] the CTF string table does not contain any duplicated strings.
Unfortunately this is entirely untrue: libctf has before now made no
attempt whatsoever to deduplicate the string table. It computes the
string table's length on the fly as it adds new strings to the dynamic
CTF file, and ctf_update() just writes each string to the table and
notes the current write position as it traverses the dynamic CTF file's
data structures and builds the final CTF buffer. There is no global
view of the strings and no deduplication.
Fix this by erasing the ctf_dtvstrlen dead-reckoning length, and adding
a new dynhash table ctf_str_atoms that maps unique strings to a list
of references to those strings: a reference is a simple uint32_t * to
some value somewhere in the under-construction CTF buffer that needs
updating to note the string offset when the strtab is laid out.
Adding a string is now a simple matter of calling ctf_str_add_ref(),
which adds a new atom to the atoms table, if one doesn't already exist,
and adding the location of the reference to this atom to the refs list
attached to the atom: this works reliably as long as one takes care to
only call ctf_str_add_ref() once the final location of the offset is
known (so you can't call it on a temporary structure and then memcpy()
that structure into place in the CTF buffer, because the ref will still
point to the old location: ctf_update() changes accordingly).
Generating the CTF string table is a matter of calling
ctf_str_write_strtab(), which counts the length and number of elements
in the atoms table using the ctf_dynhash_iter() function we just added,
populating an array of pointers into the atoms table and sorting it into
order (to help compressors), then traversing this table and emitting it,
updating the refs to each atom as we go. The only complexity here is
arranging to keep the null string at offset zero, since a lot of code in
libctf depends on being able to leave strtab references at 0 to indicate
'no name'. Once the table is constructed and the refs updated, we know
how long it is, so we can realloc() the partial CTF buffer we allocated
earlier and can copy the table on to the end of it (and purge the refs
because they're not needed any more and have been invalidated by the
realloc() call in any case).
The net effect of all this is a reduction in uncompressed strtab sizes
of about 30% (perhaps a quarter to a half of all strings across the
Linux kernel are eliminated as duplicates). Of course, duplicated
strings are highly redundant, so the space saving after compression is
only about 20%: when the other non-strtab sections are factored in, CTF
sizes shrink by about 10%.
No change in externally-visible API or file format (other than the
reduction in pointless redundancy).
libctf/
* ctf-impl.h: (struct ctf_strs_writable): New, non-const version of
struct ctf_strs.
(struct ctf_dtdef): Note that dtd_data.ctt_name is unpopulated.
(struct ctf_str_atom): New, disambiguated single string.
(struct ctf_str_atom_ref): New, points to some other location that
references this string's offset.
(struct ctf_file): New members ctf_str_atoms and ctf_str_num_refs.
Remove member ctf_dtvstrlen: we no longer track the total strlen
as we add strings.
(ctf_str_create_atoms): Declare new function in ctf-string.c.
(ctf_str_free_atoms): Likewise.
(ctf_str_add): Likewise.
(ctf_str_add_ref): Likewise.
(ctf_str_purge_refs): Likewise.
(ctf_str_write_strtab): Likewise.
(ctf_realloc): Declare new function in ctf-util.c.
* ctf-open.c (ctf_bufopen): Create the atoms table.
(ctf_file_close): Destroy it.
* ctf-create.c (ctf_update): Copy-and-free it on update. No longer
special-case the position of the parname string. Construct the
strtab by calling ctf_str_add_ref and ctf_str_write_strtab after the
rest of each buffer element is constructed, not via open-coding:
realloc the CTF buffer and append the strtab to it. No longer
maintain ctf_dtvstrlen. Sort the variable entry table later, after
strtab construction.
(ctf_copy_membnames): Remove: integrated into ctf_copy_{s,l,e}members.
(ctf_copy_smembers): Drop the string offset: call ctf_str_add_ref
after buffer element construction instead.
(ctf_copy_lmembers): Likewise.
(ctf_copy_emembers): Likewise.
(ctf_create): No longer maintain the ctf_dtvstrlen.
(ctf_dtd_delete): Likewise.
(ctf_dvd_delete): Likewise.
(ctf_add_generic): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(membadd): Likewise.
* ctf-util.c (ctf_realloc): New, wrapper around realloc that aborts
if there are active ctf_str_num_refs.
(ctf_strraw): Move to ctf-string.c.
(ctf_strptr): Likewise.
* ctf-string.c: New file, strtab manipulation.
* Makefile.am (libctf_a_SOURCES): Add it.
* Makefile.in: Regenerate.
2019-06-27 14:51:10 +02:00
|
|
|
/* Atoms associate strings with a list of the CTF items that reference that
|
|
|
|
string, so that ctf_update() can instantiate all the strings using the
|
|
|
|
ctf_str_atoms and then reassociate them with the real string later.
|
|
|
|
|
|
|
|
Strings can be interned into ctf_str_atom without having refs associated
|
|
|
|
with them, for values that are returned to callers, etc. Items are only
|
|
|
|
removed from this table on ctf_close(), but on every ctf_update(), all the
|
|
|
|
csa_refs in all entries are purged. */
|
|
|
|
|
|
|
|
typedef struct ctf_str_atom
|
|
|
|
{
|
|
|
|
const char *csa_str; /* Backpointer to string (hash key). */
|
|
|
|
ctf_list_t csa_refs; /* This string's refs. */
|
|
|
|
unsigned long csa_snapshot_id; /* Snapshot ID at time of creation. */
|
|
|
|
} ctf_str_atom_t;
|
|
|
|
|
|
|
|
/* The refs of a single string in the atoms table. */
|
|
|
|
|
|
|
|
typedef struct ctf_str_atom_ref
|
|
|
|
{
|
|
|
|
ctf_list_t caf_list; /* List forward/back pointers. */
|
|
|
|
uint32_t *caf_ref; /* A single ref to this string. */
|
|
|
|
} ctf_str_atom_ref_t;
|
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
/* The ctf_file is the structure used to represent a CTF container to library
|
|
|
|
clients, who see it only as an opaque pointer. Modifications can therefore
|
|
|
|
be made freely to this structure without regard to client versioning. The
|
|
|
|
ctf_file_t typedef appears in <ctf-api.h> and declares a forward tag.
|
|
|
|
|
|
|
|
NOTE: ctf_update() requires that everything inside of ctf_file either be an
|
|
|
|
immediate value, a pointer to dynamically allocated data *outside* of the
|
|
|
|
ctf_file itself, or a pointer to statically allocated data. If you add a
|
|
|
|
pointer to ctf_file that points to something within the ctf_file itself,
|
|
|
|
you must make corresponding changes to ctf_update(). */
|
|
|
|
|
|
|
|
struct ctf_file
|
|
|
|
{
|
|
|
|
const ctf_fileops_t *ctf_fileops; /* Version-specific file operations. */
|
libctf: allow the header to change between versions
libctf supports dynamic upgrading of the type table as file format
versions change, but before now has not supported changes to the CTF
header. Doing this is complicated by the baroque storage method used:
the CTF header is kept prepended to the rest of the CTF data, just as
when read from the file, and written out from there, and is
endian-flipped in place.
This makes accessing it needlessly hard and makes it almost impossible
to make the header larger if we add fields. The general storage
machinery around the malloced ctf pointer (the 'ctf_base') is also
overcomplicated: the pointer is sometimes malloced locally and sometimes
assigned from a parameter, so freeing it requires checking to see if
that parameter was used, needlessly coupling ctf_bufopen and
ctf_file_close together.
So split the header out into a new ctf_file_t.ctf_header, which is
written out explicitly: squeeze it out of the CTF buffer whenever we
reallocate it, and use ctf_file_t.ctf_buf to skip past the header when
we do not need to reallocate (when no upgrading or endian-flipping is
required). We now track whether the CTF base can be freed explicitly
via a new ctf_dynbase pointer which is non-NULL only when freeing is
possible.
With all this done, we can upgrade the header on the fly and add new
fields as desired, via a new upgrade_header function in ctf-open.
As with other forms of upgrading, libctf upgrades older headers
automatically to the latest supported version at open time.
For a first use of this field, we add a new string field cth_cuname, and
a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this
is used by debuggers to determine whether a CTF section's types relate
to a single compilation unit, or to all compilation units in the
program. (Types with ambiguous definitions in different CUs have only
one of these types placed in the top-level shared .ctf container: the
rest are placed in much smaller per-CU containers, which have the shared
container as their parent. Since CTF must be useful in the absence of
DWARF, we store the names of the relevant CUs ourselves, so the debugger
can look them up.)
v5: fix tabdamage.
include/
* ctf-api.h (ctf_cuname): New function.
(ctf_cuname_set): Likewise.
* ctf.h: Improve comment around upgrading, no longer
implying that v2 is the target of upgrades (it is v3 now).
(ctf_header_v2_t): New, old-format header for backward
compatibility.
(ctf_header_t): Add cth_cuname: this is the first of several
header changes in format v3.
libctf/
* ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase,
ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const.
* ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and
ctf_base: do not assume that it is always sizeof (ctf_header_t).
Print out ctf_cuname: only print out ctf_parname if set.
(ctf_free_base): Removed, ctf_base is no longer freed: free
ctf_dynbase instead.
(ctf_set_version): Fix spacing.
(upgrade_header): New, in-place header upgrading.
(upgrade_types): Rename to...
(upgrade_types_v1): ... this. Free ctf_dynbase, not ctf_base. No
longer track old and new headers separately. No longer allow for
header sizes explicitly: squeeze the headers out on upgrade (they
are preserved in fp->ctf_header). Set ctf_dynbase, ctf_base and
ctf_buf explicitly. Use ctf_free, not ctf_free_base.
(upgrade_types): New, also handle ctf_parmax updating.
(flip_header): Flip ctf_cuname.
(flip_types): Flip BUF explicitly rather than deriving BUF from
BASE.
(ctf_bufopen): Store the header in fp->ctf_header. Correct minimum
required alignment of objtoff and funcoff. No longer store it in
the ctf_buf unless that buf is derived unmodified from the input.
Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals
that duplicate fields in ctf_file: move allocation of ctf_file
further up instead. Call upgrade_header as needed. Move
version-specific ctf_parmax initialization into upgrade_types. More
concise error handling.
(ctf_file_close): No longer test for null pointers before freeing.
Free ctf_dyncuname, ctf_dynbase, and ctf_header. Do not call
ctf_free_base.
(ctf_cuname): New.
(ctf_cuname_set): New.
* ctf-create.c (ctf_update): Populate ctf_cuname.
(ctf_gzwrite): Write out the header explicitly. Remove obsolescent
comment.
(ctf_write): Likewise.
(ctf_compress_write): Get the header from ctf_header, not ctf_base.
Fix the compression length: fp->ctf_size never counted the CTF
header. Simplify the compress call accordingly.
2019-07-06 18:36:21 +02:00
|
|
|
struct ctf_header *ctf_header; /* The header from this CTF file. */
|
2019-07-08 14:59:15 +02:00
|
|
|
unsigned char ctf_openflags; /* Flags the file had when opened. */
|
2019-04-23 23:24:13 +02:00
|
|
|
ctf_sect_t ctf_data; /* CTF data from object file. */
|
|
|
|
ctf_sect_t ctf_symtab; /* Symbol table from object file. */
|
|
|
|
ctf_sect_t ctf_strtab; /* String table from object file. */
|
libctf: ELF file opening via BFD
These functions let you open an ELF file with a customarily-named CTF
section in it, automatically opening the CTF file or archive and
associating the symbol and string tables in the ELF file with the CTF
container, so that you can look up the types of symbols in the ELF file
via ctf_lookup_by_symbol(), and so that strings can be shared between
the ELF file and CTF container, to save space.
It uses BFD machinery to do so. This has now been lightly tested and
seems to work. In particular, if you already have a bfd you can pass
it in to ctf_bfdopen(), and if you want a bfd made for you you can
call ctf_open() or ctf_fdopen(), optionally specifying a target (or
try once without a target and then again with one if you get
ECTF_BFD_AMBIGUOUS back).
We use a forward declaration for the struct bfd in ctf-api.h, so that
ctf-api.h users are not required to pull in <bfd.h>. (This is mostly
for the sake of readelf.)
libctf/
* ctf-open-bfd.c: New file.
* ctf-open.c (ctf_close): New.
* ctf-impl.h: Include bfd.h.
(ctf_file): New members ctf_data_mmapped, ctf_data_mmapped_len.
(ctf_archive_internal): New members ctfi_abfd, ctfi_data,
ctfi_bfd_close.
(ctf_bfdopen_ctfsect): New declaration.
(_CTF_SECTION): likewise.
include/
* ctf-api.h (struct bfd): New forward.
(ctf_fdopen): New.
(ctf_bfdopen): Likewise.
(ctf_open): Likewise.
(ctf_arc_open): Likewise.
2019-04-24 11:46:39 +02:00
|
|
|
void *ctf_data_mmapped; /* CTF data we mmapped, to free later. */
|
|
|
|
size_t ctf_data_mmapped_len; /* Length of CTF data we mmapped. */
|
2019-04-23 23:24:13 +02:00
|
|
|
ctf_hash_t *ctf_structs; /* Hash table of struct types. */
|
|
|
|
ctf_hash_t *ctf_unions; /* Hash table of union types. */
|
|
|
|
ctf_hash_t *ctf_enums; /* Hash table of enum types. */
|
|
|
|
ctf_hash_t *ctf_names; /* Hash table of remaining type names. */
|
|
|
|
ctf_lookup_t ctf_lookups[5]; /* Pointers to hashes for name lookup. */
|
|
|
|
ctf_strs_t ctf_str[2]; /* Array of string table base and bounds. */
|
libctf: deduplicate and sort the string table
ctf.h states:
> [...] the CTF string table does not contain any duplicated strings.
Unfortunately this is entirely untrue: libctf has before now made no
attempt whatsoever to deduplicate the string table. It computes the
string table's length on the fly as it adds new strings to the dynamic
CTF file, and ctf_update() just writes each string to the table and
notes the current write position as it traverses the dynamic CTF file's
data structures and builds the final CTF buffer. There is no global
view of the strings and no deduplication.
Fix this by erasing the ctf_dtvstrlen dead-reckoning length, and adding
a new dynhash table ctf_str_atoms that maps unique strings to a list
of references to those strings: a reference is a simple uint32_t * to
some value somewhere in the under-construction CTF buffer that needs
updating to note the string offset when the strtab is laid out.
Adding a string is now a simple matter of calling ctf_str_add_ref(),
which adds a new atom to the atoms table, if one doesn't already exist,
and adding the location of the reference to this atom to the refs list
attached to the atom: this works reliably as long as one takes care to
only call ctf_str_add_ref() once the final location of the offset is
known (so you can't call it on a temporary structure and then memcpy()
that structure into place in the CTF buffer, because the ref will still
point to the old location: ctf_update() changes accordingly).
Generating the CTF string table is a matter of calling
ctf_str_write_strtab(), which counts the length and number of elements
in the atoms table using the ctf_dynhash_iter() function we just added,
populating an array of pointers into the atoms table and sorting it into
order (to help compressors), then traversing this table and emitting it,
updating the refs to each atom as we go. The only complexity here is
arranging to keep the null string at offset zero, since a lot of code in
libctf depends on being able to leave strtab references at 0 to indicate
'no name'. Once the table is constructed and the refs updated, we know
how long it is, so we can realloc() the partial CTF buffer we allocated
earlier and can copy the table on to the end of it (and purge the refs
because they're not needed any more and have been invalidated by the
realloc() call in any case).
The net effect of all this is a reduction in uncompressed strtab sizes
of about 30% (perhaps a quarter to a half of all strings across the
Linux kernel are eliminated as duplicates). Of course, duplicated
strings are highly redundant, so the space saving after compression is
only about 20%: when the other non-strtab sections are factored in, CTF
sizes shrink by about 10%.
No change in externally-visible API or file format (other than the
reduction in pointless redundancy).
libctf/
* ctf-impl.h: (struct ctf_strs_writable): New, non-const version of
struct ctf_strs.
(struct ctf_dtdef): Note that dtd_data.ctt_name is unpopulated.
(struct ctf_str_atom): New, disambiguated single string.
(struct ctf_str_atom_ref): New, points to some other location that
references this string's offset.
(struct ctf_file): New members ctf_str_atoms and ctf_str_num_refs.
Remove member ctf_dtvstrlen: we no longer track the total strlen
as we add strings.
(ctf_str_create_atoms): Declare new function in ctf-string.c.
(ctf_str_free_atoms): Likewise.
(ctf_str_add): Likewise.
(ctf_str_add_ref): Likewise.
(ctf_str_purge_refs): Likewise.
(ctf_str_write_strtab): Likewise.
(ctf_realloc): Declare new function in ctf-util.c.
* ctf-open.c (ctf_bufopen): Create the atoms table.
(ctf_file_close): Destroy it.
* ctf-create.c (ctf_update): Copy-and-free it on update. No longer
special-case the position of the parname string. Construct the
strtab by calling ctf_str_add_ref and ctf_str_write_strtab after the
rest of each buffer element is constructed, not via open-coding:
realloc the CTF buffer and append the strtab to it. No longer
maintain ctf_dtvstrlen. Sort the variable entry table later, after
strtab construction.
(ctf_copy_membnames): Remove: integrated into ctf_copy_{s,l,e}members.
(ctf_copy_smembers): Drop the string offset: call ctf_str_add_ref
after buffer element construction instead.
(ctf_copy_lmembers): Likewise.
(ctf_copy_emembers): Likewise.
(ctf_create): No longer maintain the ctf_dtvstrlen.
(ctf_dtd_delete): Likewise.
(ctf_dvd_delete): Likewise.
(ctf_add_generic): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(membadd): Likewise.
* ctf-util.c (ctf_realloc): New, wrapper around realloc that aborts
if there are active ctf_str_num_refs.
(ctf_strraw): Move to ctf-string.c.
(ctf_strptr): Likewise.
* ctf-string.c: New file, strtab manipulation.
* Makefile.am (libctf_a_SOURCES): Add it.
* Makefile.in: Regenerate.
2019-06-27 14:51:10 +02:00
|
|
|
ctf_dynhash_t *ctf_str_atoms; /* Hash table of ctf_str_atoms_t. */
|
|
|
|
uint64_t ctf_str_num_refs; /* Number of refs to cts_str_atoms. */
|
libctf: allow the header to change between versions
libctf supports dynamic upgrading of the type table as file format
versions change, but before now has not supported changes to the CTF
header. Doing this is complicated by the baroque storage method used:
the CTF header is kept prepended to the rest of the CTF data, just as
when read from the file, and written out from there, and is
endian-flipped in place.
This makes accessing it needlessly hard and makes it almost impossible
to make the header larger if we add fields. The general storage
machinery around the malloced ctf pointer (the 'ctf_base') is also
overcomplicated: the pointer is sometimes malloced locally and sometimes
assigned from a parameter, so freeing it requires checking to see if
that parameter was used, needlessly coupling ctf_bufopen and
ctf_file_close together.
So split the header out into a new ctf_file_t.ctf_header, which is
written out explicitly: squeeze it out of the CTF buffer whenever we
reallocate it, and use ctf_file_t.ctf_buf to skip past the header when
we do not need to reallocate (when no upgrading or endian-flipping is
required). We now track whether the CTF base can be freed explicitly
via a new ctf_dynbase pointer which is non-NULL only when freeing is
possible.
With all this done, we can upgrade the header on the fly and add new
fields as desired, via a new upgrade_header function in ctf-open.
As with other forms of upgrading, libctf upgrades older headers
automatically to the latest supported version at open time.
For a first use of this field, we add a new string field cth_cuname, and
a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this
is used by debuggers to determine whether a CTF section's types relate
to a single compilation unit, or to all compilation units in the
program. (Types with ambiguous definitions in different CUs have only
one of these types placed in the top-level shared .ctf container: the
rest are placed in much smaller per-CU containers, which have the shared
container as their parent. Since CTF must be useful in the absence of
DWARF, we store the names of the relevant CUs ourselves, so the debugger
can look them up.)
v5: fix tabdamage.
include/
* ctf-api.h (ctf_cuname): New function.
(ctf_cuname_set): Likewise.
* ctf.h: Improve comment around upgrading, no longer
implying that v2 is the target of upgrades (it is v3 now).
(ctf_header_v2_t): New, old-format header for backward
compatibility.
(ctf_header_t): Add cth_cuname: this is the first of several
header changes in format v3.
libctf/
* ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase,
ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const.
* ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and
ctf_base: do not assume that it is always sizeof (ctf_header_t).
Print out ctf_cuname: only print out ctf_parname if set.
(ctf_free_base): Removed, ctf_base is no longer freed: free
ctf_dynbase instead.
(ctf_set_version): Fix spacing.
(upgrade_header): New, in-place header upgrading.
(upgrade_types): Rename to...
(upgrade_types_v1): ... this. Free ctf_dynbase, not ctf_base. No
longer track old and new headers separately. No longer allow for
header sizes explicitly: squeeze the headers out on upgrade (they
are preserved in fp->ctf_header). Set ctf_dynbase, ctf_base and
ctf_buf explicitly. Use ctf_free, not ctf_free_base.
(upgrade_types): New, also handle ctf_parmax updating.
(flip_header): Flip ctf_cuname.
(flip_types): Flip BUF explicitly rather than deriving BUF from
BASE.
(ctf_bufopen): Store the header in fp->ctf_header. Correct minimum
required alignment of objtoff and funcoff. No longer store it in
the ctf_buf unless that buf is derived unmodified from the input.
Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals
that duplicate fields in ctf_file: move allocation of ctf_file
further up instead. Call upgrade_header as needed. Move
version-specific ctf_parmax initialization into upgrade_types. More
concise error handling.
(ctf_file_close): No longer test for null pointers before freeing.
Free ctf_dyncuname, ctf_dynbase, and ctf_header. Do not call
ctf_free_base.
(ctf_cuname): New.
(ctf_cuname_set): New.
* ctf-create.c (ctf_update): Populate ctf_cuname.
(ctf_gzwrite): Write out the header explicitly. Remove obsolescent
comment.
(ctf_write): Likewise.
(ctf_compress_write): Get the header from ctf_header, not ctf_base.
Fix the compression length: fp->ctf_size never counted the CTF
header. Simplify the compress call accordingly.
2019-07-06 18:36:21 +02:00
|
|
|
unsigned char *ctf_base; /* CTF file pointer. */
|
|
|
|
unsigned char *ctf_dynbase; /* Freeable CTF file pointer. */
|
|
|
|
unsigned char *ctf_buf; /* Uncompressed CTF data buffer. */
|
2019-04-23 23:24:13 +02:00
|
|
|
size_t ctf_size; /* Size of CTF header + uncompressed data. */
|
|
|
|
uint32_t *ctf_sxlate; /* Translation table for symtab entries. */
|
|
|
|
unsigned long ctf_nsyms; /* Number of entries in symtab xlate table. */
|
|
|
|
uint32_t *ctf_txlate; /* Translation table for type IDs. */
|
|
|
|
uint32_t *ctf_ptrtab; /* Translation table for pointer-to lookups. */
|
|
|
|
struct ctf_varent *ctf_vars; /* Sorted variable->type mapping. */
|
|
|
|
unsigned long ctf_nvars; /* Number of variables in ctf_vars. */
|
|
|
|
unsigned long ctf_typemax; /* Maximum valid type ID number. */
|
|
|
|
const ctf_dmodel_t *ctf_dmodel; /* Data model pointer (see above). */
|
libctf: allow the header to change between versions
libctf supports dynamic upgrading of the type table as file format
versions change, but before now has not supported changes to the CTF
header. Doing this is complicated by the baroque storage method used:
the CTF header is kept prepended to the rest of the CTF data, just as
when read from the file, and written out from there, and is
endian-flipped in place.
This makes accessing it needlessly hard and makes it almost impossible
to make the header larger if we add fields. The general storage
machinery around the malloced ctf pointer (the 'ctf_base') is also
overcomplicated: the pointer is sometimes malloced locally and sometimes
assigned from a parameter, so freeing it requires checking to see if
that parameter was used, needlessly coupling ctf_bufopen and
ctf_file_close together.
So split the header out into a new ctf_file_t.ctf_header, which is
written out explicitly: squeeze it out of the CTF buffer whenever we
reallocate it, and use ctf_file_t.ctf_buf to skip past the header when
we do not need to reallocate (when no upgrading or endian-flipping is
required). We now track whether the CTF base can be freed explicitly
via a new ctf_dynbase pointer which is non-NULL only when freeing is
possible.
With all this done, we can upgrade the header on the fly and add new
fields as desired, via a new upgrade_header function in ctf-open.
As with other forms of upgrading, libctf upgrades older headers
automatically to the latest supported version at open time.
For a first use of this field, we add a new string field cth_cuname, and
a corresponding setter/getter pair ctf_cuname_set and ctf_cuname: this
is used by debuggers to determine whether a CTF section's types relate
to a single compilation unit, or to all compilation units in the
program. (Types with ambiguous definitions in different CUs have only
one of these types placed in the top-level shared .ctf container: the
rest are placed in much smaller per-CU containers, which have the shared
container as their parent. Since CTF must be useful in the absence of
DWARF, we store the names of the relevant CUs ourselves, so the debugger
can look them up.)
v5: fix tabdamage.
include/
* ctf-api.h (ctf_cuname): New function.
(ctf_cuname_set): Likewise.
* ctf.h: Improve comment around upgrading, no longer
implying that v2 is the target of upgrades (it is v3 now).
(ctf_header_v2_t): New, old-format header for backward
compatibility.
(ctf_header_t): Add cth_cuname: this is the first of several
header changes in format v3.
libctf/
* ctf-impl.h (ctf_file_t): New fields ctf_header, ctf_dynbase,
ctf_cuname, ctf_dyncuname: ctf_base and ctf_buf are no longer const.
* ctf-open.c (ctf_set_base): Preserve the gap between ctf_buf and
ctf_base: do not assume that it is always sizeof (ctf_header_t).
Print out ctf_cuname: only print out ctf_parname if set.
(ctf_free_base): Removed, ctf_base is no longer freed: free
ctf_dynbase instead.
(ctf_set_version): Fix spacing.
(upgrade_header): New, in-place header upgrading.
(upgrade_types): Rename to...
(upgrade_types_v1): ... this. Free ctf_dynbase, not ctf_base. No
longer track old and new headers separately. No longer allow for
header sizes explicitly: squeeze the headers out on upgrade (they
are preserved in fp->ctf_header). Set ctf_dynbase, ctf_base and
ctf_buf explicitly. Use ctf_free, not ctf_free_base.
(upgrade_types): New, also handle ctf_parmax updating.
(flip_header): Flip ctf_cuname.
(flip_types): Flip BUF explicitly rather than deriving BUF from
BASE.
(ctf_bufopen): Store the header in fp->ctf_header. Correct minimum
required alignment of objtoff and funcoff. No longer store it in
the ctf_buf unless that buf is derived unmodified from the input.
Set ctf_dynbase where ctf_base is dynamically allocated. Drop locals
that duplicate fields in ctf_file: move allocation of ctf_file
further up instead. Call upgrade_header as needed. Move
version-specific ctf_parmax initialization into upgrade_types. More
concise error handling.
(ctf_file_close): No longer test for null pointers before freeing.
Free ctf_dyncuname, ctf_dynbase, and ctf_header. Do not call
ctf_free_base.
(ctf_cuname): New.
(ctf_cuname_set): New.
* ctf-create.c (ctf_update): Populate ctf_cuname.
(ctf_gzwrite): Write out the header explicitly. Remove obsolescent
comment.
(ctf_write): Likewise.
(ctf_compress_write): Get the header from ctf_header, not ctf_base.
Fix the compression length: fp->ctf_size never counted the CTF
header. Simplify the compress call accordingly.
2019-07-06 18:36:21 +02:00
|
|
|
const char *ctf_cuname; /* Compilation unit name (if any). */
|
|
|
|
char *ctf_dyncuname; /* Dynamically allocated name of CU. */
|
2019-04-23 23:24:13 +02:00
|
|
|
struct ctf_file *ctf_parent; /* Parent CTF container (if any). */
|
|
|
|
const char *ctf_parlabel; /* Label in parent container (if any). */
|
|
|
|
const char *ctf_parname; /* Basename of parent (if any). */
|
|
|
|
char *ctf_dynparname; /* Dynamically allocated name of parent. */
|
|
|
|
uint32_t ctf_parmax; /* Highest type ID of a parent type. */
|
|
|
|
uint32_t ctf_refcnt; /* Reference count (for parent links). */
|
|
|
|
uint32_t ctf_flags; /* Libctf flags (see below). */
|
|
|
|
int ctf_errno; /* Error code for most recent error. */
|
|
|
|
int ctf_version; /* CTF data version. */
|
|
|
|
ctf_dynhash_t *ctf_dthash; /* Hash of dynamic type definitions. */
|
|
|
|
ctf_dynhash_t *ctf_dtbyname; /* DTDs, indexed by name. */
|
|
|
|
ctf_list_t ctf_dtdefs; /* List of dynamic type definitions. */
|
|
|
|
ctf_dynhash_t *ctf_dvhash; /* Hash of dynamic variable mappings. */
|
|
|
|
ctf_list_t ctf_dvdefs; /* List of dynamic variable definitions. */
|
|
|
|
unsigned long ctf_dtnextid; /* Next dynamic type id to assign. */
|
|
|
|
unsigned long ctf_dtoldid; /* Oldest id that has been committed. */
|
|
|
|
unsigned long ctf_snapshots; /* ctf_snapshot() plus ctf_update() count. */
|
|
|
|
unsigned long ctf_snapshot_lu; /* ctf_snapshot() call count at last update. */
|
|
|
|
ctf_archive_t *ctf_archive; /* Archive this ctf_file_t came from. */
|
|
|
|
char *ctf_tmp_typeslice; /* Storage for slicing up type names. */
|
|
|
|
size_t ctf_tmp_typeslicelen; /* Size of the typeslice. */
|
|
|
|
void *ctf_specific; /* Data for ctf_get/setspecific(). */
|
|
|
|
};
|
|
|
|
|
libctf: mmappable archives
If you need to store a large number of CTF containers somewhere, this
provides a dedicated facility for doing so: an mmappable archive format
like a very simple tar or ar without all the system-dependent format
horrors or need for heavy file copying, with built-in compression of
files above a particular size threshold.
libctf automatically mmap()s uncompressed elements of these archives, or
uncompresses them, as needed. (If the platform does not support mmap(),
copying into dynamically-allocated buffers is used.)
Archive iteration operations are partitioned into raw and non-raw
forms. Raw operations pass thhe raw archive contents to the callback:
non-raw forms open each member with ctf_bufopen() and pass the resulting
ctf_file_t to the iterator instead. This lets you manipulate the raw
data in the archive, or the contents interpreted as a CTF file, as
needed.
It is not yet known whether we will store CTF archives in a linked ELF
object in one of these (akin to debugdata) or whether they'll get one
section per TU plus one parent container for types shared between them.
(In the case of ELF objects with very large numbers of TUs, an archive
of all of them would seem preferable, so we might just use an archive,
and add lzma support so you can assume that .gnu_debugdata and .ctf are
compressed using the same algorithm if both are present.)
To make usage easier, the ctf_archive_t is not the on-disk
representation but an abstraction over both ctf_file_t's and archives of
many ctf_file_t's: users see both CTF archives and raw CTF files as
ctf_archive_t's upon opening, the only difference being that a raw CTF
file has only a single "archive member", named ".ctf" (the default if a
null pointer is passed in as the name). The next commit will make use
of this facility, in addition to providing the public interface to
actually open archives. (In the future, it should be possible to have
all CTF sections in an ELF file appear as an "archive" in the same
fashion.)
This machinery is also used to allow library-internal creators of
ctf_archive_t's (such as the next commit) to stash away an ELF string
and symbol table, so that all opens of members in a given archive will
use them. This lets CTF archives exploit the ELF string and symbol
table just like raw CTF files can.
(All this leads to somewhat confusing type naming. The ctf_archive_t is
a typedef for the opaque internal type, struct ctf_archive_internal: the
non-internal "struct ctf_archive" is the on-disk structure meant for
other libraries manipulating CTF files. It is probably clearest to use
the struct name for struct ctf_archive_internal inside the program, and
the typedef names outside.)
libctf/
* ctf-archive.c: New.
* ctf-impl.h (ctf_archive_internal): New type.
(ctf_arc_open_internal): New declaration.
(ctf_arc_bufopen): Likewise.
(ctf_arc_close_internal): Likewise.
include/
* ctf.h (CTFA_MAGIC): New.
(struct ctf_archive): New.
(struct ctf_archive_modent): Likewise.
* ctf-api.h (ctf_archive_member_f): New.
(ctf_archive_raw_member_f): Likewise.
(ctf_arc_write): Likewise.
(ctf_arc_close): Likewise.
(ctf_arc_open_by_name): Likewise.
(ctf_archive_iter): Likewise.
(ctf_archive_raw_iter): Likewise.
(ctf_get_arc): Likewise.
2019-04-24 12:30:17 +02:00
|
|
|
/* An abstraction over both a ctf_file_t and a ctf_archive_t. */
|
|
|
|
|
|
|
|
struct ctf_archive_internal
|
|
|
|
{
|
|
|
|
int ctfi_is_archive;
|
|
|
|
ctf_file_t *ctfi_file;
|
|
|
|
struct ctf_archive *ctfi_archive;
|
|
|
|
ctf_sect_t ctfi_symsect;
|
|
|
|
ctf_sect_t ctfi_strsect;
|
|
|
|
void *ctfi_data;
|
libctf: ELF file opening via BFD
These functions let you open an ELF file with a customarily-named CTF
section in it, automatically opening the CTF file or archive and
associating the symbol and string tables in the ELF file with the CTF
container, so that you can look up the types of symbols in the ELF file
via ctf_lookup_by_symbol(), and so that strings can be shared between
the ELF file and CTF container, to save space.
It uses BFD machinery to do so. This has now been lightly tested and
seems to work. In particular, if you already have a bfd you can pass
it in to ctf_bfdopen(), and if you want a bfd made for you you can
call ctf_open() or ctf_fdopen(), optionally specifying a target (or
try once without a target and then again with one if you get
ECTF_BFD_AMBIGUOUS back).
We use a forward declaration for the struct bfd in ctf-api.h, so that
ctf-api.h users are not required to pull in <bfd.h>. (This is mostly
for the sake of readelf.)
libctf/
* ctf-open-bfd.c: New file.
* ctf-open.c (ctf_close): New.
* ctf-impl.h: Include bfd.h.
(ctf_file): New members ctf_data_mmapped, ctf_data_mmapped_len.
(ctf_archive_internal): New members ctfi_abfd, ctfi_data,
ctfi_bfd_close.
(ctf_bfdopen_ctfsect): New declaration.
(_CTF_SECTION): likewise.
include/
* ctf-api.h (struct bfd): New forward.
(ctf_fdopen): New.
(ctf_bfdopen): Likewise.
(ctf_open): Likewise.
(ctf_arc_open): Likewise.
2019-04-24 11:46:39 +02:00
|
|
|
bfd *ctfi_abfd; /* Optional source of section data. */
|
|
|
|
void (*ctfi_bfd_close) (struct ctf_archive_internal *);
|
libctf: mmappable archives
If you need to store a large number of CTF containers somewhere, this
provides a dedicated facility for doing so: an mmappable archive format
like a very simple tar or ar without all the system-dependent format
horrors or need for heavy file copying, with built-in compression of
files above a particular size threshold.
libctf automatically mmap()s uncompressed elements of these archives, or
uncompresses them, as needed. (If the platform does not support mmap(),
copying into dynamically-allocated buffers is used.)
Archive iteration operations are partitioned into raw and non-raw
forms. Raw operations pass thhe raw archive contents to the callback:
non-raw forms open each member with ctf_bufopen() and pass the resulting
ctf_file_t to the iterator instead. This lets you manipulate the raw
data in the archive, or the contents interpreted as a CTF file, as
needed.
It is not yet known whether we will store CTF archives in a linked ELF
object in one of these (akin to debugdata) or whether they'll get one
section per TU plus one parent container for types shared between them.
(In the case of ELF objects with very large numbers of TUs, an archive
of all of them would seem preferable, so we might just use an archive,
and add lzma support so you can assume that .gnu_debugdata and .ctf are
compressed using the same algorithm if both are present.)
To make usage easier, the ctf_archive_t is not the on-disk
representation but an abstraction over both ctf_file_t's and archives of
many ctf_file_t's: users see both CTF archives and raw CTF files as
ctf_archive_t's upon opening, the only difference being that a raw CTF
file has only a single "archive member", named ".ctf" (the default if a
null pointer is passed in as the name). The next commit will make use
of this facility, in addition to providing the public interface to
actually open archives. (In the future, it should be possible to have
all CTF sections in an ELF file appear as an "archive" in the same
fashion.)
This machinery is also used to allow library-internal creators of
ctf_archive_t's (such as the next commit) to stash away an ELF string
and symbol table, so that all opens of members in a given archive will
use them. This lets CTF archives exploit the ELF string and symbol
table just like raw CTF files can.
(All this leads to somewhat confusing type naming. The ctf_archive_t is
a typedef for the opaque internal type, struct ctf_archive_internal: the
non-internal "struct ctf_archive" is the on-disk structure meant for
other libraries manipulating CTF files. It is probably clearest to use
the struct name for struct ctf_archive_internal inside the program, and
the typedef names outside.)
libctf/
* ctf-archive.c: New.
* ctf-impl.h (ctf_archive_internal): New type.
(ctf_arc_open_internal): New declaration.
(ctf_arc_bufopen): Likewise.
(ctf_arc_close_internal): Likewise.
include/
* ctf.h (CTFA_MAGIC): New.
(struct ctf_archive): New.
(struct ctf_archive_modent): Likewise.
* ctf-api.h (ctf_archive_member_f): New.
(ctf_archive_raw_member_f): Likewise.
(ctf_arc_write): Likewise.
(ctf_arc_close): Likewise.
(ctf_arc_open_by_name): Likewise.
(ctf_archive_iter): Likewise.
(ctf_archive_raw_iter): Likewise.
(ctf_get_arc): Likewise.
2019-04-24 12:30:17 +02:00
|
|
|
};
|
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
/* Return x rounded up to an alignment boundary.
|
|
|
|
eg, P2ROUNDUP(0x1234, 0x100) == 0x1300 (0x13*align)
|
|
|
|
eg, P2ROUNDUP(0x5600, 0x100) == 0x5600 (0x56*align) */
|
|
|
|
#define P2ROUNDUP(x, align) (-(-(x) & -(align)))
|
|
|
|
|
|
|
|
/* * If an offs is not aligned already then round it up and align it. */
|
|
|
|
#define LCTF_ALIGN_OFFS(offs, align) ((offs + (align - 1)) & ~(align - 1))
|
|
|
|
|
|
|
|
#define LCTF_TYPE_ISPARENT(fp, id) ((id) <= fp->ctf_parmax)
|
|
|
|
#define LCTF_TYPE_ISCHILD(fp, id) ((id) > fp->ctf_parmax)
|
|
|
|
#define LCTF_TYPE_TO_INDEX(fp, id) ((id) & (fp->ctf_parmax))
|
|
|
|
#define LCTF_INDEX_TO_TYPE(fp, id, child) (child ? ((id) | (fp->ctf_parmax+1)) : \
|
|
|
|
(id))
|
|
|
|
|
|
|
|
#define LCTF_INDEX_TO_TYPEPTR(fp, i) \
|
|
|
|
((ctf_type_t *)((uintptr_t)(fp)->ctf_buf + (fp)->ctf_txlate[(i)]))
|
|
|
|
|
|
|
|
#define LCTF_INFO_KIND(fp, info) ((fp)->ctf_fileops->ctfo_get_kind(info))
|
|
|
|
#define LCTF_INFO_ISROOT(fp, info) ((fp)->ctf_fileops->ctfo_get_root(info))
|
|
|
|
#define LCTF_INFO_VLEN(fp, info) ((fp)->ctf_fileops->ctfo_get_vlen(info))
|
|
|
|
#define LCTF_VBYTES(fp, kind, size, vlen) \
|
|
|
|
((fp)->ctf_fileops->ctfo_get_vbytes(kind, size, vlen))
|
|
|
|
|
|
|
|
static inline ssize_t ctf_get_ctt_size (const ctf_file_t *fp,
|
|
|
|
const ctf_type_t *tp,
|
|
|
|
ssize_t *sizep,
|
|
|
|
ssize_t *incrementp)
|
|
|
|
{
|
|
|
|
return (fp->ctf_fileops->ctfo_get_ctt_size (fp, tp, sizep, incrementp));
|
|
|
|
}
|
|
|
|
|
|
|
|
#define LCTF_CHILD 0x0001 /* CTF container is a child */
|
|
|
|
#define LCTF_RDWR 0x0002 /* CTF container is writable */
|
|
|
|
#define LCTF_DIRTY 0x0004 /* CTF container has been modified */
|
|
|
|
|
|
|
|
extern const ctf_type_t *ctf_lookup_by_id (ctf_file_t **, ctf_id_t);
|
|
|
|
|
2019-04-23 23:12:16 +02:00
|
|
|
typedef unsigned int (*ctf_hash_fun) (const void *ptr);
|
|
|
|
extern unsigned int ctf_hash_integer (const void *ptr);
|
|
|
|
extern unsigned int ctf_hash_string (const void *ptr);
|
|
|
|
|
|
|
|
typedef int (*ctf_hash_eq_fun) (const void *, const void *);
|
|
|
|
extern int ctf_hash_eq_integer (const void *, const void *);
|
|
|
|
extern int ctf_hash_eq_string (const void *, const void *);
|
|
|
|
|
|
|
|
typedef void (*ctf_hash_free_fun) (void *);
|
|
|
|
|
2019-06-27 14:30:22 +02:00
|
|
|
typedef void (*ctf_hash_iter_f) (void *key, void *value, void *arg);
|
|
|
|
typedef int (*ctf_hash_iter_remove_f) (void *key, void *value, void *arg);
|
|
|
|
|
2019-04-23 23:12:16 +02:00
|
|
|
extern ctf_hash_t *ctf_hash_create (unsigned long, ctf_hash_fun, ctf_hash_eq_fun);
|
|
|
|
extern int ctf_hash_insert_type (ctf_hash_t *, ctf_file_t *, uint32_t, uint32_t);
|
|
|
|
extern int ctf_hash_define_type (ctf_hash_t *, ctf_file_t *, uint32_t, uint32_t);
|
|
|
|
extern ctf_id_t ctf_hash_lookup_type (ctf_hash_t *, ctf_file_t *, const char *);
|
|
|
|
extern uint32_t ctf_hash_size (const ctf_hash_t *);
|
|
|
|
extern void ctf_hash_destroy (ctf_hash_t *);
|
|
|
|
|
|
|
|
extern ctf_dynhash_t *ctf_dynhash_create (ctf_hash_fun, ctf_hash_eq_fun,
|
|
|
|
ctf_hash_free_fun, ctf_hash_free_fun);
|
|
|
|
extern int ctf_dynhash_insert (ctf_dynhash_t *, void *, void *);
|
|
|
|
extern void ctf_dynhash_remove (ctf_dynhash_t *, const void *);
|
|
|
|
extern void *ctf_dynhash_lookup (ctf_dynhash_t *, const void *);
|
|
|
|
extern void ctf_dynhash_destroy (ctf_dynhash_t *);
|
2019-06-27 14:30:22 +02:00
|
|
|
extern void ctf_dynhash_iter (ctf_dynhash_t *, ctf_hash_iter_f, void *);
|
|
|
|
extern void ctf_dynhash_iter_remove (ctf_dynhash_t *, ctf_hash_iter_remove_f,
|
|
|
|
void *);
|
2019-04-23 23:12:16 +02:00
|
|
|
|
2019-04-23 22:45:30 +02:00
|
|
|
#define ctf_list_prev(elem) ((void *)(((ctf_list_t *)(elem))->l_prev))
|
|
|
|
#define ctf_list_next(elem) ((void *)(((ctf_list_t *)(elem))->l_next))
|
|
|
|
|
|
|
|
extern void ctf_list_append (ctf_list_t *, void *);
|
|
|
|
extern void ctf_list_prepend (ctf_list_t *, void *);
|
|
|
|
extern void ctf_list_delete (ctf_list_t *, void *);
|
|
|
|
|
2019-06-19 13:14:16 +02:00
|
|
|
extern int ctf_dtd_insert (ctf_file_t *, ctf_dtdef_t *);
|
2019-04-23 23:24:13 +02:00
|
|
|
extern void ctf_dtd_delete (ctf_file_t *, ctf_dtdef_t *);
|
|
|
|
extern ctf_dtdef_t *ctf_dtd_lookup (const ctf_file_t *, ctf_id_t);
|
|
|
|
extern ctf_dtdef_t *ctf_dynamic_type (const ctf_file_t *, ctf_id_t);
|
|
|
|
|
2019-06-19 13:14:16 +02:00
|
|
|
extern int ctf_dvd_insert (ctf_file_t *, ctf_dvdef_t *);
|
2019-04-23 23:24:13 +02:00
|
|
|
extern void ctf_dvd_delete (ctf_file_t *, ctf_dvdef_t *);
|
|
|
|
extern ctf_dvdef_t *ctf_dvd_lookup (const ctf_file_t *, const char *);
|
|
|
|
|
2019-04-24 12:03:37 +02:00
|
|
|
extern void ctf_decl_init (ctf_decl_t *);
|
|
|
|
extern void ctf_decl_fini (ctf_decl_t *);
|
|
|
|
extern void ctf_decl_push (ctf_decl_t *, ctf_file_t *, ctf_id_t);
|
|
|
|
|
|
|
|
_libctf_printflike_ (2, 3)
|
|
|
|
extern void ctf_decl_sprintf (ctf_decl_t *, const char *, ...);
|
|
|
|
extern char *ctf_decl_buf (ctf_decl_t *cd);
|
|
|
|
|
2019-04-23 22:45:30 +02:00
|
|
|
extern const char *ctf_strraw (ctf_file_t *, uint32_t);
|
|
|
|
extern const char *ctf_strptr (ctf_file_t *, uint32_t);
|
libctf: deduplicate and sort the string table
ctf.h states:
> [...] the CTF string table does not contain any duplicated strings.
Unfortunately this is entirely untrue: libctf has before now made no
attempt whatsoever to deduplicate the string table. It computes the
string table's length on the fly as it adds new strings to the dynamic
CTF file, and ctf_update() just writes each string to the table and
notes the current write position as it traverses the dynamic CTF file's
data structures and builds the final CTF buffer. There is no global
view of the strings and no deduplication.
Fix this by erasing the ctf_dtvstrlen dead-reckoning length, and adding
a new dynhash table ctf_str_atoms that maps unique strings to a list
of references to those strings: a reference is a simple uint32_t * to
some value somewhere in the under-construction CTF buffer that needs
updating to note the string offset when the strtab is laid out.
Adding a string is now a simple matter of calling ctf_str_add_ref(),
which adds a new atom to the atoms table, if one doesn't already exist,
and adding the location of the reference to this atom to the refs list
attached to the atom: this works reliably as long as one takes care to
only call ctf_str_add_ref() once the final location of the offset is
known (so you can't call it on a temporary structure and then memcpy()
that structure into place in the CTF buffer, because the ref will still
point to the old location: ctf_update() changes accordingly).
Generating the CTF string table is a matter of calling
ctf_str_write_strtab(), which counts the length and number of elements
in the atoms table using the ctf_dynhash_iter() function we just added,
populating an array of pointers into the atoms table and sorting it into
order (to help compressors), then traversing this table and emitting it,
updating the refs to each atom as we go. The only complexity here is
arranging to keep the null string at offset zero, since a lot of code in
libctf depends on being able to leave strtab references at 0 to indicate
'no name'. Once the table is constructed and the refs updated, we know
how long it is, so we can realloc() the partial CTF buffer we allocated
earlier and can copy the table on to the end of it (and purge the refs
because they're not needed any more and have been invalidated by the
realloc() call in any case).
The net effect of all this is a reduction in uncompressed strtab sizes
of about 30% (perhaps a quarter to a half of all strings across the
Linux kernel are eliminated as duplicates). Of course, duplicated
strings are highly redundant, so the space saving after compression is
only about 20%: when the other non-strtab sections are factored in, CTF
sizes shrink by about 10%.
No change in externally-visible API or file format (other than the
reduction in pointless redundancy).
libctf/
* ctf-impl.h: (struct ctf_strs_writable): New, non-const version of
struct ctf_strs.
(struct ctf_dtdef): Note that dtd_data.ctt_name is unpopulated.
(struct ctf_str_atom): New, disambiguated single string.
(struct ctf_str_atom_ref): New, points to some other location that
references this string's offset.
(struct ctf_file): New members ctf_str_atoms and ctf_str_num_refs.
Remove member ctf_dtvstrlen: we no longer track the total strlen
as we add strings.
(ctf_str_create_atoms): Declare new function in ctf-string.c.
(ctf_str_free_atoms): Likewise.
(ctf_str_add): Likewise.
(ctf_str_add_ref): Likewise.
(ctf_str_purge_refs): Likewise.
(ctf_str_write_strtab): Likewise.
(ctf_realloc): Declare new function in ctf-util.c.
* ctf-open.c (ctf_bufopen): Create the atoms table.
(ctf_file_close): Destroy it.
* ctf-create.c (ctf_update): Copy-and-free it on update. No longer
special-case the position of the parname string. Construct the
strtab by calling ctf_str_add_ref and ctf_str_write_strtab after the
rest of each buffer element is constructed, not via open-coding:
realloc the CTF buffer and append the strtab to it. No longer
maintain ctf_dtvstrlen. Sort the variable entry table later, after
strtab construction.
(ctf_copy_membnames): Remove: integrated into ctf_copy_{s,l,e}members.
(ctf_copy_smembers): Drop the string offset: call ctf_str_add_ref
after buffer element construction instead.
(ctf_copy_lmembers): Likewise.
(ctf_copy_emembers): Likewise.
(ctf_create): No longer maintain the ctf_dtvstrlen.
(ctf_dtd_delete): Likewise.
(ctf_dvd_delete): Likewise.
(ctf_add_generic): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(membadd): Likewise.
* ctf-util.c (ctf_realloc): New, wrapper around realloc that aborts
if there are active ctf_str_num_refs.
(ctf_strraw): Move to ctf-string.c.
(ctf_strptr): Likewise.
* ctf-string.c: New file, strtab manipulation.
* Makefile.am (libctf_a_SOURCES): Add it.
* Makefile.in: Regenerate.
2019-06-27 14:51:10 +02:00
|
|
|
extern int ctf_str_create_atoms (ctf_file_t *);
|
|
|
|
extern void ctf_str_free_atoms (ctf_file_t *);
|
|
|
|
extern const char *ctf_str_add (ctf_file_t *, const char *);
|
|
|
|
extern const char *ctf_str_add_ref (ctf_file_t *, const char *, uint32_t *);
|
|
|
|
extern void ctf_str_rollback (ctf_file_t *, ctf_snapshot_id_t);
|
|
|
|
extern void ctf_str_purge_refs (ctf_file_t *);
|
|
|
|
extern ctf_strs_writable_t ctf_str_write_strtab (ctf_file_t *);
|
2019-04-23 22:45:30 +02:00
|
|
|
|
libctf: mmappable archives
If you need to store a large number of CTF containers somewhere, this
provides a dedicated facility for doing so: an mmappable archive format
like a very simple tar or ar without all the system-dependent format
horrors or need for heavy file copying, with built-in compression of
files above a particular size threshold.
libctf automatically mmap()s uncompressed elements of these archives, or
uncompresses them, as needed. (If the platform does not support mmap(),
copying into dynamically-allocated buffers is used.)
Archive iteration operations are partitioned into raw and non-raw
forms. Raw operations pass thhe raw archive contents to the callback:
non-raw forms open each member with ctf_bufopen() and pass the resulting
ctf_file_t to the iterator instead. This lets you manipulate the raw
data in the archive, or the contents interpreted as a CTF file, as
needed.
It is not yet known whether we will store CTF archives in a linked ELF
object in one of these (akin to debugdata) or whether they'll get one
section per TU plus one parent container for types shared between them.
(In the case of ELF objects with very large numbers of TUs, an archive
of all of them would seem preferable, so we might just use an archive,
and add lzma support so you can assume that .gnu_debugdata and .ctf are
compressed using the same algorithm if both are present.)
To make usage easier, the ctf_archive_t is not the on-disk
representation but an abstraction over both ctf_file_t's and archives of
many ctf_file_t's: users see both CTF archives and raw CTF files as
ctf_archive_t's upon opening, the only difference being that a raw CTF
file has only a single "archive member", named ".ctf" (the default if a
null pointer is passed in as the name). The next commit will make use
of this facility, in addition to providing the public interface to
actually open archives. (In the future, it should be possible to have
all CTF sections in an ELF file appear as an "archive" in the same
fashion.)
This machinery is also used to allow library-internal creators of
ctf_archive_t's (such as the next commit) to stash away an ELF string
and symbol table, so that all opens of members in a given archive will
use them. This lets CTF archives exploit the ELF string and symbol
table just like raw CTF files can.
(All this leads to somewhat confusing type naming. The ctf_archive_t is
a typedef for the opaque internal type, struct ctf_archive_internal: the
non-internal "struct ctf_archive" is the on-disk structure meant for
other libraries manipulating CTF files. It is probably clearest to use
the struct name for struct ctf_archive_internal inside the program, and
the typedef names outside.)
libctf/
* ctf-archive.c: New.
* ctf-impl.h (ctf_archive_internal): New type.
(ctf_arc_open_internal): New declaration.
(ctf_arc_bufopen): Likewise.
(ctf_arc_close_internal): Likewise.
include/
* ctf.h (CTFA_MAGIC): New.
(struct ctf_archive): New.
(struct ctf_archive_modent): Likewise.
* ctf-api.h (ctf_archive_member_f): New.
(ctf_archive_raw_member_f): Likewise.
(ctf_arc_write): Likewise.
(ctf_arc_close): Likewise.
(ctf_arc_open_by_name): Likewise.
(ctf_archive_iter): Likewise.
(ctf_archive_raw_iter): Likewise.
(ctf_get_arc): Likewise.
2019-04-24 12:30:17 +02:00
|
|
|
extern struct ctf_archive *ctf_arc_open_internal (const char *, int *);
|
|
|
|
extern struct ctf_archive *ctf_arc_bufopen (const void *, size_t, int *);
|
|
|
|
extern void ctf_arc_close_internal (struct ctf_archive *);
|
2019-04-23 22:45:30 +02:00
|
|
|
extern void *ctf_set_open_errno (int *, int);
|
libctf: fix a number of build problems found on Solaris and NetBSD
- Use of nonportable <endian.h>
- Use of qsort_r
- Use of zlib without appropriate magic to pull in the binutils zlib
- Use of off64_t without checking (fixed by dropping the unused fields
that need off64_t entirely)
- signedness problems due to long being too short a type on 32-bit
platforms: ctf_id_t is now 'unsigned long', and CTF_ERR must be
used only for functions that return ctf_id_t
- One lingering use of bzero() and of <sys/errno.h>
All fixed, using code from gnulib where possible.
Relatedly, set cts_size in a couple of places it was missed
(string table and symbol table loading upon ctf_bfdopen()).
binutils/
* objdump.c (make_ctfsect): Drop cts_type, cts_flags, and
cts_offset.
* readelf.c (shdr_to_ctf_sect): Likewise.
include/
* ctf-api.h (ctf_sect_t): Drop cts_type, cts_flags, and cts_offset.
(ctf_id_t): This is now an unsigned type.
(CTF_ERR): Cast it to ctf_id_t. Note that it should only be used
for ctf_id_t-returning functions.
libctf/
* Makefile.am (ZLIB): New.
(ZLIBINC): Likewise.
(AM_CFLAGS): Use them.
(libctf_a_LIBADD): New, for LIBOBJS.
* configure.ac: Check for zlib, endian.h, and qsort_r.
* ctf-endian.h: New, providing htole64 and le64toh.
* swap.h: Code style fixes.
(bswap_identity_64): New.
* qsort_r.c: New, from gnulib (with one added #include).
* ctf-decls.h: New, providing a conditional qsort_r declaration,
and unconditional definitions of MIN and MAX.
* ctf-impl.h: Use it. Do not use <sys/errno.h>.
(ctf_set_errno): Now returns unsigned long.
* ctf-util.c (ctf_set_errno): Adjust here too.
* ctf-archive.c: Use ctf-endian.h.
(ctf_arc_open_by_offset): Use memset, not bzero. Drop cts_type,
cts_flags and cts_offset.
(ctf_arc_write): Drop debugging dependent on the size of off_t.
* ctf-create.c: Provide a definition of roundup if not defined.
(ctf_create): Drop cts_type, cts_flags and cts_offset.
(ctf_add_reftype): Do not check if type IDs are below zero.
(ctf_add_slice): Likewise.
(ctf_add_typedef): Likewise.
(ctf_add_member_offset): Cast error-returning ssize_t's to size_t
when known error-free. Drop CTF_ERR usage for functions returning
int.
(ctf_add_member_encoded): Drop CTF_ERR usage for functions returning
int.
(ctf_add_variable): Likewise.
(enumcmp): Likewise.
(enumadd): Likewise.
(membcmp): Likewise.
(ctf_add_type): Likewise. Cast error-returning ssize_t's to size_t
when known error-free.
* ctf-dump.c (ctf_is_slice): Drop CTF_ERR usage for functions
returning int: use CTF_ERR for functions returning ctf_type_id.
(ctf_dump_label): Likewise.
(ctf_dump_objts): Likewise.
* ctf-labels.c (ctf_label_topmost): Likewise.
(ctf_label_iter): Likewise.
(ctf_label_info): Likewise.
* ctf-lookup.c (ctf_func_args): Likewise.
* ctf-open.c (upgrade_types): Cast to size_t where appropriate.
(ctf_bufopen): Likewise. Use zlib types as needed.
* ctf-types.c (ctf_member_iter): Drop CTF_ERR usage for functions
returning int.
(ctf_enum_iter): Likewise.
(ctf_type_size): Likewise.
(ctf_type_align): Likewise. Cast to size_t where appropriate.
(ctf_type_kind_unsliced): Likewise.
(ctf_type_kind): Likewise.
(ctf_type_encoding): Likewise.
(ctf_member_info): Likewise.
(ctf_array_info): Likewise.
(ctf_enum_value): Likewise.
(ctf_type_rvisit): Likewise.
* ctf-open-bfd.c (ctf_bfdopen): Drop cts_type, cts_flags and
cts_offset.
(ctf_simple_open): Likewise.
(ctf_bfdopen_ctfsect): Likewise. Set cts_size properly.
* Makefile.in: Regenerate.
* aclocal.m4: Likewise.
* config.h: Likewise.
* configure: Likewise.
2019-05-31 11:10:51 +02:00
|
|
|
extern unsigned long ctf_set_errno (ctf_file_t *, int);
|
2019-04-23 22:45:30 +02:00
|
|
|
|
2019-04-23 19:55:27 +02:00
|
|
|
_libctf_malloc_
|
|
|
|
extern void *ctf_mmap (size_t length, size_t offset, int fd);
|
|
|
|
extern void ctf_munmap (void *, size_t);
|
|
|
|
extern ssize_t ctf_pread (int fd, void *buf, ssize_t count, off_t offset);
|
|
|
|
|
|
|
|
_libctf_malloc_
|
|
|
|
extern void *ctf_alloc (size_t);
|
|
|
|
extern void ctf_free (void *);
|
libctf: deduplicate and sort the string table
ctf.h states:
> [...] the CTF string table does not contain any duplicated strings.
Unfortunately this is entirely untrue: libctf has before now made no
attempt whatsoever to deduplicate the string table. It computes the
string table's length on the fly as it adds new strings to the dynamic
CTF file, and ctf_update() just writes each string to the table and
notes the current write position as it traverses the dynamic CTF file's
data structures and builds the final CTF buffer. There is no global
view of the strings and no deduplication.
Fix this by erasing the ctf_dtvstrlen dead-reckoning length, and adding
a new dynhash table ctf_str_atoms that maps unique strings to a list
of references to those strings: a reference is a simple uint32_t * to
some value somewhere in the under-construction CTF buffer that needs
updating to note the string offset when the strtab is laid out.
Adding a string is now a simple matter of calling ctf_str_add_ref(),
which adds a new atom to the atoms table, if one doesn't already exist,
and adding the location of the reference to this atom to the refs list
attached to the atom: this works reliably as long as one takes care to
only call ctf_str_add_ref() once the final location of the offset is
known (so you can't call it on a temporary structure and then memcpy()
that structure into place in the CTF buffer, because the ref will still
point to the old location: ctf_update() changes accordingly).
Generating the CTF string table is a matter of calling
ctf_str_write_strtab(), which counts the length and number of elements
in the atoms table using the ctf_dynhash_iter() function we just added,
populating an array of pointers into the atoms table and sorting it into
order (to help compressors), then traversing this table and emitting it,
updating the refs to each atom as we go. The only complexity here is
arranging to keep the null string at offset zero, since a lot of code in
libctf depends on being able to leave strtab references at 0 to indicate
'no name'. Once the table is constructed and the refs updated, we know
how long it is, so we can realloc() the partial CTF buffer we allocated
earlier and can copy the table on to the end of it (and purge the refs
because they're not needed any more and have been invalidated by the
realloc() call in any case).
The net effect of all this is a reduction in uncompressed strtab sizes
of about 30% (perhaps a quarter to a half of all strings across the
Linux kernel are eliminated as duplicates). Of course, duplicated
strings are highly redundant, so the space saving after compression is
only about 20%: when the other non-strtab sections are factored in, CTF
sizes shrink by about 10%.
No change in externally-visible API or file format (other than the
reduction in pointless redundancy).
libctf/
* ctf-impl.h: (struct ctf_strs_writable): New, non-const version of
struct ctf_strs.
(struct ctf_dtdef): Note that dtd_data.ctt_name is unpopulated.
(struct ctf_str_atom): New, disambiguated single string.
(struct ctf_str_atom_ref): New, points to some other location that
references this string's offset.
(struct ctf_file): New members ctf_str_atoms and ctf_str_num_refs.
Remove member ctf_dtvstrlen: we no longer track the total strlen
as we add strings.
(ctf_str_create_atoms): Declare new function in ctf-string.c.
(ctf_str_free_atoms): Likewise.
(ctf_str_add): Likewise.
(ctf_str_add_ref): Likewise.
(ctf_str_purge_refs): Likewise.
(ctf_str_write_strtab): Likewise.
(ctf_realloc): Declare new function in ctf-util.c.
* ctf-open.c (ctf_bufopen): Create the atoms table.
(ctf_file_close): Destroy it.
* ctf-create.c (ctf_update): Copy-and-free it on update. No longer
special-case the position of the parname string. Construct the
strtab by calling ctf_str_add_ref and ctf_str_write_strtab after the
rest of each buffer element is constructed, not via open-coding:
realloc the CTF buffer and append the strtab to it. No longer
maintain ctf_dtvstrlen. Sort the variable entry table later, after
strtab construction.
(ctf_copy_membnames): Remove: integrated into ctf_copy_{s,l,e}members.
(ctf_copy_smembers): Drop the string offset: call ctf_str_add_ref
after buffer element construction instead.
(ctf_copy_lmembers): Likewise.
(ctf_copy_emembers): Likewise.
(ctf_create): No longer maintain the ctf_dtvstrlen.
(ctf_dtd_delete): Likewise.
(ctf_dvd_delete): Likewise.
(ctf_add_generic): Likewise.
(ctf_add_enumerator): Likewise.
(ctf_add_member_offset): Likewise.
(ctf_add_variable): Likewise.
(membadd): Likewise.
* ctf-util.c (ctf_realloc): New, wrapper around realloc that aborts
if there are active ctf_str_num_refs.
(ctf_strraw): Move to ctf-string.c.
(ctf_strptr): Likewise.
* ctf-string.c: New file, strtab manipulation.
* Makefile.am (libctf_a_SOURCES): Add it.
* Makefile.in: Regenerate.
2019-06-27 14:51:10 +02:00
|
|
|
extern void *ctf_realloc (ctf_file_t *, void *, size_t);
|
2019-04-23 19:55:27 +02:00
|
|
|
|
2019-04-23 22:45:30 +02:00
|
|
|
_libctf_malloc_
|
|
|
|
extern char *ctf_strdup (const char *);
|
|
|
|
extern char *ctf_str_append (char *, const char *);
|
|
|
|
extern const char *ctf_strerror (int);
|
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
extern ctf_id_t ctf_type_resolve_unsliced (ctf_file_t *, ctf_id_t);
|
|
|
|
extern int ctf_type_kind_unsliced (ctf_file_t *, ctf_id_t);
|
|
|
|
|
2019-04-23 19:55:27 +02:00
|
|
|
_libctf_printflike_ (1, 2)
|
|
|
|
extern void ctf_dprintf (const char *, ...);
|
|
|
|
extern void libctf_init_debug (void);
|
|
|
|
|
2019-04-23 22:45:30 +02:00
|
|
|
extern Elf64_Sym *ctf_sym_to_elf64 (const Elf32_Sym *src, Elf64_Sym *dst);
|
2019-04-24 12:15:33 +02:00
|
|
|
extern const char *ctf_lookup_symbol_name (ctf_file_t *fp, unsigned long symidx);
|
2019-04-23 22:45:30 +02:00
|
|
|
|
2019-04-23 23:24:13 +02:00
|
|
|
/* Variables, all underscore-prepended. */
|
|
|
|
|
libctf: ELF file opening via BFD
These functions let you open an ELF file with a customarily-named CTF
section in it, automatically opening the CTF file or archive and
associating the symbol and string tables in the ELF file with the CTF
container, so that you can look up the types of symbols in the ELF file
via ctf_lookup_by_symbol(), and so that strings can be shared between
the ELF file and CTF container, to save space.
It uses BFD machinery to do so. This has now been lightly tested and
seems to work. In particular, if you already have a bfd you can pass
it in to ctf_bfdopen(), and if you want a bfd made for you you can
call ctf_open() or ctf_fdopen(), optionally specifying a target (or
try once without a target and then again with one if you get
ECTF_BFD_AMBIGUOUS back).
We use a forward declaration for the struct bfd in ctf-api.h, so that
ctf-api.h users are not required to pull in <bfd.h>. (This is mostly
for the sake of readelf.)
libctf/
* ctf-open-bfd.c: New file.
* ctf-open.c (ctf_close): New.
* ctf-impl.h: Include bfd.h.
(ctf_file): New members ctf_data_mmapped, ctf_data_mmapped_len.
(ctf_archive_internal): New members ctfi_abfd, ctfi_data,
ctfi_bfd_close.
(ctf_bfdopen_ctfsect): New declaration.
(_CTF_SECTION): likewise.
include/
* ctf-api.h (struct bfd): New forward.
(ctf_fdopen): New.
(ctf_bfdopen): Likewise.
(ctf_open): Likewise.
(ctf_arc_open): Likewise.
2019-04-24 11:46:39 +02:00
|
|
|
extern const char _CTF_SECTION[]; /* name of CTF ELF section */
|
2019-04-23 23:24:13 +02:00
|
|
|
extern const char _CTF_NULLSTR[]; /* empty string */
|
|
|
|
|
2019-04-24 12:26:42 +02:00
|
|
|
extern int _libctf_version; /* library client version */
|
2019-04-23 19:55:27 +02:00
|
|
|
extern int _libctf_debug; /* debugging messages enabled */
|
|
|
|
|
|
|
|
#ifdef __cplusplus
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#endif /* _CTF_IMPL_H */
|