For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the namespace->name case we get the bonus of removing another
user of dwarves__active_loader.
This covers unions, enums, structs and classes.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the class_member->name case we get the bonus of removing another
user of dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the parameter->name case we get the bonus of removing a user of
dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the base_type->name case we get the bonus of removing some more
functions related base types and a user of dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It looked for an index into a string table, a string_t, but since for
multithreading we'd be growing the string table while looking up stuff,
we'd be looking at realloc'ed memory, so lets move to stable char
pointer strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It looked for an index into a string table, a string_t, but since for
multithreading we'd be growing the string table while looking up stuff,
we'd be looking at realloc'ed memory, so lets move to stable char
pointer strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the function->name case we get the bonus of removing the need of a
debug_fmt_ops->function() callback receiving the 'cu', just access the
string directly.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Using the same command line option as 'make' and 'ninja': -j.
Unfortunately the argp parser in glibc expects a '=' for single letter
args, so it is:
$ pahole -j=10
or, like with 'make':
$ make --jobs=10
This is unwired at the moment, probably I'll reorder the patch, but for
testing, add it first.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Paving the way for having multiple threads creating CUs and possibly
adding them to cus->cus.
The mutex will also be used to ask elfutils-libdwfl to read the next CU,
after a thread finishes reading one.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that we can get rid of that global base_btf and use the right way
to pass load configuration to the format loaders.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When the CTF and later the BTF loaders were implemented they didn't use
obstacks, and then over time some functions, like type__delete(),
class__delete(), enumeration__delete() were shared, which can lead to
crashes by corrupting the obstack by not following its requirements or
to leaks, to avoid such corruption, stop using it.
There is a penalty, but I think its not worth the complexity to keep
using it.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
support it in a way that makes the output be the same for both cases:
$ gcc -gdwarf-4 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int a:1; /* 0: 0 8 */
long int b:1; /* 0: 1 8 */
long int c:1; /* 0: 2 8 */
/* XXX 29 bits hole, try to pack */
/* Bitfield combined with next fields */
int after_bitfield; /* 4 4 */
/* size: 8, cachelines: 1, members: 4 */
/* sum members: 4 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* last cacheline: 8 bytes */
};
$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int a:1; /* 0: 0 8 */
long int b:1; /* 0: 1 8 */
long int c:1; /* 0: 2 8 */
/* XXX 29 bits hole, try to pack */
/* Bitfield combined with next fields */
int after_bitfield; /* 4 4 */
/* size: 8, cachelines: 1, members: 4 */
/* sum members: 4 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* last cacheline: 8 bytes */
};
$
Now with an integer before the bitfield:
$ cat examples/dwarf5/bf.c
struct pea {
int before_bitfield;
long a:1, b:1, c:1;
int after_bitfield;
} p;
$ gcc -gdwarf-4 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
int before_bitfield; /* 0 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 0:32 8 */
long int b:1; /* 0:33 8 */
long int c:1; /* 0:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 8 4 */
/* size: 16, cachelines: 1, members: 5 */
/* sum members: 8 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 16 bytes */
};
$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
int before_bitfield; /* 0 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 0:32 8 */
long int b:1; /* 0:33 8 */
long int c:1; /* 0:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 8 4 */
/* size: 16, cachelines: 1, members: 5 */
/* sum members: 8 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 16 bytes */
};
$
And an array of long integers at the start, before the combination of an
integer with a long integer bitfield:
$ cat examples/dwarf5/bf.c
struct pea {
long array[3];
int before_bitfield;
long a:1, b:1, c:1;
int after_bitfield;
} p;
$ gcc -gdwarf-4 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int array[3]; /* 0 24 */
int before_bitfield; /* 24 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 24:32 8 */
long int b:1; /* 24:33 8 */
long int c:1; /* 24:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 32 4 */
/* size: 40, cachelines: 1, members: 6 */
/* sum members: 32 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 40 bytes */
};
$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int array[3]; /* 0 24 */
int before_bitfield; /* 24 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 24:32 8 */
long int b:1; /* 24:33 8 */
long int c:1; /* 24:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 32 4 */
/* size: 40, cachelines: 1, members: 6 */
/* sum members: 32 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 40 bytes */
};
$
Reported-by: Mark Wielaard <mark@klomp.org>
Tested-by: "Daniel P. Berrangé" <berrange@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1919965
Link: https://lore.kernel.org/dwarves/20210128121122.GA775562@kernel.org/
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In Makefiles we want to do purely numeric comparisions, such as in the
Linux kernel Makefiles and scripts, that have things like this at the
moment:
$ grep PAHOLE */*.sh
scripts/link-vmlinux.sh: if ! [ -x "$(command -v ${PAHOLE})" ]; then
scripts/link-vmlinux.sh: echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
scripts/link-vmlinux.sh: pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/')
scripts/link-vmlinux.sh: echo >&2 "BTF: ${1}: pahole version $(${PAHOLE} --version) is too old, need at least v1.16"
scripts/link-vmlinux.sh: LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
$
So just provide:
$ pahole --numeric_version
118
$
While keeping the --version output for older Makefiles and scripts.
Cc: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This was detected with:
In file included from /home/acme/git/pahole/strings.h:9,
from /usr/include/string.h:432,
from /home/acme/git/pahole/lib/bpf/src/libbpf_common.h:12,
from /home/acme/git/pahole/lib/bpf/src/libbpf.h:20,
from /home/acme/git/pahole/lib/bpf/src/ringbuf.c:20:
/home/acme/git/pahole/lib/bpf/src/btf.h:33:11: error: expected ‘;’ before ‘void’
33 | LIBBPF_API void btf__free(struct btf *btf);
| ^~~~~
| ;
libbpf_common.h has:
#include <string.h>
#ifndef LIBBPF_API
#define LIBBPF_API __attribute__((visibility("default")))
#endif
So before defining LIBBPF_API it includes libc's string.h that in turn
includes pahole's strings.h and now it includes:
#include "lib/bpf/src/btf.h"
That will need the LIBBPF_API, b00m.
So lets just rename pahole's strings.h to pahole_strings.h to avoid this
pitfall.
This patch was moved to before this problem takes place so that we keep
everything bisectable.
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Maintain a pointer to debug_fmt_ops corresponding to currently used debug info
format loader (DWARF, BTF, or CTF), to allow various parts of libdwarves to do
things like resolve string offset to actual string pointer in
a format-agnostic format. This allows to, say, load DWARF debug info, and use
it for BTF generation, without either of them making assumptions about how
strings are actually stored internally.
This is going to be used in the next patch to allow BTF loader and encoder to
use a very different way of storing strings (not a global shared gobuffer).
Committer notes:
Since it is available in multiple object files, add a dwarves__ prefix
namespace and add an extern for it in dwarves.h.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It is found on gcc 8.2 that global percpu variables generate the
following dwarf entry in the cu where the variable is defined[1].
Take the global variable "bpf_prog_active" defined in
kernel/bpf/syscall.c as an example. The debug info for syscall.c has two
dwarf entries for "bpf_prog_active".
> readelf -wi kernel/bpf/syscall.o
0x00013534: DW_TAG_variable
DW_AT_name ("bpf_prog_active")
DW_AT_decl_file
("/data/users/yhs/work/net-next/include/linux/bpf.h")
DW_AT_decl_line (1074)
DW_AT_decl_column (0x01)
DW_AT_type (0x000000d6 "int")
DW_AT_external (true)
DW_AT_declaration (true)
0x00021a25: DW_TAG_variable
DW_AT_specification (0x00013534 "bpf_prog_active")
DW_AT_decl_file
("/data/users/yhs/work/net-next/kernel/bpf/syscall.c")
DW_AT_decl_line (43)
DW_AT_location (DW_OP_addr 0x0)
Note that second DW_TAG_variable entry contains specification that
points to the first entry. This causes problem for btf_encoder when
encoding global variables. The tag generated for the second entry
doesn't have the type and scope info. Therefore the BTF VARs encoded
using this tag has incorrect type_id and scope.
As fix, when creating variable, examine the dwarf entry. If it has
a DW_AT_specification, store the referred struct variable in a 'spec'
field. When encoding VARs, check this 'spec', if it's non-empty, follow
the pointer to use the referred var.
[1] https://www.mail-archive.com/netdev@vger.kernel.org/msg348144.html
Tested: Tested using gcc 4.9 and gcc 8.2. The types and scopes of global
vars are now generated correctly.
[21] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[21102] VAR 'bpf_prog_active' type_id=21, linkage=global-alloc
Signed-off-by: Hao Luo <haoluo@google.com>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I.e. when we get the type= field value, look it up in the type_enum=,
get the struct enumerator and if it is the first time that we're looking
up the string representation of that enumerator, do it and then cache
the results, so that next time we reuse it.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As sometimes we have multiple enums to represent some struct type, like
with perf_event_attr->type, that has 'enum perf_event_type' in Linux's
UAPI and 'enum perf_user_event_type' for purely userspace types, like
the ones synthesized for Intel PT, like PERF_RECORD_AUXTRACE, etc.
This patch just transforms type->type_enum into a list, the support for
multiple types comes next.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For now its all in the same CU, but if not all types needed are found in
a single CU, we'll do a fallback lookup in all of them, so we need to
pair tags with the CUs where they were found.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To know from where to start reading some variable sized type we need
some information from a domain specific tool such as 'perf report -D'
for perf.data files, i.e.:
$ perf report -D -i perf.data
# To display the perf.data header info, please use --header/--header-only options.
#
0x130 [0x20]: event: 79
.
. ... raw event: size 32 bytes
. 0000: 4f 00 00 00 00 00 20 00 1f 00 00 00 00 00 00 00 O..... .........
. 0010: 31 30 9b 3c 00 00 00 00 2e 53 f8 0c 52 8c 01 00 10.<.....S<F8>.R...
0 0x130 [0x20]: PERF_RECORD_TIME_CONV: unhandled!
So we see that 0x130 is where the first PERF_RECORD_* event is located,
and we can use that with --seek_bytes and with some extra info we can
decode variable sized records:
$ pahole --seek_bytes=0x130 -C 'perf_event_header(sizeof=size,type=type,type_enum=perf_event_type)' --skip 2 --count 3 ~/bin/perf < perf.data
{
.type = 0x4a,
.misc = 0,
.size = 0x20,
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 0x28,
},
.id = 0x1,
.path = "/",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 0x30,
},
.id = 0x611,
.path = "/system.slice",
},
$
But if we decode the perf.data file header:
$ pahole ~/bin/perf -C perf_file_header --count 1 < perf.data
{
.magic = 0x32454c4946524550,
.size = 0x68,
.attr_size = 0x88,
.attrs = {
.offset = 0xa8,
.size = 0x88,
},
.data = {
.offset = 0x130,
.size = 0x588,
},
.event_types = {
.offset = 0,
.size = 0,
},
.adds_features = { 0x16717ffc, 0, 0, 0 },
},
$
We see that that 0x130 offset is at field perf_file_header->data.offset,
so lets automate this so that we can try to extract that value and then
use it with --seek_bytes:
$ pahole --header=perf_file_header --seek_bytes='$header.data.offset' -C 'perf_event_header(sizeof=size,type=type,type_enum=perf_event_type)' --skip 2 --count 3 ~/bin/perf < perf.data
pahole: the type enum 'perf_event_type' wasn't found in 'util/header.c'
$
The problem here is that pahole tries to avoid processing all the CUs
(compile units) in a binary, as, so far, it only needed to process one
main type at a time, i.e. the ones in -C/--class.
Now we need multiple types, in the above example we need:
struct perf_event_header
struct pref_file_header
enum perf_event_type
And in this case, the perf binary doesn't have any object/CU that has
all these three types.
To see if the code works we can resort to using BTF, that combines all
types into just one "CU", deduplicating them in the process, so:
$ pahole --btf_encode ~/bin/perf
And now it works:
$ pahole -V -F btf --header=perf_file_header --seek_bytes='$header.data.offset' -C 'perf_event_header(sizeof=size,type=type,type_enum=perf_event_type)' --skip 2 --count 3 ~/bin/perf < perf.data
pahole: sizeof_operator for 'perf_event_header' is 'size'
pahole: type member for 'perf_event_header' is 'type'
pahole: type enum for 'perf_event_header' is 'perf_event_type'
pahole: seek bytes evaluated from --seek_bytes=$header.data.offset is 0x130
{
.type = 0x4a,
.misc = 0,
.size = 0x20,
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 0x28,
},
.id = 0x1,
.path = "/",
},
{
.header = {
.type = PERF_RECORD_CGROUP,
.misc = 0,
.size = 0x30,
},
.id = 0x611,
.path = "/system.slice",
},
$
In the next csets a fallback approach will allow for this to work even
with DWARF, when we'll notice that stdin wasn't consumed and thus we
need to search for the types needed with cus__find_struct_by_name() &
friends.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We'll use it in an upcoming feature, to pretty print fields that albeit
not declared as an enum, have values coming from one.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This is the first step:
$ pahole -V -C 'perf_event_header(sizeof=size,type=type)' ~/bin/perf
pahole: sizeof_operator for 'perf_event_header' is 'size'
pahole: type member for 'perf_event_header' is 'type'
struct perf_event_header {
__u32 type; /* 0 4 */
__u16 misc; /* 4 2 */
__u16 size; /* 6 2 */
/* size: 8, cachelines: 1, members: 3 */
/* last cacheline: 8 bytes */
};
$
Next step is to add another modifier that will point to an enum that
maps the value in perf_event_header->type to a string that can then be
used to lookup the type to be used to pretty print what is after
'sizeof(struct perf_event_header)', i.e. things like:
$ pahole -EC perf_record_mmap ~/bin/perf
struct perf_record_mmap {
struct perf_event_header {
/* typedef __u32 */ unsigned int type; /* 0 4 */
/* typedef __u16 */ short unsigned int misc; /* 4 2 */
/* typedef __u16 */ short unsigned int size; /* 6 2 */
} header; /* 0 8 */
/* typedef __u32 */ unsigned int pid; /* 8 4 */
/* typedef __u32 */ unsigned int tid; /* 12 4 */
/* typedef __u64 */ long long unsigned int start; /* 16 8 */
/* typedef __u64 */ long long unsigned int len; /* 24 8 */
/* typedef __u64 */ long long unsigned int pgoff; /* 32 8 */
char filename[4096]; /* 40 4096 */
/* size: 4136, cachelines: 65, members: 7 */
/* last cacheline: 40 bytes */
};
$
The various validations performed:
$ pahole -V -C 'perf_event_header(sizeof=size,typ=type)' ~/bin/perf
pahole: sizeof_operator for 'perf_event_header' is 'size'
pahole: invalid arg 'typ' in 'perf_event_header(sizeof=size,typ=type)' (known args: sizeof=member, type=member)
$
$ pahole -V -C 'perf_event_header(sizeof=size,type=bla)' ~/bin/perf
pahole: sizeof_operator for 'perf_event_header' is 'size'
pahole: type member for 'perf_event_header' is 'bla'
pahole: the type member 'bla' wasn't found in the 'perf_event_header' type
$
$ pahole -V -C 'perf_event_header(sizeof=size,type=type)' ~/bin/perf
pahole: sizeof_operator for 'perf_event_header' is 'size'
pahole: type member for 'perf_event_header' is 'type'
struct perf_event_header {
__u32 type; /* 0 4 */
__u16 misc; /* 4 2 */
__u16 size; /* 6 2 */
/* size: 8, cachelines: 1, members: 3 */
/* last cacheline: 8 bytes */
};
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This one just allows that to be set, i.e.:
$ pahole -C perf_event_header ~/bin/perf
struct perf_event_header {
__u32 type; /* 0 4 */
__u16 misc; /* 4 2 */
__u16 size; /* 6 2 */
/* size: 8, cachelines: 1, members: 3 */
/* last cacheline: 8 bytes */
};
$ pahole -C 'perf_event_header(sizeof=size)' --seek_bytes=0x348 --count 1 ~/bin/perf < perf.data
{
.type = 0xa,
.misc = 0x2,
.size = 0x68,
},
$
But:
$ pahole -C 'perf_event_header(sizeof=bla)' --seek_bytes=0x348 --count 1 ~/bin/perf < perf.data
pahole: the sizeof member 'bla' not found in the 'perf_event_header' type
$
And:
$ pahole -C 'perf_event_header(size=misc)' --seek_bytes=0x348 --count 1 ~/bin/perf < perf.data
pahole: invalid arg 'size' in 'perf_event_header(size=misc)' (known args: sizeof=member)
$
The next cset will implement sizeof(type) with that modifier, using the
stdin bytes to obtain the size (0x68) in the above case, and then we'll
be able to print a sequence of variable-sized records correctly.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Needed to find the right 'struct ftype' to iterate function arguments
on. This is due to how BTF works with this and how we implemented it, at
some point this can get improved to avoid the need for checking if it is
BTF, doing it in a more format abstracted way, but for now, abstract
this away in the for_each_parameter helpers.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If we have:
inline void process_adjtimex_modes(const struct __kernel_timex * txc, s32 * time_tai)
{
}
And any other struct receiving as a parameter pointers to 'struct
__kerne_timex', then the source file with the above inline, since it
doesn't have any inline expansion, i.e. 'pfunct --compile' generates
just empty function bodies, the types won't be included in the resulting
.o.
Since the original file has the expansions, type types will be there and
thus we will not be able to compare those types, so ask for any 'inline'
to be stripped, so that we keep those types and 'fullcircle' can do its
work.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>