Add interfaces to allow users of dwarf_loader to prepare and pass
per-thread data to steal-functions running on worker threads.
Signed-off-by: Kui-Feng Lee <kuifeng@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Link: https://lore.kernel.org/r/20220126192039.2840752-3-kuifeng@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
LLVM patches ([1] for clang, [2] and [3] for BPF backend)
added support for btf_type_tag attributes. The following is
an example:
[$ ~] cat t.c
#define __tag1 __attribute__((btf_type_tag("tag1")))
#define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu")));
[$ ~] clang -O2 -g -c t.c
[$ ~] llvm-dwarfdump --debug-info t.o
t.o: file format elf64-x86-64
...
0x0000001e: DW_TAG_variable
DW_AT_name ("g")
DW_AT_type (0x00000033 "int **")
DW_AT_external (true)
DW_AT_decl_file ("/home/yhs/t.c")
DW_AT_decl_line (3)
DW_AT_location (DW_OP_addr 0x0)
0x00000033: DW_TAG_pointer_type
DW_AT_type (0x0000004b "int *")
0x00000038: DW_TAG_LLVM_annotation
DW_AT_name ("btf_type_tag")
DW_AT_const_value ("tag1")
0x00000041: DW_TAG_LLVM_annotation
DW_AT_name ("btf_type_tag")
DW_AT_const_value ("tag2")
0x0000004a: NULL
0x0000004b: DW_TAG_pointer_type
DW_AT_type (0x0000005a "int")
0x00000050: DW_TAG_LLVM_annotation
DW_AT_name ("btf_type_tag")
DW_AT_const_value ("tag1")
0x00000059: NULL
0x0000005a: DW_TAG_base_type
DW_AT_name ("int")
DW_AT_encoding (DW_ATE_signed)
DW_AT_byte_size (0x04)
0x00000061: NULL
From the above example, you can see that DW_TAG_pointer_type may contain
one or more DW_TAG_LLVM_annotation btf_type_tag tags. If
DW_TAG_LLVM_annotation tags are present inside DW_TAG_pointer_type, for
BTF encoding, pahole will need to follow [3] to generate a type chain
like:
var -> ptr -> tag2 -> tag1 -> ptr -> tag1 -> int
This patch implemented dwarf_loader support. If a pointer type contains
DW_TAG_LLVM_annotation tags, a new type btf_type_tag_ptr_type will be
created which will store the pointer tag itself and all
DW_TAG_LLVM_annotation tags. During recoding stage, the type chain will
be formed properly based on the above example.
An option "--skip_encoding_btf_type_tag" is added to disable
this new functionality.
[1] https://reviews.llvm.org/D111199
[2] https://reviews.llvm.org/D113222
[3] https://reviews.llvm.org/D113496
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Move DW_TAG_LLVM_annotation definition from dwarf_load.c to dutil.h as
it will be used later for btf_encoder.c. There is no functionality
change for this patch.
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
$ pahole ~/c/split/foo.o
WARNING: DW_TAG_skeleton_unit used, please look for a .dwo file and use it instead.
A future version of pahole will support do this automagically.
$
Reported-by: https://twitter.com/trass3r
Link: https://github.com/acmel/dwarves/issues/23
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that we can get it from user reports, i.e. instead of:
die__process: DW_TAG_compile_unit, DW_TAG_type_unit or DW_TAG_partial_unit expected got INVALID
We now get:
die__process: DW_TAG_compile_unit, DW_TAG_type_unit or DW_TAG_partial_unit expected got INVALID (0x4a)
That we can then look in dwarf.h and notice that there is this new:
DW_TAG_skeleton_unit = 0x4a,
Now lets go support it...
Reported-by: https://twitter.com/trass3r
Link: https://github.com/acmel/dwarves/issues/23
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
llvm commit ([1]) added support for btf_decl_tag attribute
with typedef declaration. Eventually, DW_TAG_LLVM_annotation
tag may appear inside dwarf typedef declaration tag.
kernel support for typedef BTF_KIND_DECL_TAG support
is introduced in [2]. There is no additional libbpf
change needed as the previous libbpf BTF_KIND_DECL_TAG
support is generic enough to cover new typedef use
cases.
This patch added parsing of DW_TAG_LLVM_annotation
for dwarf typedef decl.
$ cat t.c
$ clang -O2 -g -c t.c
$ llvm-dwarfdump --debug-info t.o
......
0x00000033: DW_TAG_typedef
DW_AT_type (0x00000051 "structure ")
DW_AT_name ("__t")
DW_AT_decl_file ("/home/yhs/t.c")
DW_AT_decl_line (3)
0x0000003e: DW_TAG_LLVM_annotation
DW_AT_name ("btf_decl_tag")
DW_AT_const_value ("tag1")
0x00000047: DW_TAG_LLVM_annotation
DW_AT_name ("btf_decl_tag")
DW_AT_const_value ("tag2")
0x00000050: NULL
Previously, pahole will issue a warning if typedef tag
contains any child tag. I removed this warning since
it is not true any more. Note that dwarf standard doesn't
prevent typedef decl tag from having nested tags.
In the future if we need to process any tag inside
typedef tag, we can just add code to process it.
[1] https://reviews.llvm.org/D110127
[2] https://lore.kernel.org/bpf/20211021195628.4018847-1-yhs@fb.com
Signed-off-by: Yonghong Song <yhs@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kernel commit ([1]) renamed btf_tag to btf_decl_tag for uapi btf.h and
libbpf api's. The reason is a new clang attribute, btf_type_tag, is
introduced ([2]). Renaming btf_tag to btf_decl_tag makes it easier to
distinghish from btf_type_tag.
I also pulled in latest libbpf repo since it contains renamed libbpf api
function btf__add_decl_tag().
[1] https://lore.kernel.org/bpf/20211012164838.3345699-1-yhs@fb.com/
[2] https://reviews.llvm.org/D111199
Signed-off-by: Yonghong Song <yhs@fb.com>
[ Minor fixups to cope with --skip_missing ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Variables can be allocated with or without specification, however,
tag__recode_dwarf_type() always tries accessing it, leading to heap read
overflows and subsequent logic bugs.
Fix by introducing a bit that tracks whether or not specification is
present.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Will be called when a thread exits, initially only in the DWARF loader,
so that pahole can call the btf_encoder associated with the exiting
thread to do the dedup as the last step done in parallel.
Then we'll iterate the btf_encoders list and combine everything into the
first btf_encoder instance that gets then written to disk.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Parse the DWARF tag DW_TAG_LLVM_annotation. Only record annotations with
btf_tag name which corresponds to btf_tag attributes in C code. Such
information will be used later by the btf_encoder for BTF conversion.
The LLVM implementation only supports btf_tag annotations on
struct/union, func, func parameter and variable ([1]). So we only check
existence of corresponding DW tags in these places.
A flag "--skip_encoding_btf_tag" is introduced if for whatever reason
this feature needs to be disabled.
[1] https://reviews.llvm.org/D106614
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Link: https://lore.kernel.org/r/20210922021326.2287095-1-yhs@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
But since it is still related to cus processing, remove that arg and
rename it to __cus__load_debug_types().
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For now this will only apply to the dwarf loader, for experimenting as
time passes and kernels grow bigger or with more symbols.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As this isn't present in most types or struct members, which ends up
making dwarf_attr() call libdw_find_attr() that will do a linear search
on all the attributes.
We don't use this in the BTF encoder, so no point in reading that.
This will be used in pahole in the following cset.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that 'pahole --sort -F dwarf' can defer printing all classes to when
it has all of them processed and sorted.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
They call each other, so do the three at once.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that we stopped using string indexes, no need for that, just set
namespace->name with the new class name.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As we'll implement that kabi_prefix thing there and without using global
variables.
This is because we're stopping usage of strings.c, where the kabi_prefix
feature was implemented.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the enumerator->name case we get the bonus of removing the last user
of dwarves__active_loader in the btf_encoder class.
This covers unions, enums, structs and classes.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the namespace->name case we get the bonus of removing another
user of dwarves__active_loader.
This covers unions, enums, structs and classes.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the class_member->name case we get the bonus of removing another
user of dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the parameter->name case we get the bonus of removing a user of
dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kui-Feng Lee
Yonghong Song
Arnaldo Carvalho de Melo
Ilya Leoshkevich