Add interfaces to allow users of dwarf_loader to prepare and pass
per-thread data to steal-functions running on worker threads.
Signed-off-by: Kui-Feng Lee <kuifeng@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Link: https://lore.kernel.org/r/20220126192039.2840752-3-kuifeng@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We use this in both the dwarf_loader.c and in fprintf.c, so define it in
dwarves.h that is included in both.
Reported-by: Nathan Chancellor <nathan@kernel.org>
Link: https://lore.kernel.org/all/YbkTAPn3EEu6BUYR@archlinux-ax161
Cc: Domenico Andreoli <domenico.andreoli@linux.com>
Cc: Douglas RAILLARD <douglas.raillard@arm.com>
Cc: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Jan Engelhardt <jengelh@inai.de>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Matteo Croce <mcroce@microsoft.com>
Cc: Matthias Schwarzott <zzam@gentoo.org>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This reverts commit 7c5e35b63b.
Dropped since it could not cope with recursive types. A new attempt will
be made on 1.24.
Signed-off-by: Douglas RAILLARD <douglas.raillard@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Allow making inner struct enums and union anonymous, so that when using
-E to expand types we don't end up with multiple definitions for
expanded inner structs, allowing the resulting expanded struct to be
compilable.
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
[ Applied it manually to cover some fuzz due to other patches ]
Link: https://lore.kernel.org/all/20211019100724.325570-2-douglas.raillard@arm.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
LLVM patches ([1] for clang, [2] and [3] for BPF backend)
added support for btf_type_tag attributes. The following is
an example:
[$ ~] cat t.c
#define __tag1 __attribute__((btf_type_tag("tag1")))
#define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu")));
[$ ~] clang -O2 -g -c t.c
[$ ~] llvm-dwarfdump --debug-info t.o
t.o: file format elf64-x86-64
...
0x0000001e: DW_TAG_variable
DW_AT_name ("g")
DW_AT_type (0x00000033 "int **")
DW_AT_external (true)
DW_AT_decl_file ("/home/yhs/t.c")
DW_AT_decl_line (3)
DW_AT_location (DW_OP_addr 0x0)
0x00000033: DW_TAG_pointer_type
DW_AT_type (0x0000004b "int *")
0x00000038: DW_TAG_LLVM_annotation
DW_AT_name ("btf_type_tag")
DW_AT_const_value ("tag1")
0x00000041: DW_TAG_LLVM_annotation
DW_AT_name ("btf_type_tag")
DW_AT_const_value ("tag2")
0x0000004a: NULL
0x0000004b: DW_TAG_pointer_type
DW_AT_type (0x0000005a "int")
0x00000050: DW_TAG_LLVM_annotation
DW_AT_name ("btf_type_tag")
DW_AT_const_value ("tag1")
0x00000059: NULL
0x0000005a: DW_TAG_base_type
DW_AT_name ("int")
DW_AT_encoding (DW_ATE_signed)
DW_AT_byte_size (0x04)
0x00000061: NULL
From the above example, you can see that DW_TAG_pointer_type may contain
one or more DW_TAG_LLVM_annotation btf_type_tag tags. If
DW_TAG_LLVM_annotation tags are present inside DW_TAG_pointer_type, for
BTF encoding, pahole will need to follow [3] to generate a type chain
like:
var -> ptr -> tag2 -> tag1 -> ptr -> tag1 -> int
This patch implemented dwarf_loader support. If a pointer type contains
DW_TAG_LLVM_annotation tags, a new type btf_type_tag_ptr_type will be
created which will store the pointer tag itself and all
DW_TAG_LLVM_annotation tags. During recoding stage, the type chain will
be formed properly based on the above example.
An option "--skip_encoding_btf_type_tag" is added to disable
this new functionality.
[1] https://reviews.llvm.org/D111199
[2] https://reviews.llvm.org/D113222
[3] https://reviews.llvm.org/D113496
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Remove the global variable and turn it into a member in struct
conf_fprintf, so that it can be used by other parts of the code.
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We'll use it in the BTF loader.
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
[ Split from a larger patch ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kernel commit ([1]) renamed btf_tag to btf_decl_tag for uapi btf.h and
libbpf api's. The reason is a new clang attribute, btf_type_tag, is
introduced ([2]). Renaming btf_tag to btf_decl_tag makes it easier to
distinghish from btf_type_tag.
I also pulled in latest libbpf repo since it contains renamed libbpf api
function btf__add_decl_tag().
[1] https://lore.kernel.org/bpf/20211012164838.3345699-1-yhs@fb.com/
[2] https://reviews.llvm.org/D111199
Signed-off-by: Yonghong Song <yhs@fb.com>
[ Minor fixups to cope with --skip_missing ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a --skip_missing option that allows pahole to keep going in case one
of the type passed to -C (e.g. via a file) does not exist.
This is useful for intropsection software such as debugging kernel
modules that can handle various kernel configurations and versions for
which some recently added types are missing. The consumer of the header
becomes responsible of gating the uses of the type with #ifdef
CONFIG_XXX, rather than pahole bailing out on the first unknown type.
Committer testing:
Before:
$ pahole tcp_splice_state,xxfrm_policy_queue,list_head tcp.o
struct tcp_splice_state {
struct pipe_inode_info * pipe; /* 0 8 */
size_t len; /* 8 8 */
unsigned int flags; /* 16 4 */
/* size: 24, cachelines: 1, members: 3 */
/* padding: 4 */
/* last cacheline: 24 bytes */
};
pahole: type 'xxfrm_policy_queue' not found
$
After:
$ pahole --help |& grep skip
--skip=COUNT Skip COUNT input records
--skip_encoding_btf_tag Do not encode TAGs in BTF.
--skip_encoding_btf_vars Do not encode VARs in BTF.
--skip_missing skip missing types passed to -C rather than stop
$ pahole --skip_missing tcp_splice_state,xxfrm_policy_queue,list_head tcp.o
struct tcp_splice_state {
struct pipe_inode_info * pipe; /* 0 8 */
size_t len; /* 8 8 */
unsigned int flags; /* 16 4 */
/* size: 24, cachelines: 1, members: 3 */
/* padding: 4 */
/* last cacheline: 24 bytes */
};
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
pahole: type 'xxfrm_policy_queue' not found
$
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Variables can be allocated with or without specification, however,
tag__recode_dwarf_type() always tries accessing it, leading to heap read
overflows and subsequent logic bugs.
Fix by introducing a bit that tracks whether or not specification is
present.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Will be called when a thread exits, initially only in the DWARF loader,
so that pahole can call the btf_encoder associated with the exiting
thread to do the dedup as the last step done in parallel.
Then we'll iterate the btf_encoders list and combine everything into the
first btf_encoder instance that gets then written to disk.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Parse the DWARF tag DW_TAG_LLVM_annotation. Only record annotations with
btf_tag name which corresponds to btf_tag attributes in C code. Such
information will be used later by the btf_encoder for BTF conversion.
The LLVM implementation only supports btf_tag annotations on
struct/union, func, func parameter and variable ([1]). So we only check
existence of corresponding DW tags in these places.
A flag "--skip_encoding_btf_tag" is introduced if for whatever reason
this feature needs to be disabled.
[1] https://reviews.llvm.org/D106614
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Link: https://lore.kernel.org/r/20210922021326.2287095-1-yhs@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Useful while developing to help in tuning the ptr tables (types, tags,
functions, maybe some more in the future).
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we don't need the cu to get the strings table, all tags have a
char pointer for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Right now its just for when we emit types, so we can reuse it for
instance, to handle different types with the same name in different CUs
in pahole.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For now this will only apply to the dwarf loader, for experimenting as
time passes and kernels grow bigger or with more symbols.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As this isn't present in most types or struct members, which ends up
making dwarf_attr() call libdw_find_attr() that will do a linear search
on all the attributes.
We don't use this in the BTF encoder, so no point in reading that.
This will be used in pahole in the following cset.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We'll use this to avoid looking for attributes that doesn't apply to the
C language, such as DW_AT_virtuality (virtual, pure_virtual) and
DW_AT_accessibility (public, protected, private),
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that loaders such as the DWARF one can store there the DWARF handler
(Dwfl) that needs to stay live while tools use the core tags (struct
class, struct union, struct tag, etc) because they point to strings that
are managed by Dwfl, so we have to defer dwfl_end() to after tools are
done processing the core tags.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Another simplification made possible by using a plain char string
instead of string_t, that was only needed in the core as prep work
for CTF encoding.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Another simplification made possible by using a plain char string
instead of string_t, that was only needed in the core as prep work
for CTF encoding.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
They call each other, so do the three at once.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we stopped using per-cu obstacks we don't need it. If we ever
want to use it we can do per thread obstacks.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that we stopped using string indexes, no need for that, just set
namespace->name with the new class name.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We were using this just for the ctf_encoder, that never really got
complete, so ditch it.
For BTF the strings table is done by libbpf, so we don't need it there
either.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kui-Feng Lee
Arnaldo Carvalho de Melo
Douglas RAILLARD
Yonghong Song
Ilya Leoshkevich