Collect the information of per-cpu variables from encoders of worker
threads to the primary encoder.
btf_encoder store per-cpu info separately, not in BTF. Previously, it
merged only BTF types generated by worker threads. So some of the
per-cpu info was missing.
Signed-off-by: Kui-Feng Lee <kuifeng@fb.com>
Tested-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: dwarves@vger.kernel.org
Link: https://lore.kernel.org/r/20220321230837.855572-1-kuifeng@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kernel commit ([1]) renamed btf_tag to btf_decl_tag for uapi btf.h and
libbpf api's. The reason is a new clang attribute, btf_type_tag, is
introduced ([2]). Renaming btf_tag to btf_decl_tag makes it easier to
distinghish from btf_type_tag.
I also pulled in latest libbpf repo since it contains renamed libbpf api
function btf__add_decl_tag().
[1] https://lore.kernel.org/bpf/20211012164838.3345699-1-yhs@fb.com/
[2] https://reviews.llvm.org/D111199
Signed-off-by: Yonghong Song <yhs@fb.com>
[ Minor fixups to cope with --skip_missing ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
pahole does not generate VARs for percpu symbols on s390. A percpu
symbol definition on a typical x86_64 kernel looks like this:
[33] .data..percpu PROGBITS 0000000000000000 01c00000
^^^^^^^^^^^^^^^^ sh_addr
LOAD 0x0000000001c00000 0x0000000000000000 0x000000000286f000
^^^^^^^^^^^^^^^^^^ p_vaddr
13559: 000000000001ba50 4 OBJECT LOCAL DEFAULT 33 cpu_profile_flip
^^^^^^^^^^^^^^^^ st_value
Most importantly, .data..percpu's sh_addr is 0, and this is what pahole
is currently assuming. However, on s390 this is different:
[37] .data..percpu PROGBITS 00000000019cd000 018ce000
^^^^^^^^^^^^^^^^ sh_addr
LOAD 0x000000000136e000 0x000000000146d000 0x000000000146d000
^^^^^^^^^^^^^^^^^^ p_vaddr
80377: 0000000001ba1440 4 OBJECT WEAK DEFAULT 37 cpu_profile_flip
^^^^^^^^^^^^^^^^ st_value
Fix by restructuring the code to always use section-relative offsets for
symbols. Change the comment to focus on this invariant.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The following is an example with latest upstream clang:
$ cat t.c
#define __tag1 __attribute__((btf_tag("tag1")))
#define __tag2 __attribute__((btf_tag("tag2")))
struct t {
int a:1 __tag1;
int b __tag2;
} __tag1 __tag2;
int g __tag1 __attribute__((section(".data..percpu")));
int __tag1 foo(struct t *a1, int a2 __tag2) {
return a1->b + a2 + g;
}
$ clang -O2 -g -c t.c
$ pahole -JV t.o
Found per-CPU symbol 'g' at address 0x0
Found 1 per-CPU variables!
Found 1 functions!
File t.o:
[1] INT int size=4 nr_bits=32 encoding=SIGNED
[2] PTR (anon) type_id=3
[3] STRUCT t size=8
a type_id=1 bitfield_size=1 bits_offset=0
b type_id=1 bitfield_size=0 bits_offset=32
[4] TAG tag1 type_id=3 component_idx=0
[5] TAG tag2 type_id=3 component_idx=1
[6] TAG tag1 type_id=3 component_idx=-1
[7] TAG tag2 type_id=3 component_idx=-1
[8] FUNC_PROTO (anon) return=1 args=(2 a1, 1 a2)
[9] FUNC foo type_id=8
[10] TAG tag2 type_id=9 component_idx=1
[11] TAG tag1 type_id=9 component_idx=-1
search cu 't.c' for percpu global variables.
Variable 'g' from CU 't.c' at address 0x0 encoded
[12] VAR g type=1 linkage=1
[13] TAG tag1 type_id=12 component_idx=-1
[14] DATASEC .data..percpu size=4 vlen=1
type=12 offset=0 size=4
$ ...
With additional option --skip_encoding_btf_tag, pahole doesn't
generate BTF_KIND_TAGs any more.
$ pahole -JV --skip_encoding_btf_tag t.o
Found per-CPU symbol 'g' at address 0x0
Found 1 per-CPU variables!
Found 1 functions!
File t.o:
[1] INT int size=4 nr_bits=32 encoding=SIGNED
[2] PTR (anon) type_id=3
[3] STRUCT t size=8
a type_id=1 bitfield_size=1 bits_offset=0
b type_id=1 bitfield_size=0 bits_offset=32
[4] FUNC_PROTO (anon) return=1 args=(2 a1, 1 a2)
[5] FUNC foo type_id=4
search cu 't.c' for percpu global variables.
Variable 'g' from CU 't.c' at address 0x0 encoded
[6] VAR g type=1 linkage=1
[7] DATASEC .data..percpu size=4 vlen=1
type=6 offset=0 size=4
$ ...
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Link: https://lore.kernel.org/r/20210922021332.2287418-1-yhs@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since the 'int' variable, 'err' was just checked for < 0, cast it to
uint32_t and compare with the 'uint32_t' one.
Fixes this clang warning:
/var/home/acme/git/pahole/btf_encoder.c: In function ‘btf_encoder__write_raw_file’:
/var/home/acme/git/pahole/btf_encoder.c:890:17: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
890 | if (err != raw_btf_size) {
| ^~
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we don't need the cu to get the strings table anymore, all tags
have a char pointer for strings.
Also rename it to ftype__has_arg_names() and simplify a bit it by
removing a needless one-type use 'name' variable.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we don't need the cu to get the strings table anymore, all tags
have a char pointer for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we don't need the cu to get the strings table anymore, all tags
have a char pointer for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we don't need the cu to get the strings table anymore, all tags
have a char pointer for strings.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We don't need it as we used it only for calling elf_section_by_name(),
that doesn't need it anymore.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In 3f8aad340b ("elf_symtab: Handle SHN_XINDEX index in
elf_section_by_name()") we stopped using that argument as we switched to
using elf_getshdrstrndx() to get SHN_XINDEX.
So just remove that argument and fixup its callers, this will allow
removing a good chunk of calls and variables.
Cc: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Another simplification made possible by using a plain char string
instead of string_t, that was only needed in the core as prep work
for CTF encoding.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the enumerator->name case we get the bonus of removing the last user
of dwarves__active_loader in the btf_encoder class.
This covers unions, enums, structs and classes.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the namespace->name case we get the bonus of removing another
user of dwarves__active_loader.
This covers unions, enums, structs and classes.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the class_member->name case we get the bonus of removing another
user of dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the parameter->name case we get the bonus of removing a user of
dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the base_type->name case we get the bonus of removing some more
functions related base types and a user of dwarves__active_loader.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.
The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.
For the function->name case we get the bonus of removing the need of a
debug_fmt_ops->function() callback receiving the 'cu', just access the
string directly.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This reverts commit de3a7f9125.
Andrii reports that this cset is breaking the CI, as reported at:
https://travis-ci.com/github/libbpf/libbpf/jobs/514329152
So revert it while investigation about the root cause continues.
Reported-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The ->filename only points out to the ELF file when we're not writing to
a raw, detached BTF file, so reuse it to store the detached filename
when using that mode.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We were passing two 'btf_encoder' fields, so pass just the btf_encoder
instance. Ditch the unused 'flags' arg.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a new CMake option, LIBBPF_EMBEDDED, to switch between the embedded
version and the system version (searched via pkg-config) of libbpf. Set
the embedded version as the default.
Committer testing:
The default build works as before:
⬢[acme@toolbox pahole]$ rm -rf build ; mkdir build ; cd build ; cmake -DCMAKE_BUILD_TYPE=Release .. ; cd ..
-- The C compiler identification is GNU 11.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Checking availability of DWARF and ELF development libraries
-- Looking for dwfl_module_build_id in elf
-- Looking for dwfl_module_build_id in elf - found
-- Found dwarf.h header: /usr/include
-- Found elfutils/libdw.h header: /usr/include
-- Found libdw library: /usr/lib64/libdw.so
-- Found libelf library: /usr/lib64/libelf.so
-- Checking availability of DWARF and ELF development libraries - done
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")
-- Submodule update
-- Submodule update - done
-- Performing Test HAVE_REALLOCARRAY_SUPPORT
-- Performing Test HAVE_REALLOCARRAY_SUPPORT - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /var/home/acme/git/pahole/build
⬢[acme@toolbox pahole]$ make -j28 -C build
make: Entering directory '/var/home/acme/git/pahole/build'
make[1]: Entering directory '/var/home/acme/git/pahole/build'
make[2]: Entering directory '/var/home/acme/git/pahole/build'
Scanning dependencies of target bpf
make[2]: Leaving directory '/var/home/acme/git/pahole/build'
make[2]: Entering directory '/var/home/acme/git/pahole/build'
[ 5%] Building C object CMakeFiles/bpf.dir/lib/bpf/src/btf.c.o
[ 5%] Building C object CMakeFiles/bpf.dir/lib/bpf/src/bpf_prog_linfo.c.o
[ 5%] Building C object CMakeFiles/bpf.dir/lib/bpf/src/bpf.c.o
[ 7%] Building C object CMakeFiles/bpf.dir/lib/bpf/src/libbpf_errno.c.o
<SNIP>
make[2]: Leaving directory '/var/home/acme/git/pahole/build'
[ 98%] Built target ctracer
[100%] Linking C executable pahole
make[2]: Leaving directory '/var/home/acme/git/pahole/build'
[100%] Built target pahole
make[1]: Leaving directory '/var/home/acme/git/pahole/build'
make: Leaving directory '/var/home/acme/git/pahole/build'
⬢[acme@toolbox pahole]$ ldd build/pahole
linux-vdso.so.1 (0x00007ffcf4d92000)
libdwarves_reorganize.so.1 => /var/home/acme/git/pahole/build/libdwarves_reorganize.so.1 (0x00007f059c289000)
libdwarves.so.1 => /var/home/acme/git/pahole/build/libdwarves.so.1 (0x00007f059c226000)
libdw.so.1 => /lib64/libdw.so.1 (0x00007f059c186000)
libelf.so.1 => /lib64/libelf.so.1 (0x00007f059c16b000)
libz.so.1 => /lib64/libz.so.1 (0x00007f059c151000)
libc.so.6 => /lib64/libc.so.6 (0x00007f059bf82000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f059bf79000)
libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f059be83000)
liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f059be57000)
libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f059be44000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f059be23000)
/lib64/ld-linux-x86-64.so.2 (0x00007f059c290000)
⬢[acme@toolbox pahole]$
Then, trying to use the libbpf-devel present in Fedora 34:
⬢[acme@toolbox pahole]$ rm -rf build ; mkdir build ; cd build ; cmake -DCMAKE_BUILD_TYPE=Release -DLIBBPF_EMBEDDED=Off .. ; cd ..
-- The C compiler identification is GNU 11.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PkgConfig: /usr/bin/pkg-config (found version "1.7.3")
-- Checking for module 'libbpf>=0.3.0'
-- Found libbpf, version 0.3.0
-- Checking availability of DWARF and ELF development libraries
-- Looking for dwfl_module_build_id in elf
-- Looking for dwfl_module_build_id in elf - found
-- Found dwarf.h header: /usr/include
-- Found elfutils/libdw.h header: /usr/include
-- Found libdw library: /usr/lib64/libdw.so
-- Found libelf library: /usr/lib64/libelf.so
-- Checking availability of DWARF and ELF development libraries - done
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")
-- Submodule update
-- Submodule update - done
-- Performing Test HAVE_REALLOCARRAY_SUPPORT
-- Performing Test HAVE_REALLOCARRAY_SUPPORT - Success
-- Configuring done
-- Generating done
-- Build files have been written to: /var/home/acme/git/pahole/build
⬢[acme@toolbox pahole]$ m
make: Entering directory '/var/home/acme/git/pahole/build'
make[1]: Entering directory '/var/home/acme/git/pahole/build'
make[2]: Entering directory '/var/home/acme/git/pahole/build'
Scanning dependencies of target dwarves
make[2]: Leaving directory '/var/home/acme/git/pahole/build'
make[2]: Entering directory '/var/home/acme/git/pahole/build'
[ 2%] Building C object CMakeFiles/dwarves.dir/dwarves.c.o
[ 5%] Building C object CMakeFiles/dwarves.dir/dwarves_fprintf.c.o
[ 7%] Building C object CMakeFiles/dwarves.dir/gobuffer.c.o
<SNIP>
[ 33%] Building C object CMakeFiles/dwarves.dir/rbtree.c.o
/var/home/acme/git/pahole/btf_encoder.c:84:10: error: ‘BTF_KIND_FLOAT’ undeclared here (not in a function); did you mean ‘BTF_KIND_INT’?
84 | [BTF_KIND_FLOAT] = "FLOAT",
| ^~~~~~~~~~~~~~
| BTF_KIND_INT
/var/home/acme/git/pahole/btf_encoder.c:84:10: error: array index in initializer not of integer type
/var/home/acme/git/pahole/btf_encoder.c:84:10: note: (near initialization for ‘btf_kind_str’)
/var/home/acme/git/pahole/btf_encoder.c:84:35: warning: excess elements in array initializer
84 | [BTF_KIND_FLOAT] = "FLOAT",
| ^~~~~~~
/var/home/acme/git/pahole/btf_encoder.c:84:35: note: (near initialization for ‘btf_kind_str’)
/var/home/acme/git/pahole/btf_encoder.c: In function ‘btf_encoder__add_float’:
/var/home/acme/git/pahole/btf_encoder.c:224:22: warning: implicit declaration of function ‘btf__add_float’; did you mean ‘btf__add_var’? [-Wimplicit-function-declaration]
224 | int32_t id = btf__add_float(encoder->btf, name, BITS_ROUNDUP_BYTES(bt->bit_size));
| ^~~~~~~~~~~~~~
| btf__add_var
/var/home/acme/git/pahole/btf_loader.c: In function ‘btf__load_types’:
/var/home/acme/git/pahole/btf_loader.c:455:22: error: ‘BTF_KIND_FLOAT’ undeclared (first use in this function); did you mean ‘BTF_KIND_INT’?
455 | case BTF_KIND_FLOAT:
| ^~~~~~~~~~~~~~
| BTF_KIND_INT
/var/home/acme/git/pahole/btf_loader.c:455:22: note: each undeclared identifier is reported only once for each function it appears in
make[2]: *** [CMakeFiles/dwarves.dir/build.make:173: CMakeFiles/dwarves.dir/btf_encoder.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/dwarves.dir/build.make:186: CMakeFiles/dwarves.dir/btf_loader.c.o] Error 1
make[2]: Leaving directory '/var/home/acme/git/pahole/build'
make[1]: *** [CMakeFiles/Makefile2:173: CMakeFiles/dwarves.dir/all] Error 2
make[1]: Leaving directory '/var/home/acme/git/pahole/build'
make: *** [Makefile:149: all] Error 2
make: Leaving directory '/var/home/acme/git/pahole/build'
⬢[acme@toolbox pahole]$
It doesn't build as libbpf is old and doesn't have support for
BTF_KIND_FLOAT.
Signed-off-by: Luca Boccassi <bluca@debian.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kui-Feng Lee
Yonghong Song
Ilya Leoshkevich
Arnaldo Carvalho de Melo
Luca Boccassi