We'll use it in the BTF loader.
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
[ Split from a larger patch ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We just need to record if we printed it for a member and if so, deduce
that from the number of spaces left to print before the end of line
comment (offset, size).
Fixes: a59459bb80 ("fprintf: Account inline type __aligned__ member types for spacing")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Kernel commit ([1]) renamed btf_tag to btf_decl_tag for uapi btf.h and
libbpf api's. The reason is a new clang attribute, btf_type_tag, is
introduced ([2]). Renaming btf_tag to btf_decl_tag makes it easier to
distinghish from btf_type_tag.
I also pulled in latest libbpf repo since it contains renamed libbpf api
function btf__add_decl_tag().
[1] https://lore.kernel.org/bpf/20211012164838.3345699-1-yhs@fb.com/
[2] https://reviews.llvm.org/D111199
Signed-off-by: Yonghong Song <yhs@fb.com>
[ Minor fixups to cope with --skip_missing ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
A typo, some escaping for paths and the header for an option.
Signed-off-by: Domenico Andreoli <domenico.andreoli@linux.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Refactor class__fixup_btf_bitfields to remove a "continue" statement, to
prepare the ground for alignment fixup that is relevant for some types
matching:
type->tag != DW_TAG_base_type && type->tag != DW_TAG_enumeration_type
Committer testing:
btfdiff passes for a x86_64 kernel built with gcc and for a clang
thin-LTO vmlinux build.
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a --skip_missing option that allows pahole to keep going in case one
of the type passed to -C (e.g. via a file) does not exist.
This is useful for intropsection software such as debugging kernel
modules that can handle various kernel configurations and versions for
which some recently added types are missing. The consumer of the header
becomes responsible of gating the uses of the type with #ifdef
CONFIG_XXX, rather than pahole bailing out on the first unknown type.
Committer testing:
Before:
$ pahole tcp_splice_state,xxfrm_policy_queue,list_head tcp.o
struct tcp_splice_state {
struct pipe_inode_info * pipe; /* 0 8 */
size_t len; /* 8 8 */
unsigned int flags; /* 16 4 */
/* size: 24, cachelines: 1, members: 3 */
/* padding: 4 */
/* last cacheline: 24 bytes */
};
pahole: type 'xxfrm_policy_queue' not found
$
After:
$ pahole --help |& grep skip
--skip=COUNT Skip COUNT input records
--skip_encoding_btf_tag Do not encode TAGs in BTF.
--skip_encoding_btf_vars Do not encode VARs in BTF.
--skip_missing skip missing types passed to -C rather than stop
$ pahole --skip_missing tcp_splice_state,xxfrm_policy_queue,list_head tcp.o
struct tcp_splice_state {
struct pipe_inode_info * pipe; /* 0 8 */
size_t len; /* 8 8 */
unsigned int flags; /* 16 4 */
/* size: 24, cachelines: 1, members: 3 */
/* padding: 4 */
/* last cacheline: 24 bytes */
};
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
pahole: type 'xxfrm_policy_queue' not found
$
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This code:
struct X {
struct {
} __attribute__((foo)) x __attribute__((bar));
}
Was wrongly printed as:
struct X {
struct {
} x __attribute__((foo)) __attribute__((bar));
}
This unfortunately matters a lot, since "bar" is suppose to apply to
"x", but "foo" to typeof(x). In the wrong form, both apply to "x",
leading to e.g. incorrect layout for __aligned__ attribute.
Signed-off-by: Douglas Raillard <douglas.raillard@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
pahole does not generate VARs for percpu symbols on s390. A percpu
symbol definition on a typical x86_64 kernel looks like this:
[33] .data..percpu PROGBITS 0000000000000000 01c00000
^^^^^^^^^^^^^^^^ sh_addr
LOAD 0x0000000001c00000 0x0000000000000000 0x000000000286f000
^^^^^^^^^^^^^^^^^^ p_vaddr
13559: 000000000001ba50 4 OBJECT LOCAL DEFAULT 33 cpu_profile_flip
^^^^^^^^^^^^^^^^ st_value
Most importantly, .data..percpu's sh_addr is 0, and this is what pahole
is currently assuming. However, on s390 this is different:
[37] .data..percpu PROGBITS 00000000019cd000 018ce000
^^^^^^^^^^^^^^^^ sh_addr
LOAD 0x000000000136e000 0x000000000146d000 0x000000000146d000
^^^^^^^^^^^^^^^^^^ p_vaddr
80377: 0000000001ba1440 4 OBJECT WEAK DEFAULT 37 cpu_profile_flip
^^^^^^^^^^^^^^^^ st_value
Fix by restructuring the code to always use section-relative offsets for
symbols. Change the comment to focus on this invariant.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Variables can be allocated with or without specification, however,
tag__recode_dwarf_type() always tries accessing it, leading to heap read
overflows and subsequent logic bugs.
Fix by introducing a bit that tracks whether or not specification is
present.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Will be called when a thread exits, initially only in the DWARF loader,
so that pahole can call the btf_encoder associated with the exiting
thread to do the dedup as the last step done in parallel.
Then we'll iterate the btf_encoders list and combine everything into the
first btf_encoder instance that gets then written to disk.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I.e. the one in:
13ebb60ab66799ab libbpf: Add API that copies all BTF types from one BTF object to another
This will be used to paralellize the BTF encoding phase.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The following is an example with latest upstream clang:
$ cat t.c
#define __tag1 __attribute__((btf_tag("tag1")))
#define __tag2 __attribute__((btf_tag("tag2")))
struct t {
int a:1 __tag1;
int b __tag2;
} __tag1 __tag2;
int g __tag1 __attribute__((section(".data..percpu")));
int __tag1 foo(struct t *a1, int a2 __tag2) {
return a1->b + a2 + g;
}
$ clang -O2 -g -c t.c
$ pahole -JV t.o
Found per-CPU symbol 'g' at address 0x0
Found 1 per-CPU variables!
Found 1 functions!
File t.o:
[1] INT int size=4 nr_bits=32 encoding=SIGNED
[2] PTR (anon) type_id=3
[3] STRUCT t size=8
a type_id=1 bitfield_size=1 bits_offset=0
b type_id=1 bitfield_size=0 bits_offset=32
[4] TAG tag1 type_id=3 component_idx=0
[5] TAG tag2 type_id=3 component_idx=1
[6] TAG tag1 type_id=3 component_idx=-1
[7] TAG tag2 type_id=3 component_idx=-1
[8] FUNC_PROTO (anon) return=1 args=(2 a1, 1 a2)
[9] FUNC foo type_id=8
[10] TAG tag2 type_id=9 component_idx=1
[11] TAG tag1 type_id=9 component_idx=-1
search cu 't.c' for percpu global variables.
Variable 'g' from CU 't.c' at address 0x0 encoded
[12] VAR g type=1 linkage=1
[13] TAG tag1 type_id=12 component_idx=-1
[14] DATASEC .data..percpu size=4 vlen=1
type=12 offset=0 size=4
$ ...
With additional option --skip_encoding_btf_tag, pahole doesn't
generate BTF_KIND_TAGs any more.
$ pahole -JV --skip_encoding_btf_tag t.o
Found per-CPU symbol 'g' at address 0x0
Found 1 per-CPU variables!
Found 1 functions!
File t.o:
[1] INT int size=4 nr_bits=32 encoding=SIGNED
[2] PTR (anon) type_id=3
[3] STRUCT t size=8
a type_id=1 bitfield_size=1 bits_offset=0
b type_id=1 bitfield_size=0 bits_offset=32
[4] FUNC_PROTO (anon) return=1 args=(2 a1, 1 a2)
[5] FUNC foo type_id=4
search cu 't.c' for percpu global variables.
Variable 'g' from CU 't.c' at address 0x0 encoded
[6] VAR g type=1 linkage=1
[7] DATASEC .data..percpu size=4 vlen=1
type=6 offset=0 size=4
$ ...
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Link: https://lore.kernel.org/r/20210922021332.2287418-1-yhs@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Parse the DWARF tag DW_TAG_LLVM_annotation. Only record annotations with
btf_tag name which corresponds to btf_tag attributes in C code. Such
information will be used later by the btf_encoder for BTF conversion.
The LLVM implementation only supports btf_tag annotations on
struct/union, func, func parameter and variable ([1]). So we only check
existence of corresponding DW tags in these places.
A flag "--skip_encoding_btf_tag" is introduced if for whatever reason
this feature needs to be disabled.
[1] https://reviews.llvm.org/D106614
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Arnaldo Carvalho de Melo <arnaldo.melo@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Link: https://lore.kernel.org/r/20210922021326.2287095-1-yhs@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The build system always download the libbpf submodule, regardless if
we're using the embedded or the system version.
Download the libbpf source only if we're using the embedded one.
Signed-off-by: Matteo Croce <mcroce@microsoft.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Latest upstream LLVM now supports to emit btf_tag to dwarf ([1]) and the
kernel support for btf_tag is also landed ([2]). Sync with latest libbpf
which has btf_tag support. Next step will be to implement dwarf -> btf
conversion for btf_tag.
[1] https://reviews.llvm.org/D106621
[2] https://lore.kernel.org/bpf/20210914223015.245546-1-yhs@fb.com
Signed-off-by: Yonghong Song <yhs@fb.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
ostra-cg, which requires python, is installed in the destination dir.
Make it optional for embedded distributions which doesn't have the
python interpreter available.
Signed-off-by: Matteo Croce <mcroce@microsoft.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Useful while developing to help in tuning the ptr tables (types, tags,
functions, maybe some more in the future).
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When using multithreaded DWARF loading we can't really freely access
some tables, as they may grow and lead to stale data accesses generating
segfaults.
So use a type comparision that takes into account just the immutable
information for structs, unions.
This isn't enough to discern if two types with the same name are really
the same, as we need to look at the member types to figure that out.
So if there are types for which member types need to be checked, leave
it for when all CUs were processed and are thus completely immutable to
resort and fully compare such types.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The main musl libc, used in Alpine Linux, doesn't have argp support, so
check if it is somewhere else, maybe in the musl-obstack package.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The musl libc, used in Alpine Linux, doesn't have argp support, so check
if it is somewhere else, maybe in the argp-standalone package.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Systems such as Alpine Linux don't have support for obtaining _SC_LEVEL1_DCACHE_LINESIZE
via sysconf(), provide altenative method reading it from sysfs.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
By doing the tarball using the parent directory ('pahole') and then
replacing ^pahole/ with dwarves-${version} we avoid tar replacing the
symlink destination, i.e. before we were getting:
$ ls -la dwarves-1.22/lib/include/
total 8
drwxr-xr-x. 2 acme acme 4096 Aug 12 15:54 .
drwxr-xr-x. 4 acme acme 4096 Aug 12 15:54 ..
lrwxrwxrwx. 1 acme acme 23 Jun 18 17:56 bpf -> dwarves-1.22/../bpf/src
$
Now we get:
$ v=1.$(($(git tag | sort -V | tail -1 | cut -d. -f2) + 1)) ; tar cvfJ ~/rpmbuild/SOURCES/dwarves-${v}.tar.xz --transform "s,^pahole/,dwarves-${v}/," `sed s%^%../pahole/%g MANIFEST`
$ tar tvf ~/rpmbuild/SOURCES/dwarves-1.22.tar.xz | tail -3
-rwxr-xr-x acme/acme 1132 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/run_vmtest.sh
-rw-r--r-- acme/acme 2904227 2021-06-14 15:51 dwarves-1.22/lib/bpf/travis-ci/vmtest/vmlinux.h
lrwxrwxrwx acme/acme 0 2021-08-11 08:55 dwarves-1.22/lib/include/bpf -> ../bpf/src
$
And building from the resulting tarball succeeds.
See the references in the threads and bug reportgs in the Link tags
below for more details.
Link: https://lore.kernel.org/dwarves/YRVxNEPEspmuHcKr@kernel.org/
Link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775247
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The tarball wasn't working without this file:
[ 32%] Building C object CMakeFiles/dwarves.dir/btf_encoder.c.o
/tmp/dwarves-1.22/btf_encoder.c:18:10: fatal error: bpf/btf.h: No such file or directory
#include <bpf/btf.h>
^~~~~~~~~~~
compilation terminated.
Cc: Luca Boccassi <bluca@debian.org>
Fixes: 82756ea3ea ("libbpf: Fixup patch to allow to use packaged version")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It should have been removed from the specfile when it was deleted from
the source tree, fix it.
Fixes: 743f2536d8 ("btf_encoder: Move libbtf.c to btf_encoder.c, the only user of its functions")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
But this still doesn't cover all types in the kernel, at least not for
btfdiff's needs, which is to have the output of pahole for BTF and DWARF
to be the same, so if we have two types that are ABI equal, it will
still complain if...
$ btfdiff vmlinux
--- /tmp/btfdiff.dwarf.VUXlsB 2021-08-06 18:11:51.371012024 -0300
+++ /tmp/btfdiff.btf.CeZ7hA 2021-08-06 18:11:51.604017029 -0300
@@ -48226,8 +48226,8 @@ struct intel_ir_data {
/* last cacheline: 56 bytes */
};
struct intel_pad_context {
- u32 padctrl0; /* 0 4 */
- u32 padctrl1; /* 4 4 */
+ u32 conf0; /* 0 4 */
+ u32 val; /* 4 4 */
/* size: 8, cachelines: 1, members: 2 */
/* last cacheline: 8 bytes */
$
The name of some members are different :-\ Consider it in the next
patch and possibly add a knob to consider both types equal, i.e. don't
compare member names, just size, number of members and types of pairs of
members (at each offset in both types).
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
But since it is still related to cus processing, remove that arg and
rename it to __cus__load_debug_types().
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Douglas Raillard
Arnaldo Carvalho de Melo
Yonghong Song
Domenico Andreoli
Ilya Leoshkevich
Matteo Croce