Some BPF programs compiled on s390 fail to load, because s390
arch-specific linux headers contain float and double types.
Fix as follows:
- Make the DWARF loader fill base_type.float_type.
- Introduce the --btf_gen_floats command-line parameter, so that
pahole could be used to build both the older and the newer kernels.
- libbpf introduced the support for the floating-point types in commit
986962fade5, so update the libbpf submodule to that version and use
the new btf__add_float() function in order to emit the floating-point
types when not in the compatibility mode.
- Make the BTF loader recognize the new BTF kind.
Example of the resulting entry in the vmlinux BTF:
[7164] FLOAT 'double' size=8
when building with:
LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1} --btf_gen_floats
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This reverts commit 82749180b2.
Getting in the way of releasing 1.20, breaking the build of a dwarves
rpm when a libbpf package is installed in a fedora 33 system:
In file included from /home/acme/rpmbuild/BUILD/dwarves-1.20/strings.c:7:
/home/acme/rpmbuild/BUILD/dwarves-1.20/pahole_strings.h:9:10: fatal error: bpf/btf.h: No such file or directory
9 | #include <bpf/btf.h>
| ^~~~~~~~~~~
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a new CMake option, LIBBPF_EMBEDDED, to switch between the
embedded version and the system version (searched via pkg-config)
of libbpf. Set the embedded version as the default.
Signed-off-by: Luca Boccassi <bluca@debian.org>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add support for generating split BTF, in which there is a designated base
BTF, containing a base set of types, and a split BTF, which extends main BTF
with extra types, that can reference types and strings from the main BTF.
This is going to be used to generate compact BTFs for kernel modules, with
vmlinux BTF being a main BTF, which all kernel modules are based off of.
These changes rely on patch set [0] to be present in libbpf submodule.
[0] https://patchwork.kernel.org/project/netdevbpf/list/?series=377859&state=*
Committer notes:
Fixed up wrt ARGP_numeric_version and added a man page entry.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Switch BTF loading to completely use libbpf's own struct btf and related
APIs.
BTF encoding is still happening with pahole's own code, so these two
code paths are not sharing anything now. String fetching is happening
based on whether btfe->strings were set to non-NULL pointer by
btf_encoder.
Committer testing:
$ cp ~/git/build/bpf-next-v5.9.0-rc8+/vmlinux .
$ readelf -SW vmlinux | grep BTF
[24] .BTF PROGBITS ffffffff82494ac0 1694ac0 340207 00 A 0 0 1
[25] .BTF_ids PROGBITS ffffffff827d4cc8 19d4cc8 0000a4 00 A 0 0 1
$ ./btfdiff vmlinux
$
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andrii@kernel.org>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
pdwtags -F btf vmlinux
Was segfaulting when trying to iterate on the function main lexblock,
which was zeroed instead of INIT_LIST_HEAD'ed, fix it.
This also made pfunct misbehave when used with BTF. pahole unnafected as
it doesn't try to go thru functions in most cases.
Fixes: ccf3eebfcd ("btf_loader: Add support for BTF_KIND_FUNC")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This one just allows that to be set, i.e.:
$ pahole -C perf_event_header ~/bin/perf
struct perf_event_header {
__u32 type; /* 0 4 */
__u16 misc; /* 4 2 */
__u16 size; /* 6 2 */
/* size: 8, cachelines: 1, members: 3 */
/* last cacheline: 8 bytes */
};
$ pahole -C 'perf_event_header(sizeof=size)' --seek_bytes=0x348 --count 1 ~/bin/perf < perf.data
{
.type = 0xa,
.misc = 0x2,
.size = 0x68,
},
$
But:
$ pahole -C 'perf_event_header(sizeof=bla)' --seek_bytes=0x348 --count 1 ~/bin/perf < perf.data
pahole: the sizeof member 'bla' not found in the 'perf_event_header' type
$
And:
$ pahole -C 'perf_event_header(size=misc)' --seek_bytes=0x348 --count 1 ~/bin/perf < perf.data
pahole: invalid arg 'size' in 'perf_event_header(size=misc)' (known args: sizeof=member)
$
The next cset will implement sizeof(type) with that modifier, using the
stdin bytes to obtain the size (0x68) in the above case, and then we'll
be able to print a sequence of variable-sized records correctly.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Just skip them, we have no use for them so far, skipping them will allow
us to process the other kinds we have use for.
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Instead of just telling that some unknown BTF_KIND_ was found in a file,
show the number to help in debugging.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Addressing this compiler warning:
/home/acme/git/pahole/btf_loader.c:279:41: error: pointer targets in passing argument 2 of ‘btf_elf__get32’ differ in signedness [-Werror=pointer-sign]
uint32_t value = btf_elf__get32(btfe, &ep[i].val);
^~~~~~~~~~
In file included from /home/acme/git/pahole/btf_loader.c:27:
/home/acme/git/pahole/libbtf.h:63:10: note: expected ‘uint32_t *’ {aka ‘unsigned int *’} but argument is of type ‘__s32 *’ {aka ‘int *’}
uint32_t btf_elf__get32(struct btf_elf *btf, uint32_t *p);
^~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fixes this coverity report:
Error: RESOURCE_LEAK (CWE-772): [#def2]
dwarves-1.13/btf_loader.c:342: alloc_fn: Storage is returned from allocation function "zalloc".
dwarves-1.13/btf_loader.c:342: var_assign: Assigning: "tag" = storage returned from "zalloc(40UL)".
dwarves-1.13/btf_loader.c:354: leaked_storage: Variable "tag" going out of scope leaks the storage it points to.
# 352| default:
# 353| printf("%s: FOO %d\n\n", __func__, type);
# 354|-> return 0;
# 355| }
# 356|
Reported-by: William Cohen <wcohen@redhat.com>
Fixes: 472256d3c5 ("btf_loader: Introduce a loader for the BTF format")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch records for each CU whether it's in little-endian or
big-endian data format. This flag will be used in subsequent commits to
adjust bit offsets where necessary, to make them uniform across
endianness. This patch doesn't have any effect on pahole's output.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Mark Wielaard <mark@klomp.org>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Bitfield offsets can be negative, if field "borrows" few bits from
following aligned field. This causes a bunch of surprises down the line
in pahole's logic (e.g., for hole calculation logic), so instead of
waiting till printf routines adjust this for display, adjust them early
and keep less surprising semantics.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
the CTF and BTF loaders come already with the id to use, while the DWARF
loader comes with a Dwarf_Off that needs to be converted into the
ptr_table index.
So keep the cu__add_tag(cu, tag, &id) method to ask ask for the index to
be allocated in the ptr_table and the result to come back via the 'id'
parameter, now a uint32_t and introduce a cu__add_tag_with_id(cu, tag, id)
method to indicate that the 'uint32_t id' is the one to use.
With this we can use a uint32_t for the id both on 32-bit and 64-bit
arches.
Reported-by: Andrii Nakryiko <andriin@fb.com>
Tested-by:Andrii Nakryiko <andrii.nakryiko@gmail.com>
Link: https://lore.kernel.org/bpf/CAEf4Bzb0SpvXdDKMMnUof==kp4Y0AP54bKFjeCzX_AsmDm7k7g@mail.gmail.com/
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Existing code base assumes that single CU doesn't have more than 65535
types per each CU, which might be a reasonable assumption for DWARF
data. With BTF, though, all we get is single, potentially huge, CU which
can easily have more than 65k types. For example, this is the case for
allyesconfig version of Linux kernel, which has >200k types.
Due to this assumption, libdwarves and other parts of pahole are using
16-bit counters to iterate over entities within CU. This can cause
infinite loops when iterating BTF data, if there are more than 65535
types. This patch changes non-public variables to use 32-bit integers,
where appropriate.
This still leads to invalid reported data when using BTF loader (due to using
(X & 0xFFFF) type ID, instead of X, when X > 65535) and loading huge files,
but at least it's not stuck in an infinite loop anymore.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
[ Removed non type ID conversions, for instance for the kind of tag, like in type->namespace.tag.tag, that can remain a uint16_t ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Existing code assumes alignment of any integer type, which breaks for
packed structs.
This patch fixes all the current discrepanies between dwarf and btf
loader, when compared using btfdiff.
It preserves bit_offset of non-bitfield members, while for bitfield ones
it re-calculates initial byte/bit offset using natural alignment of the
underlying integer type, which seems to be always the case for
bitfields.
I've tested this on toy examples for both x86-64 and arm targets, there
were no differences reported by btfdiff.
Testing on vmlinux on x86-64 shows only these discrepancies, which are
unrelated to bit offsets:
$ ./btfdiff /tmp/vmlinux4
--- /tmp/btfdiff.dwarf.GIVfpr 2019-02-20 12:18:29.138788970 -0800
+++ /tmp/btfdiff.btf.c3x2KY 2019-02-20 12:18:29.351786365 -0800
@@ -16884,7 +16884,7 @@ struct pebs_record_nhm {
};
union hsw_tsx_tuning {
struct {
- unsigned int cycles_last_block:32; /* 0: 0 4 */
+ u32 cycles_last_block:32; /* 0: 0 4 */
u32 hle_abort:1; /* 4:31 4 */
u32 rtm_abort:1; /* 4:30 4 */
u32 instruction_abort:1; /* 4:29 4 */
@@ -26154,7 +26154,7 @@ struct acpi_device_power {
/* last cacheline: 40 bytes */
};
struct acpi_device_perf_flags {
- unsigned char reserved:8; /* 0: 0 1 */
+ u8 reserved:8; /* 0: 0 1 */
/* size: 1, cachelines: 1, members: 1 */
/* last cacheline: 1 bytes */
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
That is the idiom for free its members and then free itself, 'free' is
just to free its members.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that we don't clash with libbpf's 'struct btf', in time more internal
state now in 'struct btf_elf' will refer to the equivalent internal
state in libbpf's 'struct btf', as they have lots in common.
Requested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Acked-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that libbpf is a submodule, we don't need to copy/paste btf.h header
with BTF type definitions.
This is a first step in migrating parts of libbtf, btf_encoder and
btf_loader to use libbpf and starting to use btf__dedup().
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Because we don't have to lookup the member type that is encoded after
it, i.e. some member fields depend on first processing everything, then
lookup the types to get the sizes, etc.
But the:
member->byte_offset = member->bit_offset / 8;
Can be done soon after we have member->bit_offset loaded from the BTF
section.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I wrote the BTF loader from the CTF loader, where the size of
enumerations is expressed in bits, while BTF encodes it in bytes, fix
it.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
BTF_KIND_FUNC is generated by llvm (latest trunk, 8.0 or later).
Without BTF_KIND_FUNC support, we will see the following errors,
-bash-4.4$ cat t.c
struct t {
int a;
char b1:1;
char b2:3;
int c;
} g;
int main() { return 0; }
-bash-4.4$ clang -O2 -target bpf -g -c t.c -Xclang -target-feature -Xclang +dwarfris
-bash-4.4$ pahole -F btf t.o
BTF: idx: 3, off: 28, Unknown
struct t {
int a; /* 0 4 */
/* Bitfield combined with previous fields */
<ERROR(__class__fprintf:1342): 5 not found!>
/* Bitfield combined with previous fields */
<ERROR(__class__fprintf:1342): 5 not found!>
int c; /* 8 4 */
/* size: 12, cachelines: 1, members: 4 */
/* last cacheline: 12 bytes */
/* BRAIN FART ALERT! 12 != 8 + 0(holes), diff = 4 */
};
-bash-4.4$
The reason is that llvm generates BTF_KIND_FUNC which btf_loader does not
recognize.
This patch added support for BTF_KIND_FUNC. Since BTF_KIND_FUNC represents
a defined subprogram and not a real type. A null type is used to
represent BTF_KIND_FUNC to avoid skipping type index.
With this fix:
-bash-4.4$ pahole -F btf t.o
struct t {
int a; /* 0 4 */
char b1:1; /* 4: 0 1 */
char b2:3; /* 4: 1 1 */
int c; /* 8 4 */
/* size: 12, cachelines: 1, members: 4 */
/* last cacheline: 12 bytes */
/* BRAIN FART ALERT! 12 != 9 + 0(holes), diff = 3 */
};
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Commit 2a82d593be ("btf: Add kind_flag support for btf_loader") added
kind_flag supported in btf_loader to get correct bitfield_size and
bit_offset for struct/union members. The commit unnecessarily stored
the bit in the structure type which is not used later.
So let us remove the kind_flag from the struct type and any other
changes whose purpose is to set this bit.
Signed-off-by: Yonghong Song <yhs@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Fixes: 2a82d593be ("btf: Add kind_flag support for btf_loader")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Yonghong explains:
<quote>
The bitfield offset in BTF starts from lower number to bigger one. That
is, it is always following the big endian convention, bitfield_offset
will be smaller if close to the top of structure.
This is different from what dwarf is doing, which will show different
things on little endian vs. big endian.
You can make simple adjustment based on all available info. In
btf_encoder.s, we did similar adjustment for little endian from
dwarf to btf.
</>
So fix it up while loading, so that the rebuilt C output shows the same
thing from BTF and from DWARF.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For struct/union members, the struct/union type info kind_flag is needed
to calculate correct bitfield_size and bit_offset.
if (kind_flag) {
bitfield_size = BTF_MEMBER_BITFIELD_SIZE(member->offset);
bit_offset = BTF_MEMBER_BIT_OFFSET(member->offset);
} else {
bitfield_size = 0;
bit_offset = member->offset;
}
Note that bitfield_size and bit_offset will not depend on the member
type. The member type will help calculate correct bitfield_offset,
byte_size, byte_offset, bit_size.
For example, with the fix, we will be able to display
bit offset and bitfield size properly.
-bash-4.4$ cat t.c
struct t {
int a:2;
int b:3;
int c:2;
} g;
-bash-4.4$ gcc -c -O2 -g t.c
-bash-4.4$ pahole -JV t.o
File t.o:
[1] STRUCT t kind_flag=1 size=4 vlen=3
a type_id=2 bitfield_size=2 bits_offset=0
b type_id=2 bitfield_size=3 bits_offset=2
c type_id=2 bitfield_size=2 bits_offset=5
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
-bash-4.4$ pahole -F btf t.o
struct t {
int a:2; /* 0: 0 4 */
int b:3; /* 0: 2 4 */
int c:2; /* 0: 5 4 */
/* size: 4, cachelines: 1, members: 3 */
/* bit_padding: 25 bits */
/* last cacheline: 4 bytes */
};
Note that the above offset showing is different from the below dwarf as
BTF bitfield_offset is always the offset from the start of structure,
kindly like big endian encoding. This may need adjustment to be
conforming to the dwarf dump format.
-bash-4.4$ pahole -F dwarf t.o
struct t {
int a:2; /* 0:30 4 */
int b:3; /* 0:27 4 */
int c:2; /* 0:25 4 */
/* size: 4, cachelines: 1, members: 3 */
/* bit_padding: 25 bits */
/* last cacheline: 4 bytes */
};
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Ilya Leoshkevich
Arnaldo Carvalho de Melo
Luca Boccassi
Andrii Nakryiko
Andrii Nakryiko
Yonghong Song