Be it automatically when no -F option is passed and
/sys/kernel/btf/vmlinux is available, or when /sys/kernel/btf/vmlinux is
passed as the filename to the tool, i.e.:
$ pahole -C list_head
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$ strace -e openat pahole -C list_head |& grep /sys/kernel/btf/
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
$
$ pahole -C list_head /sys/kernel/btf/vmlinux
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$
If one wants to grab the matching vmlinux to use its DWARF info instead,
which is useful to compare the results with what we have from BTF, for
instance, its just a matter of using '-F dwarf'.
This in turn shows something that at first came as a surprise, but then
has a simple explanation:
For very common data structures, that will probably appear in all of the
DWARF CUs (Compilation Units), like 'struct list_head', using '-F dwarf'
is faster:
[acme@quaco pahole]$ perf stat -e cycles pahole -F btf -C list_head > /dev/null
Performance counter stats for 'pahole -F btf -C list_head':
45,722,518 cycles:u
0.023717300 seconds time elapsed
0.016474000 seconds user
0.007212000 seconds sys
[acme@quaco pahole]$ perf stat -e cycles pahole -F dwarf -C list_head > /dev/null
Performance counter stats for 'pahole -F dwarf -C list_head':
14,170,321 cycles:u
0.006668904 seconds time elapsed
0.005562000 seconds user
0.001109000 seconds sys
[acme@quaco pahole]$
But for something that is more specific to a subsystem, the DWARF loader
will have to process way more stuff till it gets to that struct:
$ perf stat -e cycles pahole -F dwarf -C tcp_sock > /dev/null
Performance counter stats for 'pahole -F dwarf -C tcp_sock':
31,579,795,238 cycles:u
8.332272930 seconds time elapsed
8.032124000 seconds user
0.286537000 seconds sys
$
While using the BTF loader the time should be constant, as it loads
everything from /sys/kernel/btf/vmlinux:
$ perf stat -e cycles pahole -F btf -C tcp_sock > /dev/null
Performance counter stats for 'pahole -F btf -C tcp_sock':
48,823,488 cycles:u
0.024102760 seconds time elapsed
0.012035000 seconds user
0.012046000 seconds sys
$
Above I used '-F btf' just to show that it can be used, but its not
really needed, i.e. those are equivalent:
$ strace -e openat pahole -F btf -C list_head |& grep /sys/kernel/btf/vmlinux
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
$ strace -e openat pahole -C list_head |& grep /sys/kernel/btf/vmlinux
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
$
The btf_raw__load() function that ends up being grafted into the
preexisting btf_elf routines was based on libbpf's btf_load_raw().
Acked-by: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
That is the idiom for free its members and then free itself, 'free' is
just to free its members.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that we don't clash with libbpf's 'struct btf', in time more internal
state now in 'struct btf_elf' will refer to the equivalent internal
state in libbpf's 'struct btf', as they have lots in common.
Requested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Acked-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Two new btf kinds, BTF_KIND_FUNC and BTF_KIND_FUNC_PROTO,
have been added in kernel since
https://patchwork.ozlabs.org/cover/1000176/
to support better func introspection.
Currently, for a DW_TAG_subroutine_type dwarf type,
a simple "void *" is generated instead of real subroutine type.
This patch teaches pahole to generate BTF_KIND_FUNC_PROTO
properly. After this patch, pahole should have complete
type coverage for C frontend with types a bpf program cares.
For example,
$ cat t1.c
typedef int __int32;
struct t1 {
int a1;
int (*f1)(char p1, __int32 p2);
} g1;
$ cat t2.c
typedef int __int32;
struct t2 {
int a2;
int (*f2)(char q1, __int32 q2, ...);
int (*f3)();
} g2;
int main() { return 0; }
$ gcc -O2 -o t1 -g t1.c t2.c
$ pahole -JV t1
File t1:
[1] TYPEDEF __int32 type_id=2
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] STRUCT t1 kind_flag=0 size=16 vlen=2
a1 type_id=2 bits_offset=0
f1 type_id=6 bits_offset=64
[4] FUNC_PROTO (anon) return=2 args=(5 (anon), 1 (anon))
[5] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[6] PTR (anon) type_id=4
[7] TYPEDEF __int32 type_id=8
[8] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[9] STRUCT t2 kind_flag=0 size=24 vlen=3
a2 type_id=8 bits_offset=0
f2 type_id=12 bits_offset=64
f3 type_id=14 bits_offset=128
[10] FUNC_PROTO (anon) return=8 args=(11 (anon), 7 (anon), vararg)
[11] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[12] PTR (anon) type_id=10
[13] FUNC_PROTO (anon) return=8 args=(vararg)
[14] PTR (anon) type_id=13
$
In the above example, type [4], [10] and [13] represent the
func_proto types.
BTF_KIND_FUNC, which represents a real subprogram, is not generated in
this patch and will be considered later.
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch fixed two issues with BTF. One is related to struct/union
bitfield encoding and the other is related to forward type.
Issue #1 and solution:
======================
Current btf encoding of bitfield follows what pahole generates.
For each bitfield, pahole will duplicate the type chain and
put the bitfield size at the final int or enum type.
Since the BTF enum type cannot encode bit size,
commit b18354f64c ("btf: Generate correct struct bitfield
member types") workarounds the issue by generating
an int type whenever the enum bit size is not 32.
The above workaround is not ideal as we lost original type
in BTF. Another undesiable fact is the type duplication
as the pahole duplicates the type chain.
To fix this issue, this patch implemented a compatible
change for BTF struct type encoding:
. the bit 31 of type->info, previously reserved,
now is used to indicate whether bitfield_size is
encoded in btf_member or not.
. if bit 31 of struct_type->info is set,
btf_member->offset will encode like:
bit 0 - 23: bit offset
bit 24 - 31: bitfield size
if bit 31 is not set, the old behavior is preserved:
bit 0 - 31: bit offset
So if the struct contains a bit field, the maximum bit offset
will be reduced to (2^24 - 1) instead of MAX_UINT. The maximum
bitfield size will be 255 which is enough for today as maximum
bitfield in compiler can be 128 where int128 type is supported.
A new global, no_bitfield_type_recode, is introduced and which
will be set to true if BTF encoding is enabled. This global
will prevent pahole duplicating the bitfield types to avoid
type duplication in BTF.
Issue #2 and solution:
======================
Current forward type in BTF does not specify whether the original
type is struct or union. This will not work for type pretty print
and BTF-to-header-file conversion as struct/union must be specified.
To fix this issue, similar to issue #1, type->info bit 31
is used. If the bit is set, it is union type. Otherwise, it is
a struct type.
Examples:
=========
-bash-4.4$ cat t.c
struct s;
union u;
typedef int ___int;
enum A { A1, A2, A3 };
struct t {
int a[5];
___int b:4;
volatile enum A c:4;
struct s *p1;
union u *p2;
} g;
-bash-4.4$ gcc -c -O2 -g t.c
Without this patch:
$ pahole -JV t.o
[1] TYPEDEF ___int type_id=2
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] ENUM A size=4 vlen=3
A1 val=0
A2 val=1
A3 val=2
[4] STRUCT t size=40 vlen=5
a type_id=5 bits_offset=0
b type_id=13 bits_offset=160
c type_id=15 bits_offset=164
p1 type_id=9 bits_offset=192
p2 type_id=11 bits_offset=256
[5] ARRAY (anon) type_id=2 index_type_id=2 nr_elems=5
[6] INT sizetype size=8 bit_offset=0 nr_bits=64 encoding=(none)
[7] VOLATILE (anon) type_id=3
[8] FWD s type_id=0
[9] PTR (anon) type_id=8
[10] FWD u type_id=0
[11] PTR (anon) type_id=10
[12] INT int size=1 bit_offset=0 nr_bits=4 encoding=(none)
[13] TYPEDEF ___int type_id=12
[14] INT (anon) size=1 bit_offset=0 nr_bits=4 encoding=SIGNED
[15] VOLATILE (anon) type_id=14
With this patch:
$ pahole -JV t.o
File t.o:
[1] TYPEDEF ___int type_id=2
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] ENUM A size=4 vlen=3
A1 val=0
A2 val=1
A3 val=2
[4] STRUCT t kind_flag=1 size=40 vlen=5
a type_id=5 bitfield_size=0 bits_offset=0
b type_id=1 bitfield_size=4 bits_offset=160
c type_id=7 bitfield_size=4 bits_offset=164
p1 type_id=9 bitfield_size=0 bits_offset=192
p2 type_id=11 bitfield_size=0 bits_offset=256
[5] ARRAY (anon) type_id=2 index_type_id=2 nr_elems=5
[6] INT sizetype size=8 bit_offset=0 nr_bits=64 encoding=(none)
[7] VOLATILE (anon) type_id=3
[8] FWD s struct
[9] PTR (anon) type_id=8
[10] FWD u union
[11] PTR (anon) type_id=10
The fix removed the type duplication, preserved the enum type for the
bitfield, and have correct struct/union information for the forward
type.
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The btf bitfield encoding is broken.
For the following example:
-bash-4.2$ cat t.c
struct t {
int a:2;
int b:1;
int :3;
int c:1;
int d;
char e:1;
char f:1;
int g;
};
void test(struct t *t) {
return;
}
-bash-4.2$ clang -S -g -emit-llvm t.c
The output for bpf "little and big" endian results with pahole dwarf2btf
conversion:
-bash-4.2$ llc -march=bpfel -mattr=dwarfris -filetype=obj t.ll
-bash-4.2$ pahole -JV t.o
[1] PTR (anon) type_id=2
[2] STRUCT t size=16 vlen=7
a type_id=5 bits_offset=30
b type_id=6 bits_offset=29
c type_id=6 bits_offset=25
d type_id=3 bits_offset=32
e type_id=7 bits_offset=71
f type_id=7 bits_offset=70
g type_id=3 bits_offset=96
[3] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[4] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[5] INT int size=1 bit_offset=0 nr_bits=2 encoding=(none)
[6] INT int size=1 bit_offset=0 nr_bits=1 encoding=(none)
[7] INT char size=1 bit_offset=0 nr_bits=1 encoding=(none)
-bash-4.2$ llc -march=bpfeb -mattr=dwarfris -filetype=obj t.ll
-bash-4.2$ pahole -JV t.o
[1] PTR (anon) type_id=2
[2] STRUCT t size=16 vlen=7
a type_id=5 bits_offset=0
b type_id=6 bits_offset=2
c type_id=6 bits_offset=6
d type_id=3 bits_offset=32
e type_id=7 bits_offset=64
f type_id=7 bits_offset=65
g type_id=3 bits_offset=96
[3] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[4] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[5] INT int size=1 bit_offset=0 nr_bits=2 encoding=(none)
[6] INT int size=1 bit_offset=0 nr_bits=1 encoding=(none)
[7] INT char size=1 bit_offset=0 nr_bits=1 encoding=(none)
The BTF struct member bits_offset counts bits from the beginning of the
containing entity regardless of endianness, similar to what
DW_AT_bit_offset from DWARF4 does. Such counting is equivalent to the
big endian conversion in the above.
But the little endian conversion is not correct since dwarf generates
DW_AT_bit_offset based on actual bit position in the little endian
architecture. For example, for the above struct member "a", the dwarf
would generate DW_AT_bit_offset=30 for little endian, and
DW_AT_bit_offset=0 for big endian.
This patch fixed the little endian structure member bits_offset problem
with proper calculation based on dwarf attributes.
With the fix, we get:
-bash-4.2$ llc -march=bpfel -mattr=dwarfris -filetype=obj t.ll
-bash-4.2$ pahole -JV t.o
[1] STRUCT t size=16 vlen=7
a type_id=5 bits_offset=0
b type_id=6 bits_offset=2
c type_id=6 bits_offset=6
d type_id=2 bits_offset=32
e type_id=7 bits_offset=64
f type_id=7 bits_offset=65
g type_id=2 bits_offset=96
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[4] PTR (anon) type_id=1
[5] INT int size=1 bit_offset=0 nr_bits=2 encoding=(none)
[6] INT int size=1 bit_offset=0 nr_bits=1 encoding=(none)
[7] INT char size=1 bit_offset=0 nr_bits=1 encoding=(none)
-bash-4.2$ llc -march=bpfeb -mattr=dwarfris -filetype=obj t.ll
-bash-4.2$ pahole -JV t.o
[1] PTR (anon) type_id=2
[2] STRUCT t size=16 vlen=7
a type_id=5 bits_offset=0
b type_id=6 bits_offset=2
c type_id=6 bits_offset=6
d type_id=3 bits_offset=32
e type_id=7 bits_offset=64
f type_id=7 bits_offset=65
g type_id=3 bits_offset=96
[3] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[4] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[5] INT int size=1 bit_offset=0 nr_bits=2 encoding=(none)
[6] INT int size=1 bit_offset=0 nr_bits=1 encoding=(none)
[7] INT char size=1 bit_offset=0 nr_bits=1 encoding=(none)
-bash-4.2$
For both little endian and big endian, we have correct and
same bits_offset for struct members.
We could fix pos->bit_offset, but pos->bit_offset will be inconsistent
to pos->bitfield_offset in the meaning and pos->bitfield_offset is used
to print out pahole data structure:
-bash-4.2$ llc -march=bpfel -mattr=dwarfris -filetype=obj t.ll
-bash-4.2$ /bin/pahole t.o
struct t {
int a:2; /* 0:30 4 */
int b:1; /* 0:29 4 */
int c:1; /* 0:25 4 */
.....
So this patch just made the change in btf specific routines.
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch introduces BPF Type Format (BTF).
BTF (BPF Type Format) is the meta data format which describes
the data types of BPF program/map. Hence, it basically focus
on the C programming language which the modern BPF is primary
using. The first use case is to provide a generic pretty print
capability for a BPF map.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>