When we started supporting unions in pahole some corner cases where
classes were expected needed fixing, this was one such case, that
printed bogus number of holes when handling unions, that is represented
as a 'struct type', that doesn't have ->nr_holes, just 'struct class'
that is has 'struct type' as its first member, i.e. extends it.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I.e. no point in looking for holes in unions.
Now 'pahole -H 10' will only show the structs with 10 or more alignment
holes, not those + all unions.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We had to use the same hack as in pfunct, as implemented in ccf3eebfcd
("btf_loader: Add support for BTF_KIND_FUNC"), will hide that 'struct
ftype' (aka function prototype) indirection behind the parameter
iterator (function__for_each_parameter).
For now, here is the top 10 Linux kernel data structures in terms of
number of functions receiving as one of its parameters a pointer to it,
using /sys/kernel/btf/vmlinux to look at all the vmlinux types and
functions (the ones visible in kallsyms, but with the parameters and its
types):
$ pahole -m | sort -k2 -nr | head
device 955
sock 568
sk_buff 541
task_struct 437
inode 421
pci_dev 390
page 351
net_device 347
file 315
net 312
$
$ pahole --help |& grep -- -m
-m, --nr_methods show number of methods
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we default to operating on the running kernel data structures, we
should make the default to, with no options passed, to pretty print all
the running kernel data structures, or do what was asked in terms of
number of members, size of structs, etc, i.e.:
[root@quaco ~]# pahole --help |& head
Usage: pahole [OPTION...] FILE
-a, --anon_include include anonymous classes
-A, --nested_anon_include include nested (inside other structs) anonymous
classes
-B, --bit_holes=NR_HOLES Show only structs at least NR_HOLES bit holes
-c, --cacheline_size=SIZE set cacheline size to SIZE
--classes_as_structs Use 'struct' when printing classes
-C, --class_name=CLASS_NAME Show just this class
-d, --recursive recursive mode, affects several other flags
[root@quaco ~]#
Continues working as before, but if you do:
pahole
It will work just as if you did:
pahole vmlinux
and that vmlinux file is the running kernel vmlinux.
And since the default now is to read BTF info, then it will do all its
operations on /sys/kernel/btf/vmlinux, when present, i.e. want to know
what are the fattest data structures in the running kernel:
[root@quaco ~]# pahole -s | sort -k2 -nr | head
cmp_data 290904 1
dec_data 274520 1
cpu_entry_area 217088 0
pglist_data 172928 4
saved_cmdlines_buffer 131104 1
debug_store_buffers 131072 0
hid_parser 110848 1
hid_local 110608 0
zonelist 81936 0
e820_table 64004 0
[root@quaco ~]#
How many data structures in the running kernel vmlinux area embbed
'struct list_head'?
[root@quaco ~]# pahole -i list_head | wc -l
260
[root@quaco ~]#
Lets see some of those?
[root@quaco ~]# pahole -C fsnotify_event
struct fsnotify_event {
struct list_head list; /* 0 16 */
struct inode * inode; /* 16 8 */
/* size: 24, cachelines: 1, members: 2 */
/* last cacheline: 24 bytes */
};
[root@quaco ~]# pahole -C audit_chunk
struct audit_chunk {
struct list_head hash; /* 0 16 */
long unsigned int key; /* 16 8 */
struct fsnotify_mark * mark; /* 24 8 */
struct list_head trees; /* 32 16 */
int count; /* 48 4 */
/* XXX 4 bytes hole, try to pack */
atomic_long_t refs; /* 56 8 */
/* --- cacheline 1 boundary (64 bytes) --- */
struct callback_head head; /* 64 16 */
struct node owners[]; /* 80 0 */
/* size: 80, cachelines: 2, members: 8 */
/* sum members: 76, holes: 1, sum holes: 4 */
/* last cacheline: 16 bytes */
};
[root@quaco ~]#
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We use things like DW_AT_alignment, so not all of those attributes are
inferred by formats like BTF that lack that info, allow suppressing the
output and make btfdiff ask for both DWARF and BTF output to have this
suppressed.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For instance, DWARF has DW_AT_alignment, and some output features
require that, so let loaders advertise such things, next patch will use
this info.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For the usual idiom to ask if a tag is a pointer, removing a bit of
DWARFism and shortening the operation.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that we can use it in things like btfdiff.
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mark Wielaard <mark@klomp.org>
Cc: Yonghong Song <yhs@fb.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
After we made -C apply for unions we had to stop calling
class__find_holes() on them, but forgot to also filter them out when
using --packable, which lead us to, in print_packable_info to access
class->priv for unions, as they were not filtered out, which made
it think that class->priv had the reordered cloned class, b00m.
Fix it by filtering out unions when doing --packable.
Fixes: 3ffe5ba93b ("pahole: Do not apply 'struct class' filters to 'struct type'")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Existing code base assumes that single CU doesn't have more than 65535
types per each CU, which might be a reasonable assumption for DWARF
data. With BTF, though, all we get is single, potentially huge, CU which
can easily have more than 65k types. For example, this is the case for
allyesconfig version of Linux kernel, which has >200k types.
Due to this assumption, libdwarves and other parts of pahole are using
16-bit counters to iterate over entities within CU. This can cause
infinite loops when iterating BTF data, if there are more than 65535
types. This patch changes non-public variables to use 32-bit integers,
where appropriate.
This still leads to invalid reported data when using BTF loader (due to using
(X & 0xFFFF) type ID, instead of X, when X > 65535) and loading huge files,
but at least it's not stuck in an infinite loop anymore.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
[ Removed non type ID conversions, for instance for the kind of tag, like in type->namespace.tag.tag, that can remain a uint16_t ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now unions are handled as well in pahole, so bail out before checking
'struct class' members for struct specific filters when handling
non-structs in class__filter()
Tested-by: Andrii Nakryiko <andriin@fb.com>
Fixes: 31664d60ad ("pahole: Show tagged enums as well when no class is specified")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch fixed two issues with BTF. One is related to struct/union
bitfield encoding and the other is related to forward type.
Issue #1 and solution:
======================
Current btf encoding of bitfield follows what pahole generates.
For each bitfield, pahole will duplicate the type chain and
put the bitfield size at the final int or enum type.
Since the BTF enum type cannot encode bit size,
commit b18354f64c ("btf: Generate correct struct bitfield
member types") workarounds the issue by generating
an int type whenever the enum bit size is not 32.
The above workaround is not ideal as we lost original type
in BTF. Another undesiable fact is the type duplication
as the pahole duplicates the type chain.
To fix this issue, this patch implemented a compatible
change for BTF struct type encoding:
. the bit 31 of type->info, previously reserved,
now is used to indicate whether bitfield_size is
encoded in btf_member or not.
. if bit 31 of struct_type->info is set,
btf_member->offset will encode like:
bit 0 - 23: bit offset
bit 24 - 31: bitfield size
if bit 31 is not set, the old behavior is preserved:
bit 0 - 31: bit offset
So if the struct contains a bit field, the maximum bit offset
will be reduced to (2^24 - 1) instead of MAX_UINT. The maximum
bitfield size will be 255 which is enough for today as maximum
bitfield in compiler can be 128 where int128 type is supported.
A new global, no_bitfield_type_recode, is introduced and which
will be set to true if BTF encoding is enabled. This global
will prevent pahole duplicating the bitfield types to avoid
type duplication in BTF.
Issue #2 and solution:
======================
Current forward type in BTF does not specify whether the original
type is struct or union. This will not work for type pretty print
and BTF-to-header-file conversion as struct/union must be specified.
To fix this issue, similar to issue #1, type->info bit 31
is used. If the bit is set, it is union type. Otherwise, it is
a struct type.
Examples:
=========
-bash-4.4$ cat t.c
struct s;
union u;
typedef int ___int;
enum A { A1, A2, A3 };
struct t {
int a[5];
___int b:4;
volatile enum A c:4;
struct s *p1;
union u *p2;
} g;
-bash-4.4$ gcc -c -O2 -g t.c
Without this patch:
$ pahole -JV t.o
[1] TYPEDEF ___int type_id=2
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] ENUM A size=4 vlen=3
A1 val=0
A2 val=1
A3 val=2
[4] STRUCT t size=40 vlen=5
a type_id=5 bits_offset=0
b type_id=13 bits_offset=160
c type_id=15 bits_offset=164
p1 type_id=9 bits_offset=192
p2 type_id=11 bits_offset=256
[5] ARRAY (anon) type_id=2 index_type_id=2 nr_elems=5
[6] INT sizetype size=8 bit_offset=0 nr_bits=64 encoding=(none)
[7] VOLATILE (anon) type_id=3
[8] FWD s type_id=0
[9] PTR (anon) type_id=8
[10] FWD u type_id=0
[11] PTR (anon) type_id=10
[12] INT int size=1 bit_offset=0 nr_bits=4 encoding=(none)
[13] TYPEDEF ___int type_id=12
[14] INT (anon) size=1 bit_offset=0 nr_bits=4 encoding=SIGNED
[15] VOLATILE (anon) type_id=14
With this patch:
$ pahole -JV t.o
File t.o:
[1] TYPEDEF ___int type_id=2
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] ENUM A size=4 vlen=3
A1 val=0
A2 val=1
A3 val=2
[4] STRUCT t kind_flag=1 size=40 vlen=5
a type_id=5 bitfield_size=0 bits_offset=0
b type_id=1 bitfield_size=4 bits_offset=160
c type_id=7 bitfield_size=4 bits_offset=164
p1 type_id=9 bitfield_size=0 bits_offset=192
p2 type_id=11 bitfield_size=0 bits_offset=256
[5] ARRAY (anon) type_id=2 index_type_id=2 nr_elems=5
[6] INT sizetype size=8 bit_offset=0 nr_bits=64 encoding=(none)
[7] VOLATILE (anon) type_id=3
[8] FWD s struct
[9] PTR (anon) type_id=8
[10] FWD u union
[11] PTR (anon) type_id=10
The fix removed the type duplication, preserved the enum type for the
bitfield, and have correct struct/union information for the forward
type.
Signed-off-by: Yonghong Song <yhs@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently, the pahole dwarf->btf conversion only supports one
compilation unit. This is not ideal since we would like using pahole to
generate BTF for vmlinux which has a lot of compilation units.
This patch added support to process multiple compilation units per ELF
file. Multiple ELF files are also supported properly.
The following is a demonstration example:
-bash-4.4$ cat t1.c
struct t1 {
int a1;
} g1;
int main(void) { return 0; }
-bash-4.4$ cat t2.c
struct t2 {
char a2;
} g2;
int main() { return 0; }
-bash-4.4$ cat t3.c
struct t3 {
unsigned char a1:4;
} g1;
int main(void) { return 0; }
-bash-4.4$ cat t4.c
struct t4 {
volatile char a4;
} g2;
int main() { return 0; }
-bash-4.4$ gcc -O2 -o t1 -g t1.c t2.c
-bash-4.4$ gcc -O2 -o t3 -g t3.c t4.c
Note that both the binary "t1" and "t3" have two compilation units in
their respective dwarf debug_info sections. The following is the pahole
verbose output for BTF conversion for these two binaries.
-bash-4.4$ pahole -JV t1 t3
File t1:
[1] STRUCT t1 size=4 vlen=1
a1 type_id=2 bits_offset=0
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
[3] STRUCT t2 size=1 vlen=1
a2 type_id=4 bits_offset=0
[4] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[5] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
File t3:
[1] STRUCT t3 size=1 vlen=1
a1 type_id=3 bits_offset=0
[2] INT unsigned char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[3] INT unsigned char size=1 bit_offset=0 nr_bits=4 encoding=(none)
[4] INT (anon) size=4 bit_offset=0 nr_bits=32 encoding=(none)
[5] STRUCT t4 size=1 vlen=1
a4 type_id=6 bits_offset=0
[6] VOLATILE (anon) type_id=7
[7] INT char size=1 bit_offset=0 nr_bits=8 encoding=(none)
[8] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The pahole tool initial goal was to show struct holes, which can't
happen with a first level of a tagged union (a union with a name),
so those only were displayed when part of a higher level struct.
Show the first level tagged enums as well, as this is useful when just
wanting to see its members.
E.g.:
$ pahole ../build/v4.20-rc5/net/core/sock.o | grep ^union
union fpregs_state {
union irq_stack_union {
union sigval {
union __sifields {
union thread_union {
union kernfs_node_id {
union flowi_uli {
union ethtool_flow_union {
union key_payload {
union bpf_attr {
union tcp_md5_addr {
$
Suggested-by: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This patch introduces BPF Type Format (BTF).
BTF (BPF Type Format) is the meta data format which describes
the data types of BPF program/map. Hence, it basically focus
on the C programming language which the modern BPF is primary
using. The first use case is to provide a generic pretty print
capability for a BPF map.
Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Seems like a nice shortcut for kernel hackers, and one that makes sure
the right vmlinux file is used, as it reads the running kernel build id
from /sys/kernel/notes and then searches the well known path for vmlinux
files:
vmlinux
/boot/vmlinux
/boot/vmlinux-4.14.0+
/usr/lib/debug/boot/vmlinux-`uname -r`
/lib/modules/`uname -r`/build/vmlinux
/usr/lib/debug/lib/modules/`uname -r`/vmlinux
/usr/lib/debug/boot/vmlinux-`uname -r`.debug
To find one with a matching build id (.notes ELF section, then
nhdr->n_type == NT_GNU_BUILD_ID), just like the Linux kernel 'perf
tools', where this code comes from, with some minor modifications to
cope with not having symbol_conf, symfs, etc.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
$ cat foo
cat: foo: No such file or directory
Before:
$ pahole foo
pahole: No debugging information found
After:
$ pahole foo
pahole: foo: No such file or directory
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As Thomas Gleixner wisely pointed out, using 'self' is stupid, it
doesn't convey useful information, so use sensible names.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
.debug_types is a new DWARF 4 feature that adds some simple
compression to DWARF. The basic idea is that identical type
definitions are merged by the linker and put into a new .debug_types
section.
This introduces 'dwarf_off_ref', which is like Dwarf_Off, but carries
a flag indicating if the DIE came from .debug_types. This allows
future uses to find the DIE properly.
Type units are read a little differently from ordinary CUs. All type
units are read at once, then types are recoded for them. (I think
something like this is needed more generally, to support inter-CU
references, but I have not tried that.)
The type unit is also kept around longer, because any other CU may
refer to it. This necessitated a change to load_steal_kind to replace
the notion of a "stolen" CU with a request by the "stealer" for the
caller to delete the CU when appropriate.
I need elfutils patch 581c3f60e2b1fc7ddaf4260bb5a9cb59f8e3f0d0
to make this work properly; without it I see crashes.
You can make test cases by compiling with '-gdwarf-4 -fdebug-types-section'.
As it is corrupting the obstack and it will be deleted completely at
cu__delete time anyway. Stick a FIXME tho, as it is good to completely
understand and fix this eventually.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
That asks dwarf_fprintf to always use "struct" in places where it would
use "class", because CTF doesn't have the "class" concept, so for
'regtest diffctf' sake, we use this.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For now --encode_ctf/-Z only encodes the tags in the first object file
(CU) in a multi-obj/CU file, so for regtest sake, for now, introduce
this option.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Where we end up with the same struct but class__size() returns off by
some bytes result. Investigating...
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Only when debugging leaks we need to be so judicious.
When not debugging we want to exit faster :-)
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We really only need structures__add, that now returns an existing entry
or add a new one, the code that uses it just passes a bool to figure out
if this is a new entry or an existing one.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Replicating the comment added to the source code:
No sense in adding an anonymous struct to the list of structs already
printed, as we look for the name... The right fix probably will be to
call class__fprintf on a in-memory FILE, do a hash, and look it by full
contents, not by name. And this is needed for CTF as well, but its late
now and I'm sleepy, will leave for later...
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I wasn't especifying the optimization level and the default, despite
using -Wall, was for this so simple case not to be warned about, so
now I'm using -O2.
Alexandre provided a patch initializing the variables to NULL, so that
when we called cus__delete it would bail out and not possibly act on
a random value, I preferred to add extra goto labels and do the exit
path only on the resources that were successfully allocated/initialized,
avoiding, for instance, to call dwarves_exit() if dwarves_init() wasn't
called, which wasn't a problem so far, but could be in the future.
Reported-by: Alexandre Vassalotti <alexandre@peadrop.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Instead pass thru cu__strings(cu, i) so that we can figure out if the
underlying debugging format handler can do that more efficiently, such as by
looking up directly the ELF section ".strtab".
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It ain't gonna be like that soon, when every cu that comes from a CTF file will
have all its strings_t pointing to strings in .strtab.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Too DWARF specific and wasn't working since I implemented type recoding and
removed the Dwarf_Off from struct tag.
To achieve the same result one can use --show_decl_info that also shows the
dwarf offset, since it needs to keep the DWARF specific line number and file
and then use a text editor to find the offset.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Arnaldo Carvalho de Melo
Andrii Nakryiko
Domenico Andreoli
Yonghong Song
Martin KaFai Lau
Tom Tromey