pahole -J .tmp_linux.btf during Linux build process always crashes
on my system.
Problem is that when gobuffer is initialized via gobuffer__init(),
it is in state where 'index' (AKA its size) is set to 1, but
'entries' is NULL.
State corrects itself if 'gobuffer__add()' is invoked, as that
will allocate buffer (even if added len == 0). But if __add()
is never invoked because only anonymous symbols are present,
one ends up with gobuffer that crashes gobuffer__copy.
Instead of allocating single-byte buffer always I opted for
checking if gobuffer entries is NULL before use in copy and
compress - gobuffer__init() would need prototype change to
report malloc failures, and it seems unnecessary to allocate
memory always - even if file does not have any symbols after
all.
(gdb) bt
#0 __memmove_sse2_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:287
#1 0x00007f2f2c1ec2ee in btf_elf__encode (btfe=0x5654e31e2e30, flags=flags@entry=0 '\000') at libbtf.c:750
#2 0x00007f2f2c1e9af0 in btf_encoder__encode () at btf_encoder.c:164
#3 0x00005654e2407599 in main (argc=3, argv=0x7ffcd8783f18) at pahole.c:1344
(gdb) frame 1
#1 0x00007f2f2c1ec2ee in btf_elf__encode (btfe=0x5654e31e2e30, flags=flags@entry=0 '\000') at libbtf.c:750
750 gobuffer__copy(btfe->strings, btf_elf__nohdr_data(btfe) + hdr->str_off);
(gdb) print btfe->strings
$1 = (struct gobuffer *) 0x5654e31db2c8
(gdb) print *btfe->strings
$2 = {entries = 0x0, nr_entries = 0, index = 1, allocated_size = 0}
(gdb) print btfe->types
$3 = {entries = 0x5654e31e2ef0 "", nr_entries = 1, index = 16, allocated_size = 8192}
(gdb) x /16bx btfe->types.entries
0x5654e31e2ef0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01
0x5654e31e2ef8: 0x04 0x00 0x00 0x00 0x20 0x00 0x00 0x00
Signed-off-by: Petr Vandrovec <petr@vmware.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
That str variable _will_ be initialized at that point, but lets set it
to NULL to silence gcc version 4.8.5 20150623.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
The default is to show structs and unions, if just structs should be
considered, use --structs, just don't use it together with --unions or
nothing will be shown 8-)
$ pahole --find_pointers_to ehci_qh
ehci_hcd: qh_scan_next
ehci_hcd: async
ehci_hcd: dummy
ehci_shadow: qh
$ pahole --structs --find_pointers_to ehci_qh
ehci_hcd: qh_scan_next
ehci_hcd: async
ehci_hcd: dummy
$ pahole --unions --find_pointers_to ehci_qh
ehci_shadow: qh
$ pahole --structs --unions --find_pointers_to ehci_qh
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I.e. when looking for unions or structs that contains/embeds or looking
for unions/structs that have pointers to a given type.
E.g:
$ pahole --contains inet_sock
sctp_sock
inet_connection_sock
raw_sock
udp_sock
raw6_sock
$ pahole --unions --contains inet_sock
$
We have structs embedding 'struct inet_sock', but no unions doing that.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When we started supporting unions in pahole some corner cases where
classes were expected needed fixing, this was one such case, that
printed bogus number of holes when handling unions, that is represented
as a 'struct type', that doesn't have ->nr_holes, just 'struct class'
that is has 'struct type' as its first member, i.e. extends it.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I.e. no point in looking for holes in unions.
Now 'pahole -H 10' will only show the structs with 10 or more alignment
holes, not those + all unions.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Needed to find the right 'struct ftype' to iterate function arguments
on. This is due to how BTF works with this and how we implemented it, at
some point this can get improved to avoid the need for checking if it is
BTF, doing it in a more format abstracted way, but for now, abstract
this away in the for_each_parameter helpers.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We had to use the same hack as in pfunct, as implemented in ccf3eebfcd
("btf_loader: Add support for BTF_KIND_FUNC"), will hide that 'struct
ftype' (aka function prototype) indirection behind the parameter
iterator (function__for_each_parameter).
For now, here is the top 10 Linux kernel data structures in terms of
number of functions receiving as one of its parameters a pointer to it,
using /sys/kernel/btf/vmlinux to look at all the vmlinux types and
functions (the ones visible in kallsyms, but with the parameters and its
types):
$ pahole -m | sort -k2 -nr | head
device 955
sock 568
sk_buff 541
task_struct 437
inode 421
pci_dev 390
page 351
net_device 347
file 315
net 312
$
$ pahole --help |& grep -- -m
-m, --nr_methods show number of methods
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we default to operating on the running kernel data structures, we
should make the default to, with no options passed, to pretty print all
the running kernel data structures, or do what was asked in terms of
number of members, size of structs, etc, i.e.:
[root@quaco ~]# pahole --help |& head
Usage: pahole [OPTION...] FILE
-a, --anon_include include anonymous classes
-A, --nested_anon_include include nested (inside other structs) anonymous
classes
-B, --bit_holes=NR_HOLES Show only structs at least NR_HOLES bit holes
-c, --cacheline_size=SIZE set cacheline size to SIZE
--classes_as_structs Use 'struct' when printing classes
-C, --class_name=CLASS_NAME Show just this class
-d, --recursive recursive mode, affects several other flags
[root@quaco ~]#
Continues working as before, but if you do:
pahole
It will work just as if you did:
pahole vmlinux
and that vmlinux file is the running kernel vmlinux.
And since the default now is to read BTF info, then it will do all its
operations on /sys/kernel/btf/vmlinux, when present, i.e. want to know
what are the fattest data structures in the running kernel:
[root@quaco ~]# pahole -s | sort -k2 -nr | head
cmp_data 290904 1
dec_data 274520 1
cpu_entry_area 217088 0
pglist_data 172928 4
saved_cmdlines_buffer 131104 1
debug_store_buffers 131072 0
hid_parser 110848 1
hid_local 110608 0
zonelist 81936 0
e820_table 64004 0
[root@quaco ~]#
How many data structures in the running kernel vmlinux area embbed
'struct list_head'?
[root@quaco ~]# pahole -i list_head | wc -l
260
[root@quaco ~]#
Lets see some of those?
[root@quaco ~]# pahole -C fsnotify_event
struct fsnotify_event {
struct list_head list; /* 0 16 */
struct inode * inode; /* 16 8 */
/* size: 24, cachelines: 1, members: 2 */
/* last cacheline: 24 bytes */
};
[root@quaco ~]# pahole -C audit_chunk
struct audit_chunk {
struct list_head hash; /* 0 16 */
long unsigned int key; /* 16 8 */
struct fsnotify_mark * mark; /* 24 8 */
struct list_head trees; /* 32 16 */
int count; /* 48 4 */
/* XXX 4 bytes hole, try to pack */
atomic_long_t refs; /* 56 8 */
/* --- cacheline 1 boundary (64 bytes) --- */
struct callback_head head; /* 64 16 */
struct node owners[]; /* 80 0 */
/* size: 80, cachelines: 2, members: 8 */
/* sum members: 76, holes: 1, sum holes: 4 */
/* last cacheline: 16 bytes */
};
[root@quaco ~]#
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When we call the tools without specifying a file format, all the
available format loaders are called, in sequence, till we find the
relevant information, now that we support raw BTF we better move it in
front of the CTF one, as it is way more common in the Linux community.
With this we will not see this warning anymore:
$ pahole -C list_head /sys/kernel/btf/vmlinux
ctf__new: cannot get elf header.
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$
That warning has to go, so that other formats, if present after the CTF
one, can be tried.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Be it automatically when no -F option is passed and
/sys/kernel/btf/vmlinux is available, or when /sys/kernel/btf/vmlinux is
passed as the filename to the tool, i.e.:
$ pahole -C list_head
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$ strace -e openat pahole -C list_head |& grep /sys/kernel/btf/
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
$
$ pahole -C list_head /sys/kernel/btf/vmlinux
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$
If one wants to grab the matching vmlinux to use its DWARF info instead,
which is useful to compare the results with what we have from BTF, for
instance, its just a matter of using '-F dwarf'.
This in turn shows something that at first came as a surprise, but then
has a simple explanation:
For very common data structures, that will probably appear in all of the
DWARF CUs (Compilation Units), like 'struct list_head', using '-F dwarf'
is faster:
[acme@quaco pahole]$ perf stat -e cycles pahole -F btf -C list_head > /dev/null
Performance counter stats for 'pahole -F btf -C list_head':
45,722,518 cycles:u
0.023717300 seconds time elapsed
0.016474000 seconds user
0.007212000 seconds sys
[acme@quaco pahole]$ perf stat -e cycles pahole -F dwarf -C list_head > /dev/null
Performance counter stats for 'pahole -F dwarf -C list_head':
14,170,321 cycles:u
0.006668904 seconds time elapsed
0.005562000 seconds user
0.001109000 seconds sys
[acme@quaco pahole]$
But for something that is more specific to a subsystem, the DWARF loader
will have to process way more stuff till it gets to that struct:
$ perf stat -e cycles pahole -F dwarf -C tcp_sock > /dev/null
Performance counter stats for 'pahole -F dwarf -C tcp_sock':
31,579,795,238 cycles:u
8.332272930 seconds time elapsed
8.032124000 seconds user
0.286537000 seconds sys
$
While using the BTF loader the time should be constant, as it loads
everything from /sys/kernel/btf/vmlinux:
$ perf stat -e cycles pahole -F btf -C tcp_sock > /dev/null
Performance counter stats for 'pahole -F btf -C tcp_sock':
48,823,488 cycles:u
0.024102760 seconds time elapsed
0.012035000 seconds user
0.012046000 seconds sys
$
Above I used '-F btf' just to show that it can be used, but its not
really needed, i.e. those are equivalent:
$ strace -e openat pahole -F btf -C list_head |& grep /sys/kernel/btf/vmlinux
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
$ strace -e openat pahole -C list_head |& grep /sys/kernel/btf/vmlinux
openat(AT_FDCWD, "/sys/kernel/btf/vmlinux", O_RDONLY) = 3
$
The btf_raw__load() function that ends up being grafted into the
preexisting btf_elf routines was based on libbpf's btf_load_raw().
Acked-by: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Addresses this warning:
[ 67%] Building C object CMakeFiles/scncopy.dir/elfcreator.c.o
/home/acme/git/pahole/elfcreator.c: In function ‘rel_dyn_fixup_fn’:
/home/acme/git/pahole/elfcreator.c:229:3: error: ‘idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
229 | gelf_update_dyn(ctor->dyndata, idx, dyn);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/acme/git/pahole/elfcreator.c: In function ‘rela_dyn_fixup_fn’:
/home/acme/git/pahole/elfcreator.c:208:3: error: ‘idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
208 | gelf_update_dyn(ctor->dyndata, idx, dyn);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/acme/git/pahole/elfcreator.c: In function ‘elfcreator_end’:
/home/acme/git/pahole/elfcreator.c:192:3: error: ‘idx’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
192 | gelf_update_dyn(ctor->dyndata, idx, dyn);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/acme/git/pahole/elfcreator.c:186:9: note: ‘idx’ was declared here
186 | size_t idx;
| ^~~
cc1: all warnings being treated as errors
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Addresses this warning:
[ 26%] Building C object CMakeFiles/dwarves.dir/dwarves_fprintf.c.o
/home/acme/git/pahole/dwarves_fprintf.c: In function ‘type__fprintf’:
/home/acme/git/pahole/dwarves_fprintf.c:709:47: error: ‘%s’ directive output may be truncated writing up to 257 bytes into a region of size 256 [-Werror=format-truncation=]
709 | snprintf(namebfptr, sizeof(namebfptr), "* %s", name);
| ^~
/home/acme/git/pahole/dwarves_fprintf.c:709:5: note: ‘snprintf’ output between 3 and 260 bytes into a destination of size 258
709 | snprintf(namebfptr, sizeof(namebfptr), "* %s", name);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Since we iterate over these ptr_tables entry by entry, we better zero
out the new ranges, i.e. whatever is after the previous allocated space
up to the new one, returned by realloc.
Fixes a bug found using pahole -F btf on a vmlinux file, now btfdiff on
this same file comes clean, i.e. the output from its BTF tags is the
same as with its DWARF ones, for the features present in both type
information formats.
Tested-by: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
That was affecting --suppress_aligned_attribute, detected using the
'btfdiff' utility, that uses pahole with DWARF and BTF and compares
their outputs, and since we don't have this in BTF, we use
--suppress_aligned_attribute with '-F dwarf'.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It was possible to have an infinite loop trying to determine the
alignment of a type that had a static data member of its own type.
Example code illustrating the problem.
struct X { static X thing; };
Committer testing:
Before:
$ pwd
/home/acme/git/pahole/examples/gareth
$ cat static_member.cpp
struct X { static X thing; };
static struct X f;
$ g++ -g -c static_member.cpp -o static_member.o
$ pahole static_member.o
Segmentation fault (core dumped)
$
After:
$ pahole static_member.o
struct X {
static struct X thing; /* 0 0 */
/* XXX last struct has 1 byte of padding */
/* size: 1, cachelines: 0, members: 0, static members: 1 */
/* padding: 1 */
/* paddings: 1, sum paddings: 1 */
/* last cacheline: 1 bytes */
};
$
Signed-off-by: Gareth Lloyd <gareth.lloyd@uk.ibm.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
while adding the RHEL package one of the checks failed
saying that there's no actual version dependency between
dwarves nad libdwarves1:
Subpackage dwarves on aarch64 x86_64 ppc64le s390x consumes libraries libdwarves.so.1()(64bit) libdwarves_emit.so.1()(64bit) libdwarves_reorganize.so.1()(64bit) from subpackage libdwarves1 but does not have explicit package version requirement.
Please add Requires: libdwarves1 = %{version}-%{release} to dwarves in the specfile to avoid the need to test interoperability between the various combinations of old and new subpackages.
It's also not present in fedora and I think that's what we want,
because it's also present for -devel pacakge.. please consider
patch below for fedora package.
koji: https://koji.fedoraproject.org/koji/taskinfo?taskID=38777298
[jolsa@krava trash]$ rpm -qp --requires dwarves-1.15-4.fc32.x86_64.rpm | grep libdwarves1
libdwarves1 = 1.15-4.fc32
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Martin Cermak <mcermak@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We were using the fall back for that, i.e. 'return 0;' was being emitted
for a function returning void, noticed with using BTF as the format.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Just skip them, we have no use for them so far, skipping them will allow
us to process the other kinds we have use for.
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To agree with what ptr_table__add() does, i.e. the number of entries
(pt->nr_entries) is in fact the max id number in the table + 1, we can't
allow to add an id that is not contiguous and simply increment
nr_entries, as cu__for_each_variable(cu, id, pos) will iterate based on
nr_entries, skipping NULL buckets.
Detected when adding support for BTF variables, that share the id space
with types, but end up in different ptr_tables, so we end up with:
cu__table_add_tag(BTF_KIND_VAR)
ptr_table__add_with_id(cu->tags_table)
but:
cu__table_add_tag()
ptr_table__add_with_id(cu->types_table)
for all the other BTF_KIND_s, fix it so that ptr->nr_entries copes with
not receiving strictly monotonically incremented by one ids.
We'll have to fix this more properly by defining which tags are
supported by some debugging format and this what should go to a
ptr_table so that we have all its entries monononically incremented by
one and avoid tag__check_id_drift() returning true in the per debugging
format encoders (e.g.: btf_encoder.c).
This patch allows us to move forward and apply the BTF_KIND_VAR patch
where we'll at least produce, for global variables and a simple BPF
program with both DWARF and BTF tags the same results:
[root@quaco tracebuffer]# pglobal -F dwarf -v bristot.o
struct ____btf_map_tracebuffer__bristot ____btf_map_tracebuffer__bristot;; /* 0 */
char _license[4];; /* 0 */
int _version;; /* 0 */
struct bpf_map tracebuffer__bristot;; /* 0 */
[root@quaco tracebuffer]# pglobal -F btf -v bristot.o
BTF: idx: 17, off: 352, Unknown kind 15
BTF: idx: 18, off: 364, Unknown kind 0
BTF: idx: 19, off: 376, Unknown kind 15
BTF: idx: 20, off: 388, Unknown kind 0
BTF: idx: 21, off: 400, Unknown kind 15
BTF: idx: 22, off: 412, Unknown kind 0
BTF: idx: 23, off: 424, Unknown kind 15
BTF: idx: 24, off: 436, Unknown kind 0
struct ____btf_map_tracebuffer__bristot ____btf_map_tracebuffer__bristot;; /* 0 */
char _license[4];; /* 0 */
int _version;; /* 0 */
struct bpf_map tracebuffer__bristot;; /* 0 */
[root@quaco tracebuffer]#
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Daniel Bristot de Oliveira <bristot@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Instead of just telling that some unknown BTF_KIND_ was found in a file,
show the number to help in debugging.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I.e. now we can, just like with pahole, use:
pglobal -F btf --variable foo.o
To get the global variables.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
To avoid using /usr/include/linux/btf.h, as it has:
[acme@quaco pahole]$ grep ^# /usr/include/linux/btf.h | head -2
#ifndef __LINUX_BTF_H__
#define __LINUX_BTF_H__
[acme@quaco pahole]$
While:
[acme@quaco pahole]$ grep ^# lib/bpf/include/uapi/linux/btf.h | head -2
#ifndef _UAPI__LINUX_BTF_H__
#define _UAPI__LINUX_BTF_H__
[acme@quaco pahole]$
Then when both get included we get duplicate definitions, to avoid that
put lib/bpf/include/uapi/ first in the include path for libbdwarves.
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Domenico reported that the README.DEBUG and changes-v1.13 files were not
in MANIFEST, so were not making into the release tarballs, fix that.
Reported-by: Domenico Andreoli <domenico.andreoli@linux.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
As there are cases where we jump to the type not found label without
having done the full copy of *conf to tconf, so at least the
type_spacing needs to have been set.
This addresses this coverity report entry:
Error: UNINIT (CWE-457): [#def23]
dwarves-1.13/dwarves_fprintf.c:600: var_decl: Declaring variable "tconf" without initializer.
dwarves-1.13/dwarves_fprintf.c:779: uninit_use_in_call: Using uninitialized value "tconf.type_spacing" when calling "fprintf".
# 777| return printed;
# 778| out_type_not_found:
# 779|-> printed = fprintf(fp, "%-*s %s", tconf.type_spacing, "<ERROR>", name);
# 780| goto out;
# 781| }
Reported-by: William Cohen <wcohen@redhat.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In some exit paths we were accessing tconf before we had copied it from
conf, and we also were losing track of the original type that could have
been expanded and where we bumped the recursivity level member, so just
store that original type and if it is set, decrement its recursivity
level.
This addresses these coverity report entries:
Error: UNINIT (CWE-457): [#def21]
dwarves-1.13/dwarves_fprintf.c:600: var_decl: Declaring variable "tconf" without initializer.
dwarves-1.13/dwarves_fprintf.c:774: uninit_use: Using uninitialized value "tconf.expand_types".
# 772| }
# 773| out:
# 774|-> if (tconf.expand_types)
# 775| --type->recursivity_level;
# 776|
Error: FORWARD_NULL (CWE-476): [#def22]
dwarves-1.13/dwarves_fprintf.c:605: var_compare_op: Comparing "type" to null implies that "type" might be null.
dwarves-1.13/dwarves_fprintf.c:775: var_deref_op: Dereferencing null pointer "type".
# 773| out:
# 774| if (tconf.expand_types)
# 775|-> --type->recursivity_level;
# 776|
# 777| return printed;
Reported-by: William Cohen <wcohen@redhat.com>
Fixes: f84bf73d54 ("dwarves: Move the fprintf code to a new source file.")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>