This appeared in DWARF4 but is supported only in gcc's -gdwarf-5,
support it in a way that makes the output be the same for both cases:
$ gcc -gdwarf-4 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int a:1; /* 0: 0 8 */
long int b:1; /* 0: 1 8 */
long int c:1; /* 0: 2 8 */
/* XXX 29 bits hole, try to pack */
/* Bitfield combined with next fields */
int after_bitfield; /* 4 4 */
/* size: 8, cachelines: 1, members: 4 */
/* sum members: 4 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* last cacheline: 8 bytes */
};
$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int a:1; /* 0: 0 8 */
long int b:1; /* 0: 1 8 */
long int c:1; /* 0: 2 8 */
/* XXX 29 bits hole, try to pack */
/* Bitfield combined with next fields */
int after_bitfield; /* 4 4 */
/* size: 8, cachelines: 1, members: 4 */
/* sum members: 4 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* last cacheline: 8 bytes */
};
$
Now with an integer before the bitfield:
$ cat examples/dwarf5/bf.c
struct pea {
int before_bitfield;
long a:1, b:1, c:1;
int after_bitfield;
} p;
$ gcc -gdwarf-4 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
int before_bitfield; /* 0 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 0:32 8 */
long int b:1; /* 0:33 8 */
long int c:1; /* 0:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 8 4 */
/* size: 16, cachelines: 1, members: 5 */
/* sum members: 8 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 16 bytes */
};
$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
int before_bitfield; /* 0 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 0:32 8 */
long int b:1; /* 0:33 8 */
long int c:1; /* 0:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 8 4 */
/* size: 16, cachelines: 1, members: 5 */
/* sum members: 8 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 16 bytes */
};
$
And an array of long integers at the start, before the combination of an
integer with a long integer bitfield:
$ cat examples/dwarf5/bf.c
struct pea {
long array[3];
int before_bitfield;
long a:1, b:1, c:1;
int after_bitfield;
} p;
$ gcc -gdwarf-4 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int array[3]; /* 0 24 */
int before_bitfield; /* 24 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 24:32 8 */
long int b:1; /* 24:33 8 */
long int c:1; /* 24:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 32 4 */
/* size: 40, cachelines: 1, members: 6 */
/* sum members: 32 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 40 bytes */
};
$ gcc -gdwarf-5 -c examples/dwarf5/bf.c
$ pahole bf.o
struct pea {
long int array[3]; /* 0 24 */
int before_bitfield; /* 24 4 */
/* Bitfield combined with previous fields */
long int a:1; /* 24:32 8 */
long int b:1; /* 24:33 8 */
long int c:1; /* 24:34 8 */
/* XXX 29 bits hole, try to pack */
int after_bitfield; /* 32 4 */
/* size: 40, cachelines: 1, members: 6 */
/* sum members: 32 */
/* sum bitfield members: 3 bits, bit holes: 1, sum bit holes: 29 bits */
/* padding: 4 */
/* last cacheline: 40 bytes */
};
$
Reported-by: Mark Wielaard <mark@klomp.org>
Tested-by: "Daniel P. Berrangé" <berrange@redhat.com>
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1919965
Link: https://lore.kernel.org/dwarves/20210128121122.GA775562@kernel.org/
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Using the newly added __attr_offset(), so that we try to read it, if it
isn't there, then we don't need to use dwarf_hasattr() for setting
member->is_static.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Sometimes we need to check if a attribute is present and if so, use its
result, so split the part that acts just on the Dwarf_Attribute from
attr_offset() and allow using it directly via __attr_offset().
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For very large ELF objects (with many sections), we could get special
value SHN_XINDEX (65535) for symbol's st_shndx.
This patch is adding code to detect the optional extended section index
table and use it to resolve symbol's section index.
Adding elf_symtab__for_each_symbol_index macro that returns symbol's
section index and usign it in collect functions.
Tested by running pahole on kernel compiled with:
make KCFLAGS="-ffunction-sections -fdata-sections" -j$(nproc) vmlinux
and ensure FUNC records are generated and match normal build (without
above KCFLAGS).
Also bpf selftest passed and generated kernel BTF, is same as without
the patch.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Hao Luo <haoluo@google.com>
Cc: Joe Lawrence <joe.lawrence@redhat.com>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: KP Singh <kpsingh@chromium.org>
Cc: Mark Wieelard <mjw@redhat.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: netdev@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Use elf_getshdrstrndx() to cover the case where the ELF header
'e_shstrndx' field contains the special value SHN_XINDEX so that we get
the proper string table index.
This is necessary to handle files with over 65536 sections, such as when
building the kernel with -f[function|data]-sections. Other cases may
include when using FG-ASLR, LTO.
With so many sections, ELF is using extended section index table, which
is used to hold values for some of the indexes and extra code is needed
to retrieve them.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Hao Luo <haoluo@google.com>
Cc: Joe Lawrence <joe.lawrence@redhat.com>
Cc: John Fastabend <john.fastabend@gmail.com>
Cc: KP Singh <kpsingh@chromium.org>
Cc: Mark Wieelard <mjw@redhat.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Song Liu <songliubraving@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: netdev@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
When processing kernel images built by clang we can find some functions
without a name, which causes pahole to segfault.
Add extra checks to make sure we always have function's name defined
before using it.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Reported-by: Sedat Dilek <sedat.dilek@gmail.com>
Tested-by: Tom Stellard <tstellar@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a new CMake option, LIBBPF_EMBEDDED, to switch between the
embedded version and the system version (searched via pkg-config)
of libbpf. Set the embedded version as the default.
Signed-off-by: Luca Boccassi <bluca@debian.org>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Adjust pahole logic of skipping any per-CPU symbol with offset 0, which is
especially bad for kernel modules, because it most certainly skips the very
first per-CPU variable.
Instead, do collect per-CPU ELF symbol with 0 offset, but do extra check for
non-kernel module case by verifying that ELF symbol name and DWARF variable
name match. Due to the bug of DWARF name of variable sometimes being NULL,
this is necessarily too pessimistic check (e.g., on my vmlinux image,
fixed_percpu_data variable is still not emitted due to missing DWARF variable
name), it allows to emit data for all module per-CPU variables.
Fixes: f3d9054ba8 ("btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.")
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Jiri Olsa <jolsa@redhat.com>
Cc: Hao Luo <haoluo@google.com>
Cc: kernel-team@fb.com
Link: https://lore.kernel.org/r/20201211041139.589692-3-andrii@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Fix pahole's logic for determining per-CPU variables. For vmlinux,
btfe->percpu_base_addr is always 0, so it didn't matter at which point to
subtract it to get offset that later was matched against corresponding ELF
symbol.
For kernel module, though, the situation is different. Kernel module's per-CPU
data section has non-zero offset, which is taken into account in all DWARF
variable addresses calculation. For such cases, it's important to subtract
section offset (btfe->percpu_base_addr) before ELF symbol look up is
performed.
This patch also records per-CPU data section size and uses it for early
filtering of non-per-CPU variables by their address.
Fixes: 2e719cca66 ("btf_encoder: revamp how per-CPU variables are encoded")
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Jiri Olsa <jolsa@redhat.com>
Cc: kernel-team@fb.com
Cc: Hao Luo <haoluo@google.com>
Link: https://lore.kernel.org/r/20201211041139.589692-2-andrii@kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Replace `%lx' for addr (uint64_t) with PRIx64. `%ld' for seek_bytes
(off_t) is replaced with PRIx64 too, likewise in other places it's
printed.
Fixes these error messages on i586 and arm-32:
btf_encoder.c:445:52: error: format '%lx' expects argument of type 'long unsigned int', but argument 3 has type 'uint64_t'
btf_encoder.c:687:54: error: format '%lx' expects argument of type 'long unsigned int', but argument 4 has type 'uint64_t'
btf_encoder.c:695:71: error: format '%lx' expects argument of type 'long unsigned int', but argument 4 has type 'uint64_t'
btf_encoder.c:708:88: error: format '%lx' expects argument of type 'long unsigned int', but argument 4 has type 'uint64_t'
pahole.c:1872:20: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'off_t'
Signed-off-by: Cc: Vitaly Chikunov <vt@altlinux.org>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add support to detect kernel module ftrace addresses and use
it as filter for detected functions.
For kernel modules the ftrace addresses are stored in __mcount_loc
section. Adding the code that detects this section and reads
its data into array, which is then processed as filter by
current code.
There's one tricky point with kernel modules wrt Elf object,
which we get from dwfl_module_getelf function. This function
performs all possible relocations, including __mcount_loc
section.
So addrs array contains relocated values, which we need take
into account when we compare them to functions values which
are relative to their sections.
With this change for example for xfs.ko module in my kernel
config, I'm getting slightly bigger number of functions:
before: 2373, after: 2601
The ftrace's available_filter_functions still shows 2701, but
it includes functions like:
suffix_kstrtoint.constprop.0
xchk_btree_check_minrecs.isra.0
xfs_ascii_ci_compname.part.0
which are not part of dwarf data, the rest matches BTF functions.
Because of the malfunction DWARF's declaration tag, the 'before'
functions contain also functions that are not part of the module.
The 'after' functions contain only functions that are traceable
and part of xfs.ko.
Despite filtering out some declarations, this change also adds
static functions, hence the total number of functions is bigger.
Committer notes:
Andrii test notes:
<quote>
I've tested locally on bpf_testmod that I'm adding to selftests in [0].
All worked well. I changed the test function from global to non-inlined
static, and BTF had it. And the tests passed. So LGTM.
[0] https://patchwork.kernel.org/user/todo/netdevbpf/?series=395715&delegate=121173&state=*
</>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Tested-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We can't assume the address size is always size of unsigned long, we
have to use directly the ELF's address size.
Change the 'addrs' array to __u64 and convert 32 bit address values when
copying from ELF section.
Committer notes:
Jiri tested this by:
<quote>
So to test this I built 32bit vmlinux and used 64bit pahole
to generate BTF data on both vmlinux and modules, which I
thought was valid use case.
</>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Reorder the filter_functions function so we can add processing of kernel
modules in following patch.
There's no functional change intended.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We had this heuristic to try raw BTF if the file starts with
/sys/kernel/btf/, but if we are in that directory already then we
continue trying it as an ELF file, fix it:
Before:
$ cd /sys/kernel/btf/
$ pahole -C list_head vmlinux
pahole: vmlinux: No such device
$
After:
$ pahole -C list_head vmlinux
struct list_head {
struct list_head * next; /* 0 8 */
struct list_head * prev; /* 8 8 */
/* size: 16, cachelines: 1, members: 2 */
/* last cacheline: 16 bytes */
};
$
Works as well for modules:
$ pahole -C wmi_device vmlinux
pahole: type 'wmi_device' not found
$ pahole -C wmi_device --btf_base vmlinux wmi
struct wmi_device {
struct device dev; /* 0 752 */
/* XXX last struct has 7 bytes of padding */
/* --- cacheline 11 boundary (704 bytes) was 48 bytes ago --- */
bool setable; /* 752 1 */
/* size: 760, cachelines: 12, members: 2 */
/* padding: 7 */
/* paddings: 1, sum paddings: 7 */
/* last cacheline: 56 bytes */
};
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
If a .ko file has both DWARF and BTF, we were needing to use '-F btf' to
make the BTF info to be used, to make command lines more concise imply
'-F btf' if --btf_base is used.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Previously we used this just for vmlinux, now that we have kernel
modules, use it for those as well.
Now this works:
$ pahole wmi_block
pahole: type 'wmi_block' not found
$ pahole --btf_base /sys/kernel/btf/vmlinux /sys/kernel/btf/wmi -C wmi_block
struct wmi_block {
struct wmi_device dev; /* 0 760 */
/* XXX last struct has 7 bytes of padding */
/* --- cacheline 11 boundary (704 bytes) was 56 bytes ago --- */
struct list_head list; /* 760 16 */
/* --- cacheline 12 boundary (768 bytes) was 8 bytes ago --- */
struct guid_block gblock; /* 776 20 */
/* XXX 4 bytes hole, try to pack */
struct miscdevice char_dev; /* 800 80 */
/* XXX last struct has 6 bytes of padding */
/* --- cacheline 13 boundary (832 bytes) was 48 bytes ago --- */
struct mutex char_mutex; /* 880 32 */
/* --- cacheline 14 boundary (896 bytes) was 16 bytes ago --- */
struct acpi_device * acpi_device; /* 912 8 */
wmi_notify_handler handler; /* 920 8 */
void * handler_data; /* 928 8 */
u64 req_buf_size; /* 936 8 */
bool read_takes_no_args; /* 944 1 */
/* size: 952, cachelines: 15, members: 10 */
/* sum members: 941, holes: 1, sum holes: 4 */
/* padding: 7 */
/* paddings: 2, sum paddings: 13 */
/* last cacheline: 56 bytes */
};
$
I.e. it assumes /sys/kernel/btf/wmi is raw BTF, and finds in it a type
'struct wmi_block' that is not present in /sys/kernel/btf/vmlinux, i.e.
its a module specific type, that uses types that are in
/sys/kernel/btf/vmlinux, such as 'struct list_head'.
For reference, here are the sizes of those files:
$ ls -la /sys/kernel/btf/vmlinux /sys/kernel/btf/wmi
-r--r--r--. 1 root root 4241472 Nov 17 20:14 /sys/kernel/btf/vmlinux
-r--r--r--. 1 root root 2866 Nov 17 20:14 /sys/kernel/btf/wmi
$
It is also possible to use the .ko file:
$ uname -r
5.10.0-rc3.bpfsign+
$ pahole wmi_notify_handler
pahole: type 'wmi_notify_handler' not found
$ pahole -F btf --btf_base /sys/kernel/btf/vmlinux /lib/modules/5.10.0-rc3.bpfsign+/kernel/drivers/platform/x86/wmi.ko -C wmi_notify_handler
typedef void (*wmi_notify_handler)(u32, void *);
$ pahole -F btf --btf_base /sys/kernel/btf/vmlinux /lib/modules/5.10.0-rc3.bpfsign+/kernel/drivers/platform/x86/wmi.ko -C wmi_device
struct wmi_device {
struct device dev; /* 0 752 */
/* XXX last struct has 7 bytes of padding */
/* --- cacheline 11 boundary (704 bytes) was 48 bytes ago --- */
bool setable; /* 752 1 */
/* size: 760, cachelines: 12, members: 2 */
/* padding: 7 */
/* paddings: 1, sum paddings: 7 */
/* last cacheline: 56 bytes */
};
$
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Using more suitable fallback message for the case when the ftrace filter
can't be used because of missing symbols.
Committer notes:
Before:
vmlinux not detected, falling back to dwarf data
Now:
ftrace symbols not detected, falling back to DWARF data
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Requested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
With introduction of collect_symbols function, we moved the percpu
variables code before btf_elf__verbose/btf_elf__force setup, so they
don't have any effect in that code anymore.
Also btf_elf__verbose is used in code that prepares ftrace filter for
functions generations, also called within collect_symbols function.
Moving btf_elf__verbose/btf_elf__force setup early in the cu__encode_btf
function, so we can get verbose messages and see the effect of the force
option.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Current conditions for picking up function records break BTF data on
some gcc versions.
Some function records can appear with no arguments but with declaration
tag set, so moving the 'fn->declaration' in front of other checks.
Then checking if argument names are present and finally checking ftrace
filter if it's present. If ftrace filter is not available, using the
external tag to filter out non external functions.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently we skip functions under .init* sections, Removing the .init*
section check, BTF now contains also functions from .init* sections.
Andrii's explanation from email:
> ... I think we should just drop the __init check and
> include all the __init functions into BTF. There could be cases where
> we'd need to attach BPF programs to __init functions (e.g., bpf_lsm
> security cases), so having BTFs for those FUNCs are necessary as well.
> Ftrace currently disallows that, but it's only because no user-space
> application has a way to attach probes early enough. This might change
> in the future, so there is no need to invent special mechanisms now
> for bpf_iter function preservation. Let's just include all __init
> functions in BTF.
It's over ~2000 functions on my .config:
$ bpftool btf dump file ./vmlinux | grep 'FUNC ' | wc -l
41505
$ bpftool btf dump file /sys/kernel/btf/vmlinux | grep 'FUNC ' | wc -l
39256
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add support for generating split BTF, in which there is a designated base
BTF, containing a base set of types, and a split BTF, which extends main BTF
with extra types, that can reference types and strings from the main BTF.
This is going to be used to generate compact BTFs for kernel modules, with
vmlinux BTF being a main BTF, which all kernel modules are based off of.
These changes rely on patch set [0] to be present in libbpf submodule.
[0] https://patchwork.kernel.org/project/netdevbpf/list/?series=377859&state=*
Committer notes:
Fixed up wrt ARGP_numeric_version and added a man page entry.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Rename a few local variables to reflects the purpose a bit better. Also
separate writing out BTF raw data and objcopy invocation into two
separate steps and improve error reporting for each.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Take into account type ID offset, accumulated from previous CUs, when
calculating a new type ID for the generated array index type.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Cc: kernel-team@fb.com
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
In Makefiles we want to do purely numeric comparisions, such as in the
Linux kernel Makefiles and scripts, that have things like this at the
moment:
$ grep PAHOLE */*.sh
scripts/link-vmlinux.sh: if ! [ -x "$(command -v ${PAHOLE})" ]; then
scripts/link-vmlinux.sh: echo >&2 "BTF: ${1}: pahole (${PAHOLE}) is not available"
scripts/link-vmlinux.sh: pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/')
scripts/link-vmlinux.sh: echo >&2 "BTF: ${1}: pahole version $(${PAHOLE} --version) is too old, need at least v1.16"
scripts/link-vmlinux.sh: LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
$
So just provide:
$ pahole --numeric_version
118
$
While keeping the --version output for older Makefiles and scripts.
Cc: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Nothing changes now, this continues to work just the same:
$ pahole --version
v1.18
$ pfunct --version
v1.18
$
This just paves the way for us to have a '--numeric-version' that will
do away with the dot and the leading 'v' and that can be used in
Makefiles to check if the required minimum version is available, to
avoid what we have now in the Linux kernel:
config PAHOLE_HAS_SPLIT_BTF
def_bool $(success, test `$(PAHOLE) --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/'` -ge "119")
With the next cset we'll be able to do just:
test `$(PAHOLE) --numeric-version` -ge "119"
Cc: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Before we would have:
$ gcc x.c -Wall -ggdb3
$ strip -s a.out
$ pahole ./a.out
$
I.e. no types, no output.
Then when we started trying out the sole arg as a type to lookup either
on a vmlinux found via its build id in /sys/kernel/notes or
/sys/kernel/btf/vmlinux we instead showed this for a binary without type
info:
$ pahole ./a.out
pahole: type './a.out' not found
$
Now we show:
$ pahole a.out
pahole: file 'a.out' has no supported type information.
$
Reported-by: Jan Engelhardt <jengelh@inai.de>
Bugtracker: https://github.com/acmel/dwarves/issues/14
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We need to generate just single BTF instance for the function, while
DWARF data contains multiple instances of DW_TAG_subprogram tag.
Unfortunately we can no longer rely on DW_AT_declaration tag
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060)
Instead we apply following checks:
- argument names are defined for the function
- there's symbol and address defined for the function
- function is generated only once
Also because we want to follow kernel's ftrace traceable functions, this
patchset is adding extra check that the function is one of the ftrace's
functions.
All ftrace functions addresses are stored in vmlinux binary within
symbols:
__start_mcount_loc
__stop_mcount_loc
During object preparation code we read those addresses, sort them and
use them as filter for all detected dwarf functions.
We also filter out functions within .init section, ftrace is doing that
in runtime. At the same time we keep functions from
.init.bpf.preserve_type, because they are needed in BTF.
I can still see several differences to ftrace functions in
/sys/kernel/debug/tracing/available_filter_functions file:
- available_filter_functions includes modules
- available_filter_functions includes functions like:
__acpi_match_device.part.0.constprop.0
acpi_ns_check_sorted_list.constprop.0
acpi_os_unmap_generic_address.part.0
acpiphp_check_bridge.part.0
which are not part of dwarf data
- BTF includes multiple functions like:
__clk_register_clkdev
clk_register_clkdev
which share same code so they appear just as single function
in available_filter_functions, but dwarf keeps track of both
of them
- BTF includes iterator functions, which do not make it to
available_filter_functions
With this change I'm getting 38384 BTF functions, which when added above
functions to consideration gives same amount of functions in
available_filter_functions.
The patch still keeps the original function filter condition (that uses
current fn->declaration check) in case the object does not contain
*_mcount_loc symbol -> object is not vmlinux.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Frank Ch. Eigler <fche@redhat.com>
Cc: Hao Luo <haoluo@google.com>
Cc: Mark Wieelard <mjw@redhat.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Move find_all_percpu_vars() under generic collect_symbols() that walks
over symbols and calls collect_percpu_var().
We will add another collect function that needs to go through all the
symbols, so it's better we go through them just once.
There's no functional change intended.
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Hao Luo <haoluo@google.com>
Cc: Alexei Starovoitov <ast@kernel.org>
Cc: Frank Ch. Eigler <fche@redhat.com>
Cc: Mark Wieelard <mjw@redhat.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Commit 2e719cca66 ("btf_encoder: revamp how per-CPU variables are
encoded") adds percpu_var_exists() to filter out the symbols that are
not percpu var. However, the check comes after checking the var's type.
There can be symbols that are of zero type. If we hit that, btf_encoder
will not work without '--btf_encode_force'. So we should check
percpu_var_exists before checking var's type.
Tested:
haoluo@haoluo:~/kernel/tip$ gcc --version
gcc (GCC) 10.2.0
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Before:
haoluo@haoluo:~/kernel/tip$ make clean -s
haoluo@haoluo:~/kernel/tip$ make -j 32 -s
LINK resolve_btfids
error: found variable in CU 'kernel/bpf/btf.c' that has void type
Encountered error while encoding BTF.
FAILED: load BTF from vmlinux: Unknown error -2make: *** [Makefile:1164: vmlinux] Error 255
After:
haoluo@haoluo:~/kernel/tip$ make clean -s
haoluo@haoluo:~/kernel/tip$ make -j 32 -s
LINK resolve_btfids
haoluo@haoluo:~/kernel/tip$
Fixes: 2e719cc ("btf_encoder: revamp how per-CPU variables are encoded")
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Reported-by: Jiri Olsa <jolsa@kernel.org>
Signed-off-by: Hao Luo <haoluo@google.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Tested-by: Jiri Olsa <jolsa@redhat.com>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that libbpf is used to implement deduplicated strings container, all
of the binaries will need linux/btf.h header to compile properly. libbpf
is distributed with its own copies of Linux UAPI headers, so use them
during compilation.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Arnaldo Carvalho de Melo
Giuliano Procida
Jiri Olsa
Luca Boccassi
Andrii Nakryiko
Vitaly Chikunov
Hao Luo