f3d9054ba8
On SMP systems, the global percpu variables are placed in a special
'.data..percpu' section, which is stored in a segment whose initial
address is set to 0, the addresses of per-CPU variables are relative
positive addresses [1].
This patch extracts these variables from vmlinux and places them with
their type information in BTF. More specifically, when BTF is encoded,
we find the index of the '.data..percpu' section and then traverse the
symbol table to find those global objects which are in this section.
For each of these objects, we push a BTF_KIND_VAR into the types buffer,
and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the
CUs have finished processing, we push a BTF_KIND_DATASEC into the
btfe->types buffer, followed by the percpu_secinfo's content.
In a v5.8-rc3 linux kernel, I was able to extract 288 such variables.
The build time overhead is small and the space overhead is also small.
See testings below.
A found variable can be invalid in two ways:
- Its name found in elf_sym__name is invalid.
- Its size identified by elf_sym__size is 0.
In either case, the BTF containing such symbols will be rejected by the
BTF verifier. Normally we should not see such symbols. But if one is
seen during BTF encoding, the encoder will exit with error. An new flag
'-j' (or '--force') is implemented to help testing, which skips the
invalid symbols and force emit a BTF.
Testing:
- vmlinux size has increased by ~12kb.
Before:
$ readelf -SW vmlinux | grep BTF
[25] .BTF PROGBITS ffffffff821a905c 13a905c 2d2bf8 00
After:
$ pahole -J vmlinux
$ readelf -SW vmlinux | grep BTF
[25] .BTF PROGBITS ffffffff821a905c 13a905c 2d5bca 00
- Common global percpu VARs and DATASEC are found in BTF section.
$ bpftool btf dump file vmlinux | grep runqueues
[14152] VAR 'runqueues' type_id=13778, linkage=global-alloc
$ bpftool btf dump file vmlinux | grep 'cpu_stopper'
[17582] STRUCT 'cpu_stopper' size=72 vlen=5
[17601] VAR 'cpu_stopper' type_id=17582, linkage=static
$ bpftool btf dump file vmlinux | grep ' DATASEC '
[63652] DATASEC '.data..percpu' size=179288 vlen=288
- Tested bpf selftests.
- pahole exits with error if an invalid symbol is seen during encoding,
make -f Makefile -j 36 -s
PAHOLE: Error: Found symbol of zero size when encoding btf (sym: 'yyy', cu: 'xxx.c').
PAHOLE: Error: Use '-j' or '--force_emit' to ignore such symbols and force emit the btf.
scripts/link-vmlinux.sh: line 137: 2475712 Segmentation fault LLVM_OBJCOPY=${OBJCOPY} ${PAHOLE} -J ${1}
- With the flag '-j' or '--force', the invalid symbols are ignored.
- Further in verbose mode and with '-j' or '--force' set, a warning is generated:
PAHOLE: Warning: Found symbol of zero size when encoding btf, ignored (sym: 'yyy', cu: 'xxx.c').
PAHOLE: Warning: Found symbol of invalid name when encoding btf, ignored (sym: 'zzz', cu: 'sss.c').
References:
[1] https://lwn.net/Articles/531148/
Signed-off-by: Hao Luo <haoluo@google.com>
Tested-by: Andrii Nakryiko <andriin@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Acked-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Oleg Rombakh <olegrom@google.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
19 lines
359 B
C
19 lines
359 B
C
#ifndef _BTF_ENCODER_H_
|
|
#define _BTF_ENCODER_H_ 1
|
|
/*
|
|
SPDX-License-Identifier: GPL-2.0-only
|
|
|
|
Copyright (C) 2019 Facebook
|
|
|
|
Derived from ctf_encoder.h, which is:
|
|
Copyright (C) Arnaldo Carvalho de Melo <acme@redhat.com>
|
|
*/
|
|
|
|
struct cu;
|
|
|
|
int btf_encoder__encode();
|
|
|
|
int cu__encode_btf(struct cu *cu, int verbose, bool force);
|
|
|
|
#endif /* _BTF_ENCODER_H_ */
|