This patch records for each CU whether it's in little-endian or
big-endian data format. This flag will be used in subsequent commits to
adjust bit offsets where necessary, to make them uniform across
endianness. This patch doesn't have any effect on pahole's output.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Mark Wielaard <mark@klomp.org>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Bitfield offsets can be negative, if field "borrows" few bits from
following aligned field. This causes a bunch of surprises down the line
in pahole's logic (e.g., for hole calculation logic), so instead of
waiting till printf routines adjust this for display, adjust them early
and keep less surprising semantics.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
the CTF and BTF loaders come already with the id to use, while the DWARF
loader comes with a Dwarf_Off that needs to be converted into the
ptr_table index.
So keep the cu__add_tag(cu, tag, &id) method to ask ask for the index to
be allocated in the ptr_table and the result to come back via the 'id'
parameter, now a uint32_t and introduce a cu__add_tag_with_id(cu, tag, id)
method to indicate that the 'uint32_t id' is the one to use.
With this we can use a uint32_t for the id both on 32-bit and 64-bit
arches.
Reported-by: Andrii Nakryiko <andriin@fb.com>
Tested-by:Andrii Nakryiko <andrii.nakryiko@gmail.com>
Link: https://lore.kernel.org/bpf/CAEf4Bzb0SpvXdDKMMnUof==kp4Y0AP54bKFjeCzX_AsmDm7k7g@mail.gmail.com/
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Existing code base assumes that single CU doesn't have more than 65535
types per each CU, which might be a reasonable assumption for DWARF
data. With BTF, though, all we get is single, potentially huge, CU which
can easily have more than 65k types. For example, this is the case for
allyesconfig version of Linux kernel, which has >200k types.
Due to this assumption, libdwarves and other parts of pahole are using
16-bit counters to iterate over entities within CU. This can cause
infinite loops when iterating BTF data, if there are more than 65535
types. This patch changes non-public variables to use 32-bit integers,
where appropriate.
This still leads to invalid reported data when using BTF loader (due to using
(X & 0xFFFF) type ID, instead of X, when X > 65535) and loading huge files,
but at least it's not stuck in an infinite loop anymore.
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Reported-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
[ Removed non type ID conversions, for instance for the kind of tag, like in type->namespace.tag.tag, that can remain a uint16_t ]
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Existing code assumes alignment of any integer type, which breaks for
packed structs.
This patch fixes all the current discrepanies between dwarf and btf
loader, when compared using btfdiff.
It preserves bit_offset of non-bitfield members, while for bitfield ones
it re-calculates initial byte/bit offset using natural alignment of the
underlying integer type, which seems to be always the case for
bitfields.
I've tested this on toy examples for both x86-64 and arm targets, there
were no differences reported by btfdiff.
Testing on vmlinux on x86-64 shows only these discrepancies, which are
unrelated to bit offsets:
$ ./btfdiff /tmp/vmlinux4
--- /tmp/btfdiff.dwarf.GIVfpr 2019-02-20 12:18:29.138788970 -0800
+++ /tmp/btfdiff.btf.c3x2KY 2019-02-20 12:18:29.351786365 -0800
@@ -16884,7 +16884,7 @@ struct pebs_record_nhm {
};
union hsw_tsx_tuning {
struct {
- unsigned int cycles_last_block:32; /* 0: 0 4 */
+ u32 cycles_last_block:32; /* 0: 0 4 */
u32 hle_abort:1; /* 4:31 4 */
u32 rtm_abort:1; /* 4:30 4 */
u32 instruction_abort:1; /* 4:29 4 */
@@ -26154,7 +26154,7 @@ struct acpi_device_power {
/* last cacheline: 40 bytes */
};
struct acpi_device_perf_flags {
- unsigned char reserved:8; /* 0: 0 1 */
+ u8 reserved:8; /* 0: 0 1 */
/* size: 1, cachelines: 1, members: 1 */
/* last cacheline: 1 bytes */
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: bpf@vger.kernel.org
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
That is the idiom for free its members and then free itself, 'free' is
just to free its members.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
So that we don't clash with libbpf's 'struct btf', in time more internal
state now in 'struct btf_elf' will refer to the equivalent internal
state in libbpf's 'struct btf', as they have lots in common.
Requested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Acked-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Now that libbpf is a submodule, we don't need to copy/paste btf.h header
with BTF type definitions.
This is a first step in migrating parts of libbtf, btf_encoder and
btf_loader to use libbpf and starting to use btf__dedup().
Signed-off-by: Andrii Nakryiko <andriin@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: Yonghong Song <yhs@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Because we don't have to lookup the member type that is encoded after
it, i.e. some member fields depend on first processing everything, then
lookup the types to get the sizes, etc.
But the:
member->byte_offset = member->bit_offset / 8;
Can be done soon after we have member->bit_offset loaded from the BTF
section.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I wrote the BTF loader from the CTF loader, where the size of
enumerations is expressed in bits, while BTF encodes it in bytes, fix
it.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
BTF_KIND_FUNC is generated by llvm (latest trunk, 8.0 or later).
Without BTF_KIND_FUNC support, we will see the following errors,
-bash-4.4$ cat t.c
struct t {
int a;
char b1:1;
char b2:3;
int c;
} g;
int main() { return 0; }
-bash-4.4$ clang -O2 -target bpf -g -c t.c -Xclang -target-feature -Xclang +dwarfris
-bash-4.4$ pahole -F btf t.o
BTF: idx: 3, off: 28, Unknown
struct t {
int a; /* 0 4 */
/* Bitfield combined with previous fields */
<ERROR(__class__fprintf:1342): 5 not found!>
/* Bitfield combined with previous fields */
<ERROR(__class__fprintf:1342): 5 not found!>
int c; /* 8 4 */
/* size: 12, cachelines: 1, members: 4 */
/* last cacheline: 12 bytes */
/* BRAIN FART ALERT! 12 != 8 + 0(holes), diff = 4 */
};
-bash-4.4$
The reason is that llvm generates BTF_KIND_FUNC which btf_loader does not
recognize.
This patch added support for BTF_KIND_FUNC. Since BTF_KIND_FUNC represents
a defined subprogram and not a real type. A null type is used to
represent BTF_KIND_FUNC to avoid skipping type index.
With this fix:
-bash-4.4$ pahole -F btf t.o
struct t {
int a; /* 0 4 */
char b1:1; /* 4: 0 1 */
char b2:3; /* 4: 1 1 */
int c; /* 8 4 */
/* size: 12, cachelines: 1, members: 4 */
/* last cacheline: 12 bytes */
/* BRAIN FART ALERT! 12 != 9 + 0(holes), diff = 3 */
};
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Commit 2a82d593be ("btf: Add kind_flag support for btf_loader") added
kind_flag supported in btf_loader to get correct bitfield_size and
bit_offset for struct/union members. The commit unnecessarily stored
the bit in the structure type which is not used later.
So let us remove the kind_flag from the struct type and any other
changes whose purpose is to set this bit.
Signed-off-by: Yonghong Song <yhs@fb.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Fixes: 2a82d593be ("btf: Add kind_flag support for btf_loader")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Yonghong explains:
<quote>
The bitfield offset in BTF starts from lower number to bigger one. That
is, it is always following the big endian convention, bitfield_offset
will be smaller if close to the top of structure.
This is different from what dwarf is doing, which will show different
things on little endian vs. big endian.
You can make simple adjustment based on all available info. In
btf_encoder.s, we did similar adjustment for little endian from
dwarf to btf.
</>
So fix it up while loading, so that the rebuilt C output shows the same
thing from BTF and from DWARF.
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
For struct/union members, the struct/union type info kind_flag is needed
to calculate correct bitfield_size and bit_offset.
if (kind_flag) {
bitfield_size = BTF_MEMBER_BITFIELD_SIZE(member->offset);
bit_offset = BTF_MEMBER_BIT_OFFSET(member->offset);
} else {
bitfield_size = 0;
bit_offset = member->offset;
}
Note that bitfield_size and bit_offset will not depend on the member
type. The member type will help calculate correct bitfield_offset,
byte_size, byte_offset, bit_size.
For example, with the fix, we will be able to display
bit offset and bitfield size properly.
-bash-4.4$ cat t.c
struct t {
int a:2;
int b:3;
int c:2;
} g;
-bash-4.4$ gcc -c -O2 -g t.c
-bash-4.4$ pahole -JV t.o
File t.o:
[1] STRUCT t kind_flag=1 size=4 vlen=3
a type_id=2 bitfield_size=2 bits_offset=0
b type_id=2 bitfield_size=3 bits_offset=2
c type_id=2 bitfield_size=2 bits_offset=5
[2] INT int size=4 bit_offset=0 nr_bits=32 encoding=SIGNED
-bash-4.4$ pahole -F btf t.o
struct t {
int a:2; /* 0: 0 4 */
int b:3; /* 0: 2 4 */
int c:2; /* 0: 5 4 */
/* size: 4, cachelines: 1, members: 3 */
/* bit_padding: 25 bits */
/* last cacheline: 4 bytes */
};
Note that the above offset showing is different from the below dwarf as
BTF bitfield_offset is always the offset from the start of structure,
kindly like big endian encoding. This may need adjustment to be
conforming to the dwarf dump format.
-bash-4.4$ pahole -F dwarf t.o
struct t {
int a:2; /* 0:30 4 */
int b:3; /* 0:27 4 */
int c:2; /* 0:25 4 */
/* size: 4, cachelines: 1, members: 3 */
/* bit_padding: 25 bits */
/* last cacheline: 4 bytes */
};
Signed-off-by: Yonghong Song <yhs@fb.com>
Tested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Alexei Starovoitov <ast@fb.com>
Cc: Martin KaFai Lau <kafai@fb.com>
Cc: dwarves@vger.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Andrii Nakryiko
Arnaldo Carvalho de Melo
Yonghong Song