dwarves: Prep v1.19

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2020-11-20 11:22:07 -03:00 · 2020-11-20 11:22:07 -03:00 · dd15aa4b0a
commit dd15aa4b0a
parent bf21da4075
5 changed files with 274 additions and 14 deletions
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@ -33,9 +33,9 @@ set(CMAKE_C_FLAGS_DEBUG "-Wall -Werror -ggdb -O0")
 set(CMAKE_C_FLAGS_RELEASE "-Wall -O2")

 # Just for grepping, DWARVES_VERSION isn't used anywhere anymore
-# add_definitions(-D_GNU_SOURCE -DDWARVES_VERSION="v1.18")
+# add_definitions(-D_GNU_SOURCE -DDWARVES_VERSION="v1.19")
 add_definitions(-D_GNU_SOURCE -DDWARVES_MAJOR_VERSION=1)
-add_definitions(-D_GNU_SOURCE -DDWARVES_MINOR_VERSION=18)
+add_definitions(-D_GNU_SOURCE -DDWARVES_MINOR_VERSION=19)
 find_package(DWARF REQUIRED)
 find_package(ZLIB REQUIRED)

--- a/1
+++ b/1
@ -49,6 +49,7 @@ changes-v1.13
 changes-v1.16
 changes-v1.17
 changes-v1.18
+changes-v1.19
 COPYING
 NEWS
 README
--- a/51
+++ b/51
@ -1,3 +1,54 @@
+v1.19:
+
+Fri Nov 20 2020
+
+bf21da407593f104 fprintf: Make typedef__fprintf print anonymous enums
+9c4bdf9331bf06a7 fprintf: Align enumerators
+89cf28228a8ab55e fprintf: Add enumeration__max_entry_name_len()
+932b84eb45a9b8ba fprintf: Make typedef__fprintf print anonymous structs
+4a1479305b53933d pahole: Add heuristic to auto-add --btf_base for /sys/kernel/btf/ prefixed files
+e1d01045828a5c4c btf: Fallback to raw BTF mode if the header magic matches
+24cea890abedb15b pahole: Force '-F btf' with --btf_base
+cfad738682506ce4 libbtf: Assume its raw_btf if filename starts with "/sys/kernel/btf/"
+7293c7fceac36844 pahole: The --btf_base option receives a PATH, not a SIZE
+b3dd4f3c3d5ea59e btf_encoder: Use better fallback message
+d06048c53094d9d2 btf_encoder: Move btf_elf__verbose/btf_elf__force setup
+8156bec8aedb685b btf_encoder: Fix function generation
+d0cd007339ee509e btf_encoder: Generate also .init functions
+25753e0396abea25 pfunct: Use load stealer to speed up --class
+aa8fb8c091446467 man-pages: Add entry for -J/--btf_encode to pahole's man page
+ace05ba9414c1fe4 btf: Add support for split BTF loading and encoding
+7290d08b4a6e5876 libbpf: Update libbpf submodule reference to latest master
+344f2483cfcd4952 libbtf: Improve variable naming and error reporting when writing out BTF
+94a7535939f92e91 btf_encoder: Fix array index type numbering
+9fa3a100f71e7a13 pfunct: Use a load stealer to stop as soon as a function is found
+de18bd5fe3515358 pfunct: Try sole argument as a function name, just like pahole
+bc1afd458562f21e pahole: Introduce --numeric_version for use in scripts and Makefiles
+784c3dfbd63d5bcf dwarves: Switch from a string based version to major/minor numbers
+fc06ee1b6e9dc14a pahole: Check if the sole arg is a file, not considering it a type if so
+f47b3a2df3622204 dwarf_loader: Fix partial unit warning
+5a22c2de79fb9edf btf_encoder: Change functions check due to broken dwarf
+7b1af3f4844b36b9 btf_encoder: Move find_all_percpu_vars in generic collect_symbols
+863e6f0f2cc08592 btf_encoder: Check var type after checking var addr.
+5e7ab5b9e064a3eb btf_loader: Handle union forward declaration properly
+ec3f944102a71241 cmake: Make libbpf's Linux UAPI headers available to all binaries
+8cac1c54c83c346b btf_encoder: Ignore zero-sized ELF symbols
+040fd7f585c9b9fc btf_encoder: Support cross-compiled ELF binaries with different endianness
+29fce8dc8571c6af strings: use BTF's string APIs for strings management
+75f3520fedf6afdb strings: Rename strings.h to avoid clashing with /usr/include/strings.h
+bba7151e0fd2fb3a dwarf_loader: increase the size of lookup hash map
+2e719cca66721284 btf_encoder: revamp how per-CPU variables are encoded
+0258a47ef99500db btf_encoder: Discard CUs after BTF encoding
+3c913e18b2702a9e btf_encoder: Fix emitting __ARRAY_SIZE_TYPE__ as index range type
+48efa92933e81d28 btf_encoder: Use libbpf APIs to encode BTF type info
+5d863aa7ce539e86 btf_loader: Use libbpf to load BTF
+0a9b89910e711951 dwarves: Expose and maintain active debug info loader operations
+7bc2dd07d51cb5ee btf_encoder: detect BTF encoding errors and exit
+c35b7fa52cb8112a libbpf: Update to latest libbpf version
+ef4f971a9cf745fc dwarf_loader: Conditionally define DW_AT_alignment
+cc3f9dce3378280f pahole: Implement --packed
+08f49262f474370a man-pages: Fix 'coimbine' typo
+
 v1.18:

 Fri 02 Oct 2020
--- a/changes-v1.19
+++ b/changes-v1.19
@ -0,0 +1,134 @@
+v1.19:
+
+- Support split BTF, where a main BTF file, vmlinux, can be used to find types
+  and then a kernel module, for instance, can have just what is unique to it.
+
+  For instance, looking for a type in the main vmlinux BTF info:
+
+    $ pahole wmi_notify_handler
+    pahole: type 'wmi_notify_handler' not found
+    $
+
+  If we look at the 'wmi' module BTF info that is in:
+
+    $ ls -la /sys/kernel/btf/wmi
+    -r--r--r--. 1 root root 2866 Nov 18 13:35 /sys/kernel/btf/wmi
+    $
+
+    $ pahole /sys/kernel/btf/wmi -C wmi_notify_handler
+    typedef void (*wmi_notify_handler)(u32, void *);
+    $
+
+  '--btf_base=/sys/kernel/btf/vmlinux' was automatically added in this last
+  example, an option that was also introduced in this version where types used in
+  the wmi.ko module but present in vmlinux can be found so that there is no
+  duplicity of types.
+
+- Update libbpf to get the split BTF support and use some of its functions to
+  load BTF and speed up DWARF loading and BTF encoding.
+
+- Support cross-compiled ELF binaries with different endianness
+
+- Support showing typedefs for anonymous types, like structs, unions and enums,
+  see the "Align enumerators" entry below for an example, another:
+
+    $ pahole rwlock_t
+    typedef struct {
+            arch_rwlock_t              raw_lock;             /*     0     8 */
+
+            /* size: 8, cachelines: 1, members: 1 */
+            /* last cacheline: 8 bytes */
+    } rwlock_t;
+    $
+
+- Align enumerators:
+
+    $ pahole ZSTD_strategy
+    typedef enum {
+            ZSTD_fast    = 0,
+            ZSTD_dfast   = 1,
+            ZSTD_greedy  = 2,
+            ZSTD_lazy    = 3,
+            ZSTD_lazy2   = 4,
+            ZSTD_btlazy2 = 5,
+            ZSTD_btopt   = 6,
+            ZSTD_btopt2  = 7,
+            } ZSTD_strategy;
+    $
+
+- Workaround bugs in the generation of DWARF records for functions in some gcc
+  versions that were causing breakage in the encoding of BTF:
+
+   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=97060 "Missing DW_AT_declaration=1 in dwarf data"
+
+- Ignore zero-sized ELF symbols instead of erroring out.
+
+- Handle union forward declaration properly in the BTF loader.
+
+- Introduce --numeric_version for use in scripts and Makefiles:
+
+    $ pahole --version
+    v1.19
+    $ pahole --numeric_version
+    119
+    $
+
+  To avoid things like this in the kernel's scripts/link-vmlinux.sh:
+
+    pahole_ver=$(${PAHOLE} --version | sed -E 's/v([0-9]+)\.([0-9]+)/\1\2/')
+
+- Try sole pfunct argument as a function name, just like pahole with type names:
+
+    $ pfunct tcp_v4_rcv
+    int tcp_v4_rcv(struct sk_buff * skb);
+    $
+
+- Speed up pfunct using some of the load techniques used in pahole.
+
+- Discard CUs after BTF encoding as they're not used anymore, greatly reducing
+  memory usage and speeding up vmlinux BTF encoding.
+
+- Revamp how per-CPU variables are encoded in BTF.
+
+- Include BTF info for static functions.
+
+- Use BTF's string APIs for strings management, greatly improving performance
+  over the tsearch().
+
+- Increase size of DWARF lookup hash table, shaving off about 1 second out of
+  about 20 seconds total for Linux BTF dedup.
+
+- Stop BTF encoding when errors are found in some DWARF CU.
+
+- Implement --packed, to show just packed structures, for instance, here are
+  the top 5 packed data structures in the Linux kernel:
+
+  $ pahole --sizes --packed | sort -k2 -nr | head -5
+  e820_table	64004	0
+  boot_params	4096	0
+  efi_variable	2084	0
+  snd_soc_tplg_pcm	912	0
+  ntb_info_regs	800	0
+  $
+
+  And here is one of them:
+
+  $ pahole efi_variable
+  struct efi_variable {
+  	efi_char16_t               VariableName[512];    /*     0  1024 */
+  	/* --- cacheline 16 boundary (1024 bytes) --- */
+  	efi_guid_t                 VendorGuid;           /*  1024    16 */
+  	long unsigned int          DataSize;             /*  1040     8 */
+  	__u8                       Data[1024];           /*  1048  1024 */
+  	/* --- cacheline 32 boundary (2048 bytes) was 24 bytes ago --- */
+  	efi_status_t               Status;               /*  2072     8 */
+  	__u32                      Attributes;           /*  2080     4 */
+
+  	/* size: 2084, cachelines: 33, members: 6 */
+  	/* last cacheline: 36 bytes */
+  } __attribute__((__packed__));
+  $
+
+- Fix bug in distros such as OpenSUSE:15.2 where DW_AT_alignment isn't defined.
+
+Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
--- a/rpm/SPECS/dwarves.spec
+++ b/rpm/SPECS/dwarves.spec
@ -2,7 +2,7 @@
 %define libver 1

 Name: dwarves
-Version: 1.18
+Version: 1.19
 Release: 1%{?dist}
 License: GPLv2
 Summary: Debugging Information Manipulation Tools (pahole & friends)
@ -39,7 +39,7 @@ One example of pfunct usage is in the fullcircle tool, a shell that drivers
 pfunct to generate compileable code out of a .o file and then build it using
 gcc, with the same compiler flags, and then use codiff to make sure the
 original .o file and the new one generated from debug info produces the same
-debug info.pick
+debug info.

 Pahole also can be used to use all this type information to pretty print raw data
 according to command line directions.
@ -79,7 +79,7 @@ rm -Rf %{buildroot}
 %files
 %doc README.ctracer
 %doc README.btf
-%doc changes-v1.18
+%doc changes-v1.19
 %doc NEWS
 %{_bindir}/btfdiff
 %{_bindir}/codiff
@ -128,12 +128,21 @@ rm -Rf %{buildroot}
 %{_includedir}/dwarves/libctf.h
 %{_includedir}/dwarves/list.h
 %{_includedir}/dwarves/rbtree.h
-%{_includedir}/dwarves/strings.h
+%{_includedir}/dwarves/pahole_strings.h
 %{_libdir}/%{libname}.so
 %{_libdir}/%{libname}_emit.so
 %{_libdir}/%{libname}_reorganize.so

 %changelog
+* Fri Nov 20 2020 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.19-1
+- New release: 1.19
+- Split BTF
+- DWARF workarounds for DW_AT_declaration
+- Support cross-compiled ELF binaries with different endianness
+- Support showing typedefs for anonymous types
+- Speedups using libbpf algorithms
+- See changes-v1.19 for a complete and more detailed list of changes
+
 * Fri Oct 02 2020 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.18-1
 - New release: 1.18
 - Use debugging info to pretty print raw data
@ -141,6 +150,17 @@ rm -Rf %{buildroot}
 - Fixes to address segfaults on the gdb testsuite binaries
 - Bail out on partial units for now, avoiding segfaults and providing warning to user.

+* Mon Aug 31 2020 - Zamir SUN <sztsian@gmail.com> - 1.17-4
+- Fix FTBFS
+- Resolves: bug 1863459
+
+* Sat Aug 01 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.17-3
+- Second attempt - Rebuilt for
+  https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
+
+* Mon Jul 27 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.17-2
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_33_Mass_Rebuild
+
 * Fri Mar 13 2020 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.17-1
 - New release: 1.17
 - Support raw BTF as available in /sys/kernel/btf/vmlinux.
@ -155,7 +175,7 @@ rm -Rf %{buildroot}
 - Introduce --unions to consider just unions:
 - Fix -m/--nr_methods - Number of functions operating on a type pointer

-* Mon Dec 16 2019 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.16-1
+* Wed Feb 12 2020 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.16-1
 - New release: 1.16
 - BTF encoder: Preserve and encode exported functions as BTF_KIND_FUNC.
 - BTF loader: Add support for BTF_KIND_FUNC
@ -169,10 +189,16 @@ rm -Rf %{buildroot}
 - pglobal: Allow passing the format path specifier, to use with BTF
 - Tree wide: Fixup issues pointed out by various coverity reports.

-* Tue Nov 05 2019 Jiri Olsa <jolsa@redhat.com> - 1.15-2
- Add libdwarves version dependency to dwarves package
+* Tue Jan 28 2020 Fedora Release Engineering <releng@fedoraproject.org> - 1.15-4
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_32_Mass_Rebuild

-* Fri May 27 2019 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.15-1
+* Wed Jul 24 2019 Fedora Release Engineering <releng@fedoraproject.org> - 1.15-3
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_31_Mass_Rebuild
+
+* Mon Jul  1 2019 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.15-2
+- Fix bug when processing classes without members
+
+* Thu Jun 27 2019 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.15-1
 - New release: 1.15
 - Fix --expand_types/-E segfault
 - Fixup endless printing named structs inside structs in --expand_types
@ -188,22 +214,70 @@ rm -Rf %{buildroot}
 - Support unions as arguments to -C/--class
 - New 'pfunct --compile' generates compilable output with type definitions

-* Thu Aug 16 2018 Arnaldo Carvalho de Melo <acme@kernel.org> - 1.12-1
- New release:
+* Thu Jan 31 2019 Fedora Release Engineering <releng@fedoraproject.org> - 1.12-2
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_30_Mass_Rebuild
+
+* Fri Aug 31 2018 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.12-1
+- New release: 1.12
 - union member cacheline boundaries for all inner structs
 - print union member offsets
 - Document 'pahole --hex'
 - Encode BTF type format for use with eBPF

-* Wed Jun 28 2017 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.11-1
- New release
+* Thu Jul 12 2018 Fedora Release Engineering <releng@fedoraproject.org> - 1.10-15
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_29_Mass_Rebuild
+
+* Wed Feb 07 2018 Fedora Release Engineering <releng@fedoraproject.org> - 1.10-14
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_28_Mass_Rebuild
+
+* Wed Aug 02 2017 Fedora Release Engineering <releng@fedoraproject.org> - 1.10-13
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_27_Binutils_Mass_Rebuild
+
+* Wed Jul 26 2017 Fedora Release Engineering <releng@fedoraproject.org> - 1.10-12
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_27_Mass_Rebuild
+
+* Fri Feb 10 2017 Fedora Release Engineering <releng@fedoraproject.org> - 1.10-11
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_26_Mass_Rebuild
+
+* Wed Aug 03 2016 Cole Robinson <crobinso@redhat.com> - 1.10-9%{?dist}
+- pdwtags: don't fail on unhandled tags (bz 1348200)
+
+* Wed Feb 03 2016 Fedora Release Engineering <releng@fedoraproject.org> - 1.10-9
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_24_Mass_Rebuild
+
+* Wed Jun 17 2015 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.10-8
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_23_Mass_Rebuild
+
+* Fri Sep 05 2014 Marcin Juszkiewicz <mjuszkiewicz@redhat.com> - 1.10-7
+- backport removal of DW_TAG_mutable_type
+
+* Sat Aug 16 2014 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.10-6
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_21_22_Mass_Rebuild
+
+* Sat Jun 07 2014 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.10-5
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_21_Mass_Rebuild
+
+* Sat Aug 03 2013 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.10-4
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_20_Mass_Rebuild
+
+* Wed Feb 13 2013 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.10-3
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_19_Mass_Rebuild
+
+* Wed Jul 18 2012 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.10-2
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_18_Mass_Rebuild

 * Wed May 30 2012 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.10-1
 - New release

+* Fri Jan 13 2012 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.9-2
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_17_Mass_Rebuild
+
 * Sat Nov 20 2010 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.9-1
 - New release

+* Tue Feb 08 2011 Fedora Release Engineering <rel-eng@lists.fedoraproject.org> - 1.8-2
+- Rebuilt for https://fedoraproject.org/wiki/Fedora_15_Mass_Rebuild
+
 * Fri Dec  4 2009 Arnaldo Carvalho de Melo <acme@redhat.com> - 1.8-1
 - New release