Commit Graph

1274 Commits

Author SHA1 Message Date
Arnaldo Carvalho de Melo
cf459ca16f fprintf: Pretty print struct members that are pointers to nameless structs
I.e. to structs defined inside other structs, without a name:

Before:

  $ pahole -C parsed_partitions /home/acme/git/build/v5.1-rc4+/block/partitions/check.o
  struct parsed_partitions {
  	struct block_device *      bdev;                 /*     0     8 */
  	char                       name[32];             /*     8    32 */
  	struct  *                  parts;                /*    40     8 */
  	int                        next;                 /*    48     4 */
  	int                        limit;                /*    52     4 */
  	bool                       access_beyond_eod;    /*    56     1 */

  	/* XXX 7 bytes hole, try to pack */

  	/* --- cacheline 1 boundary (64 bytes) --- */
  	char *                     pp_buf;               /*    64     8 */

  	/* size: 72, cachelines: 2, members: 7 */
  	/* sum members: 65, holes: 1, sum holes: 7 */
  	/* last cacheline: 8 bytes */
  };
  $

After:

  $ pahole -C parsed_partitions /home/acme/git/build/v5.1-rc4+/block/partitions/check.o
  struct parsed_partitions {
  	struct block_device * bdev;                      /*     0     8 */
  	char                       name[32];             /*     8    32 */
  	struct {
  		sector_t           from;                 /*    40     8 */
  		sector_t           size;                 /*    48     8 */
  		int                flags;                /*    56     4 */
  		bool               has_info;             /*    60     1 */
  		struct partition_meta_info info;         /*    61   101 */
  	} * parts; /*    40     8 */
  	int                        next;                 /*    48     4 */
  	int                        limit;                /*    52     4 */
  	bool                       access_beyond_eod;    /*    56     1 */

  	/* XXX 7 bytes hole, try to pack */

  	/* --- cacheline 1 boundary (64 bytes) --- */
  	char *                     pp_buf;               /*    64     8 */

  	/* size: 72, cachelines: 2, members: 7 */
  	/* sum members: 65, holes: 1, sum holes: 7 */
  	/* last cacheline: 8 bytes */
  };
  $

Still need to align that offsets, leave this for later.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-16 14:33:48 -03:00
Arnaldo Carvalho de Melo
09ed2e78be pfunct: Emit definitions for pointers inside pointer to function args
When using --compile we were missing emitting the types for function
arguments in function pointers arguments, i.e.:

Before:

  /home/acme/git/build/v5.1-rc4+/kernel/events/core.o
  /tmp/fullcircle.CZeLch.c:3770:141: warning: ‘struct perf_output_handle’ declared inside parameter list will not be visible outside of this definition or declaration
   inline int __perf_event_output(struct perf_event * event, struct perf_sample_data * data, struct pt_regs * regs, int (*output_begin)(struct perf_output_handle *, struct perf_event *, unsigned int))
                                                                                                                                               ^~~~~~~~~~~~~~~~~~

After:

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/kernel/events/core.o > a.c
  $ gcc -g -c a.c
  $ grep -w perf_output_handle -m1 -A5 a.c
  struct perf_output_handle;

  inline int __perf_event_output(struct perf_event * event, struct perf_sample_data * data, struct pt_regs * regs, int (*output_begin)(struct perf_output_handle *, struct perf_event *, unsigned int))
  {
	  return 0;
  }
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-16 14:05:01 -03:00
Arnaldo Carvalho de Melo
e9fc2f6470 fullcircle: Check that we found the CFLAGS
So that we don't try to compile something that wasn't built with a
CFLAGS or even gcc, like .o made from ASM files.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-16 14:00:01 -03:00
Arnaldo Carvalho de Melo
05921c47f5 emit: Handle typedefs that are a pointer to typedefs
Before:

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/kernel/sched/autogroup.o > a.c
  $ gcc -g -c a.c
  a.c:1656:9: error: unknown type name ‘__signalfn_t’
   typedef __signalfn_t * __sighandler_t;
           ^~~~~~~~~~~~
  a.c:1658:9: error: unknown type name ‘__restorefn_t’
   typedef __restorefn_t * __sigrestore_t;
           ^~~~~~~~~~~~~
  $

After:

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/kernel/sched/autogroup.o > a.c
  $ gcc -g -c a.c
  $ egrep  '__(restore|signal)fn_t' -A10 a.c
  typedef void (__signalfn_t)(int);
  typedef __signalfn_t * __sighandler_t;

  typedef void (__restorefn_t)(void);
  typedef __restorefn_t * __sigrestore_t;

  struct sigaction {
	  __sighandler_t             sa_handler;           /*     0     8 */
	  long unsigned int          sa_flags;             /*     8     8 */
	  __sigrestore_t             sa_restorer;          /*    16     8 */
	  sigset_t                   sa_mask;              /*    24     8 */

	  /* size: 32, cachelines: 1, members: 4 */
	  /* last cacheline: 32 bytes */
  };
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-16 13:49:29 -03:00
Arnaldo Carvalho de Melo
87af9839bf fullcircle: Try building from pfunct --compile and check types
With this and running like:

  $ find ~/git/build/v5.1-rc4+/fs/ -name "*.o" | grep -v .mod.o | \
	while read obj ; do
		echo $obj ;
		fullcircle $obj ;
	done

The vast majority of the kernel single compilation unit objects get
the source code for its function prototypes and types used rebuilt,
recompiled and then the original DWARF can be compared with the one
generated from the regenerated C code.

More work needed, but should be a good start and has already helped to
fix several issues, a reported in the previous csets.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 20:54:03 -03:00
Arnaldo Carvalho de Melo
c7fd9cc1fe pfunct: Fixup more return types
Covering typedefs, unions.

Fixes: 99750f244c ("pfunct: Generate a valid return type for the --compile bodies")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 20:53:50 -03:00
Arnaldo Carvalho de Melo
56c50b8dbe emit: Make find_fwd_decl a bit more robust
We need to dig deeper in some emit cases, but for now lets make this
a bit more robust by checking if the types are unnamed.

Things like:

struct b {
	struct {
		int a;
	};
}

This is well handled in the main fprintf routines, but its interaction
with the emit routines need more work, lets at least remove the segfault
and leave fixing this for good for v1.14.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 20:53:40 -03:00
Arnaldo Carvalho de Melo
2bcb01fc2f fprintf: Handle single zero sized array member structs
/home/acme/git/build/v5.1-rc4+/fs/proc/kcore.o
  /tmp/fullcircle.Vnd2oz.c:788:29: error: flexible array member in a struct with no named members
    char                       x[];                  /*     0     0 */

Original:

include/linux/mmzone.h, line 109:

  /*
   * zone->lock and the zone lru_lock are two of the hottest locks in the kernel.
   * So add a wild amount of padding here to ensure that they fall into separate
   * cachelines.  There are very few zone structures in the machine, so space
   * consumption is not a concern here.
   */
  #if defined(CONFIG_SMP)
  struct zone_padding {
          char x[0];
  } ____cacheline_internodealigned_in_smp;
  #define ZONE_PADDING(name)      struct zone_padding name;
  #else
  #define ZONE_PADDING(name)
  #endif

B0rken:

  struct zone_padding {
          char                       x[];                 /*     0     0 */

          /* size: 0, cachelines: 0, members: 1 */
  } __attribute__((__aligned__(64)));

Fixed:

  struct zone_padding {
          char                       x[0];                 /*     0     0 */

          /* size: 0, cachelines: 0, members: 1 */
  } __attribute__((__aligned__(64)));

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 17:20:11 -03:00
Arnaldo Carvalho de Melo
0987266cd9 fprintf: Deal with zero sized arrays in the midle of a union
gcc dislikes invalid C:

  /home/acme/git/build/v5.1-rc4+/fs/ceph/export.o
  /tmp/fullcircle.CvAfpM.c:591:22: error: flexible array member in union
     __u32              raw[];                /*     0     0 */
                        ^~~
Before:

  $ pahole -C fid /home/acme/git/build/v5.1-rc4+/fs/ceph/export.o
  struct fid {
  	union {
  		struct {
  			u32        ino;                  /*     0     4 */
  			u32        gen;                  /*     4     4 */
  			u32        parent_ino;           /*     8     4 */
  			u32        parent_gen;           /*    12     4 */
  		} i32;                                   /*     0    16 */
  		struct {
  			u32        block;                /*     0     4 */
  			u16        partref;              /*     4     2 */
  			u16        parent_partref;       /*     6     2 */
  			u32        generation;           /*     8     4 */
  			u32        parent_block;         /*    12     4 */
  			u32        parent_generation;    /*    16     4 */
  		} udf;                                   /*     0    20 */
  		__u32              raw[];                /*     0     0 */
  	};                                               /*     0    20 */

  	/* size: 20, cachelines: 1, members: 1 */
  	/* last cacheline: 20 bytes */
  };
  $

After:

  $ pahole -C fid /home/acme/git/build/v5.1-rc4+/fs/ceph/export.o
  struct fid {
  	union {
  		struct {
  			u32        ino;                  /*     0     4 */
  			u32        gen;                  /*     4     4 */
  			u32        parent_ino;           /*     8     4 */
  			u32        parent_gen;           /*    12     4 */
  		} i32;                                   /*     0    16 */
  		struct {
  			u32        block;                /*     0     4 */
  			u16        partref;              /*     4     2 */
  			u16        parent_partref;       /*     6     2 */
  			u32        generation;           /*     8     4 */
  			u32        parent_block;         /*    12     4 */
  			u32        parent_generation;    /*    16     4 */
  		} udf;                                   /*     0    20 */
  		__u32              raw[0];               /*     0     0 */
  	};                                               /*     0    20 */

  	/* size: 20, cachelines: 1, members: 1 */
  	/* last cacheline: 20 bytes */
  };
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 17:09:03 -03:00
Arnaldo Carvalho de Melo
1101337a74 fprintf: Deal with zero sized arrays in the middle of a struct
Consider:

  struct ipc64_perm {
          __kernel_key_t          key;
          __kernel_uid32_t        uid;
          __kernel_gid32_t        gid;
          __kernel_uid32_t        cuid;
          __kernel_gid32_t        cgid;
          __kernel_mode_t         mode;
                                  /* pad if mode_t is u16: */
          unsigned char           __pad1[4 - sizeof(__kernel_mode_t)];
          unsigned short          seq;
          unsigned short          __pad2;
          __kernel_ulong_t        __unused1;
          __kernel_ulong_t        __unused2;
  };

That is a roundabout way of using __attribute__(__aligned__(4)), but
should work nonetheless.

We were not putting the [0] in that zero sized array which ended up
making gcc complain with:

  $ gcc -g -c shm.c
  shm.c:199:29: error: flexible array member not at end of struct
    unsigned char              __pad1[];            /*    24     0 */
                               ^~~~~~
  $

Now this works, i.e. generates compilable source code out of the
type tags, be it from BTF or from DWARF, i.e. this is all from the
internal representation of such types, agnostic wrt the original type
format.

So, the full circle:

  $ pahole -C ipc64_perm /home/acme/git/build/v5.1-rc4+/ipc/shm.o
  struct ipc64_perm {
  	__kernel_key_t             key;                  /*     0     4 */
  	__kernel_uid32_t           uid;                  /*     4     4 */
  	__kernel_gid32_t           gid;                  /*     8     4 */
  	__kernel_uid32_t           cuid;                 /*    12     4 */
  	__kernel_gid32_t           cgid;                 /*    16     4 */
  	__kernel_mode_t            mode;                 /*    20     4 */
  	unsigned char              __pad1[0];            /*    24     0 */
  	short unsigned int         seq;                  /*    24     2 */
  	short unsigned int         __pad2;               /*    26     2 */

  	/* XXX 4 bytes hole, try to pack */

  	__kernel_ulong_t           __unused1;            /*    32     8 */
  	__kernel_ulong_t           __unused2;            /*    40     8 */

  	/* size: 48, cachelines: 1, members: 11 */
  	/* sum members: 44, holes: 1, sum holes: 4 */
  	/* last cacheline: 48 bytes */
  };
  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/ipc/shm.o > shm.c
  $ gcc -g -c shm.c
  $ pahole -C ipc64_perm shm.o
  struct ipc64_perm {
  	__kernel_key_t             key;                  /*     0     4 */
  	__kernel_uid32_t           uid;                  /*     4     4 */
  	__kernel_gid32_t           gid;                  /*     8     4 */
  	__kernel_uid32_t           cuid;                 /*    12     4 */
  	__kernel_gid32_t           cgid;                 /*    16     4 */
  	__kernel_mode_t            mode;                 /*    20     4 */
  	unsigned char              __pad1[0];            /*    24     0 */
  	short unsigned int         seq;                  /*    24     2 */
  	short unsigned int         __pad2;               /*    26     2 */

  	/* XXX 4 bytes hole, try to pack */

  	__kernel_ulong_t           __unused1;            /*    32     8 */
  	__kernel_ulong_t           __unused2;            /*    40     8 */

  	/* size: 48, cachelines: 1, members: 11 */
  	/* sum members: 44, holes: 1, sum holes: 4 */
  	/* last cacheline: 48 bytes */
  };
  $

And for a chuckle, the original source code with a bit of history about
struct layout worries:

include/uapi/asm-generic/ipcbuf.h:

  /*
   * The generic ipc64_perm structure:
   * Note extra padding because this structure is passed back and forth
   * between kernel and user space.
   *
   * ipc64_perm was originally meant to be architecture specific, but
   * everyone just ended up making identical copies without specific
   * optimizations, so we may just as well all use the same one.
   *
   * Pad space is left for:
   * - 32-bit mode_t on architectures that only had 16 bit
   * - 32-bit seq
   * - 2 miscellaneous 32-bit values
   */

  struct ipc64_perm {
          __kernel_key_t          key;
          __kernel_uid32_t        uid;
          __kernel_gid32_t        gid;
          __kernel_uid32_t        cuid;
          __kernel_gid32_t        cgid;
          __kernel_mode_t         mode;
                                  /* pad if mode_t is u16: */
          unsigned char           __pad1[4 - sizeof(__kernel_mode_t)];
          unsigned short          seq;
          unsigned short          __pad2;
          __kernel_ulong_t        __unused1;
          __kernel_ulong_t        __unused2;
  };

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 16:54:15 -03:00
Arnaldo Carvalho de Melo
13aa13eb0c fprintf: Don't reuse 'type' in multiple scopes in the same function
Its confusing, so rename some variables.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 16:54:13 -03:00
Arnaldo Carvalho de Melo
c8fc6f5a7a core: Use unnatural alignment of struct embedded in another to infer __packed__
Since we don't have something like DW_AT_alignment for
__attribute__((__packed__)), we need to use whatever hints that are
there in the alignments to figure out if a naturally packed struct has
the __attribute__((packed)) in the original sources, because that is
needed to waiver its natural alignment requisites.

For instance,

  /* Used at: btrfs.c */
  /* <1e7b> /home/acme/git/pahole/btrfs.c:199 */
  struct btrfs_block_group_cache {
          struct btrfs_key   key;                          /*     0    17 */
          struct btrfs_block_group_item item;              /*    17    24 */

          /* XXX 7 bytes hole, try to pack */

          struct btrfs_fs_info *     fs_info;              /*    48     8 */
          struct inode *             inode;                /*    56     8 */

In the original source code, btrfs_block_group_item is marked
__packed__, and being so, even seemingly unnecessarily, makes it, when
embedded in another struct, like the above, forfeit its natural
alingment, that would be 8 bytes, and instead appear right at the 17th
byte offset...

  struct btrfs_block_group_item {
          __le64                     used;                 /*     0     8 */
          __le64                     chunk_objectid;       /*     8     8 */
          __le64                     flags;                /*    16     8 */

          /* size: 24, cachelines: 1, members: 3 */
          /* last cacheline: 24 bytes */
  } __attribute__((__packed__));

So we need to, seeing its use at a unnatural offset, go backwards to the
btrfs_block_group_item pahole internal data structure, 'struct type' and
mark is_packed field as 'true', despite it not looking like a packed
struct.

Same thing with:

  struct ieee80211_mcs_info {
          u8                         rx_mask[10];          /*     0    10 */
          __le16                     rx_highest;           /*    10     2 */
          u8                         tx_params;            /*    12     1 */
          u8                         reserved[3];          /*    13     3 */

          /* size: 16, cachelines: 1, members: 4 */
          /* last cacheline: 16 bytes */
  };

That is naturally aligned and as 16 bytes, a power of two, then when it appears at the end of:

  $ pahole -IC ieee80211_sta_ht_cap vht.o
  /* Used at: vht.c */
  /* <31ea> /home/acme/git/pahole/vht.c:1769 */
  struct ieee80211_sta_ht_cap {
  	u16                        cap;                  /*     0     2 */
  	bool                       ht_supported;         /*     2     1 */
  	u8                         ampdu_factor;         /*     3     1 */
  	u8                         ampdu_density;        /*     4     1 */

  	/* XXX 1 byte hole, try to pack */

  	struct ieee80211_mcs_info mcs;                   /*     6    16 */

  	/* size: 22, cachelines: 1, members: 5 */
  	/* sum members: 21, holes: 1, sum holes: 1 */
  	/* last cacheline: 22 bytes */
  };
  $

We get that one byte hole if ieee80211_mcs_info isn't marked __packed__, as soon as we mark it:

  $ pahole -IC ieee80211_sta_ht_cap vht.o
  /* Used at: vht.c */
  /* <31ea> /home/acme/git/pahole/vht.c:1769 */
  struct ieee80211_sta_ht_cap {
  	u16                        cap;                  /*     0     2 */
  	bool                       ht_supported;         /*     2     1 */
  	u8                         ampdu_factor;         /*     3     1 */
  	u8                         ampdu_density;        /*     4     1 */
  	struct ieee80211_mcs_info mcs;                   /*     5    16 */

  	/* size: 22, cachelines: 1, members: 5 */
  	/* padding: 1 */
  	/* last cacheline: 22 bytes */
  };
  [acme@quaco pahole]$

It works, so __packed__ in this case just says: trow away the natural
alignment, make it 1 in whatever container structs.

So, before emitting the types for some struct, we go back looking at
each of its members and checking for such unnatural offsets, marking the
types as __packed__. Now:

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/net/mac80211/vht.o | grep "^struct ieee80211_mcs_info" -A8
  struct ieee80211_mcs_info {
  	u8                         rx_mask[10];          /*     0    10 */
  	__le16                     rx_highest;           /*    10     2 */
  	u8                         tx_params;            /*    12     1 */
  	u8                         reserved[3];          /*    13     3 */

  	/* size: 16, cachelines: 1, members: 4 */
  	/* last cacheline: 16 bytes */
  } __attribute__((__packed__));
  $

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/fs/btrfs/free-space-tree.o | grep "^struct btrfs_block_group_item" -A7
  struct btrfs_block_group_item {
  	__le64                     used;                 /*     0     8 */
  	__le64                     chunk_objectid;       /*     8     8 */
  	__le64                     flags;                /*    16     8 */

  	/* size: 24, cachelines: 1, members: 3 */
  	/* last cacheline: 24 bytes */
  } __attribute__((__packed__));
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 15:51:16 -03:00
Arnaldo Carvalho de Melo
6f0f9a8815 fprintf: Fixup multi-dimensional zero sized arrays const handling
Before:

  $ pahole -C piix_map_db /home/acme/git/build/v5.1-rc4+/drivers/ata/ata_piix.o
  struct piix_map_db {
  	const u32                  mask;                 /*     0     4 */
  	const u16                  port_enable;          /*     4     2 */

  	/* XXX 2 bytes hole, try to pack */

  	const const int            map[][4];             /*     8     0 */

  	/* size: 8, cachelines: 1, members: 3 */
  	/* sum members: 6, holes: 1, sum holes: 2 */
  	/* last cacheline: 8 bytes */
  };
  $

After:

  $ pahole -C piix_map_db /home/acme/git/build/v5.1-rc4+/drivers/ata/ata_piix.o
  struct piix_map_db {
  	const u32                  mask;                 /*     0     4 */
  	const u16                  port_enable;          /*     4     2 */

  	/* XXX 2 bytes hole, try to pack */

  	const int                  map[][4];             /*     8     0 */

  	/* size: 8, cachelines: 1, members: 3 */
  	/* sum members: 6, holes: 1, sum holes: 2 */
  	/* last cacheline: 8 bytes */
  };
  $

The DWARF tag sequence:

 <2><17e50>: Abbrev Number: 12 (DW_TAG_member)
    <17e51>   DW_AT_name        : map
    <17e55>   DW_AT_decl_file   : 1
    <17e56>   DW_AT_decl_line   : 160
    <17e57>   DW_AT_decl_column : 12
    <17e58>   DW_AT_type        : <0x17e78>
    <17e5c>   DW_AT_data_member_location: 8

 <1><17e78>: Abbrev Number: 15 (DW_TAG_const_type)
    <17e79>   DW_AT_type        : <0x17e63>

 <1><17e63>: Abbrev Number: 11 (DW_TAG_array_type)
    <17e64>   DW_AT_type        : <0xd8>

 <1><d8>: Abbrev Number: 15 (DW_TAG_const_type)
    <d9>   DW_AT_type        : <0xd1>

 <1><d1>: Abbrev Number: 120 (DW_TAG_base_type)
    <d2>   DW_AT_byte_size   : 4
    <d3>   DW_AT_encoding    : 5        (signed)
    <d4>   DW_AT_name        : int

  const -> array -> const -> int

So just make the check be at least one dimension, if the number of
elements is zero, then drop the double const.

With this btfdiff for the allyesconfig ppc64 reference kernel we're
using is again clean.

  $ pahole -F btf --sizes examples/vmlinux-aarch64 | wc -l
  51023
  $

> 50K types with output from BTF and DWARF matching.

Fixes: ccd67bdb20 ("fprintf: Print "const" for class members more early, in type__fprintf()")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 15:38:23 -03:00
Arnaldo Carvalho de Melo
9a4d719304 fprintf: Allow suppressing the inferred __attribute__((__packed__))
We use things like DW_AT_alignment, so not all of those attributes are
inferred by formats like BTF that lack that info, allow suppressing the
output and make btfdiff ask for both DWARF and BTF output to have this
suppressed.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 15:01:53 -03:00
Arnaldo Carvalho de Melo
ec935ee422 fprintf: Allow suppressing the output of force paddings at the end of structs
Things like 'struct timex' in the linux kernel led to this output:

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 14:45:29 -03:00
Arnaldo Carvalho de Melo
8471736f3c core: Cope with zero sized types when looking for natural alignment
We'll use it in a division, so make it 1, solving cases like:

  $ pahole examples/tcp.o -C u64_stats_sync
  struct u64_stats_sync {

	  /* size: 0, cachelines: 0, members: 0 */
  };
  $

  $ pahole -C bpf_prog_stats examples/tcp.o
  struct bpf_prog_stats {
  	u64                        cnt;                  /*     0     8 */
  	u64                        nsecs;                /*     8     8 */
  	struct u64_stats_sync syncp;                     /*    16     0 */

  	/* size: 16, cachelines: 1, members: 3 */
  	/* last cacheline: 16 bytes */
  };
  $

That were sefaulting.

Fixes: dc3d441961 ("core: Improve the natural alignment calculation")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 14:45:29 -03:00
Arnaldo Carvalho de Melo
986a3b58a8 fprintf: Only add bitfield forced paddings when alignment info available
I.e. BTF doesn't have DW_AT_alignment, so avoid emitting the bitfield
for padding at the end:

  $ pahole -F btf -C rseq examples/tcp.o
  struct rseq {
	  __u32                      cpu_id_start;         /*     0     4 */
	  __u32                      cpu_id;               /*     4     4 */
  	union {
  		__u64              ptr64;                /*     8     8 */
  		__u64              ptr;                  /*     8     8 */
  	} rseq_cs;                                       /*     8     8 */
  	__u32                      flags;                /*    16     4 */

  	/* Force padding: */
  	__u32                      :32;
  	__u32                      :32;
  	__u32                      :32;

  	/* size: 32, cachelines: 1, members: 4 */
  	/* padding: 12 */
  	/* last cacheline: 32 bytes */
  };
  $

After:

  $ pahole -F btf -C rseq examples/tcp.o
  struct rseq {
	__u32                      cpu_id_start;         /*     0     4 */
	__u32                      cpu_id;               /*     4     4 */
	union {
		__u64              ptr64;                /*     8     8 */
		__u64              ptr;                  /*     8     8 */
	} rseq_cs;                                       /*     8     8 */
	__u32                      flags;                /*    16     4 */

	/* size: 32, cachelines: 1, members: 4 */
	/* padding: 12 */
	/* last cacheline: 32 bytes */
  };
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 14:45:29 -03:00
Arnaldo Carvalho de Melo
49c27bdd66 core: Allow the loaders to advertise features they have
For instance, DWARF has DW_AT_alignment, and some output features
require that, so let loaders advertise such things, next patch will use
this info.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 14:45:27 -03:00
Arnaldo Carvalho de Melo
dc6b9437a3 emit: Handle structs with DW_AT_alignment=1 meaning __packed__
In the following struct the ceph_entity_addr entries all appear marked with a
__attribute__((__aligned__(8)), which, for the first two members of this type,
'peer_addr' and 'peer_addr_for_me', don't cause the regenerated struct to
differ in layout from the original layout put in place by the compiler as per
the original source code.

But the third member of this type, 'actual_peer_addr' ends up in a different
offset, even in a different cacheline, here is how it looks like in the code generated
from the original source code, at offset 568.

          char                       in_banner[30];        /*   472    30 */
          struct ceph_msg_connect out_connect;             /*   502    33 */
          /* --- cacheline 8 boundary (512 bytes) was 23 bytes ago --- */
          struct ceph_msg_connect_reply in_reply;          /*   535    26 */
          struct ceph_entity_addr actual_peer_addr __attribute__((__aligned__(1))); /*   561   136 */
          /* --- cacheline 10 boundary (640 bytes) was 57 bytes ago --- */
          struct ceph_msg_header out_hdr;                  /*   697    53 */

          /* XXX 2 bytes hole, try to pack */

          /* --- cacheline 11 boundary (704 bytes) was 48 bytes ago --- */

And here is how it looks like when built from the regenerated source code, at
offset 568:

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o > ceph.c
  $ gcc -g -c  ceph.c
  $ pahole -C ceph_connection ceph.o | head -46
  struct ceph_connection {
          void *                     private;              /*     0     8 */
          const struct ceph_connection_operations  * ops;  /*     8     8 */
          struct ceph_messenger *    msgr;                 /*    16     8 */
          atomic_t                   sock_state;           /*    24     4 */

          /* XXX 4 bytes hole, try to pack */

          struct socket *            sock;                 /*    32     8 */
          struct ceph_entity_addr peer_addr __attribute__((__aligned__(8))); /*    40   136 */
          /* --- cacheline 2 boundary (128 bytes) was 48 bytes ago --- */
          struct ceph_entity_addr peer_addr_for_me __attribute__((__aligned__(8))); /*   176   136 */
          /* --- cacheline 4 boundary (256 bytes) was 56 bytes ago --- */
          long unsigned int          flags;                /*   312     8 */
          /* --- cacheline 5 boundary (320 bytes) --- */
          long unsigned int          state;                /*   320     8 */
          const char  *              error_msg;            /*   328     8 */
          struct ceph_entity_name peer_name;               /*   336     9 */

          /* XXX 7 bytes hole, try to pack */

          u64                        peer_features;        /*   352     8 */
          u32                        connect_seq;          /*   360     4 */
          u32                        peer_global_seq;      /*   364     4 */
          struct ceph_auth_handshake * auth;               /*   368     8 */
          int                        auth_retry;           /*   376     4 */

          /* XXX 4 bytes hole, try to pack */

          /* --- cacheline 6 boundary (384 bytes) --- */
          struct mutex       mutex;                        /*   384    32 */
          struct list_head   out_queue;                    /*   416    16 */
          struct list_head   out_sent;                     /*   432    16 */
          /* --- cacheline 7 boundary (448 bytes) --- */
          u64                        out_seq;              /*   448     8 */
          u64                        in_seq;               /*   456     8 */
          u64                        in_seq_acked;         /*   464     8 */
          char                       in_banner[30];        /*   472    30 */
          struct ceph_msg_connect out_connect;             /*   502    33 */
          /* --- cacheline 8 boundary (512 bytes) was 23 bytes ago --- */
          struct ceph_msg_connect_reply in_reply;          /*   535    26 */

          /* XXX 7 bytes hole, try to pack */

          struct ceph_entity_addr actual_peer_addr __attribute__((__aligned__(8))); /*   568   136 */
          /* --- cacheline 11 boundary (704 bytes) --- */
    $

That happens because 'struct ceph_entity_addr' has that __attribute__
((__aligned__(8)) in the regenerated source code, above, now look at how it
gets regenerated:

  $ pahole -C ceph_entity_addr ceph.o
  struct ceph_entity_addr {
          __le32                     type;                 /*     0     4 */
          __le32                     nonce;                /*     4     4 */
          struct __kernel_sockaddr_storage in_addr __attribute__((__aligned__(8))); /*     8   128 */

          /* size: 136, cachelines: 3, members: 3 */
          /* forced alignments: 1 */
          /* last cacheline: 8 bytes */
  } __attribute__((__aligned__(8)));
  $

While when looking at the original DWARF:

  $ pahole -C ceph_entity_addr /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o
  struct ceph_entity_addr {
          __le32                     type;                 /*     0     4 */
          __le32                     nonce;                /*     4     4 */
          struct __kernel_sockaddr_storage in_addr __attribute__((__aligned__(1))); /*     8   128 */

          /* size: 136, cachelines: 3, members: 3 */
          /* forced alignments: 1 */
          /* last cacheline: 8 bytes */
  } __attribute__((__aligned__(1)));
  $

The confusion may further come from the fact that 'struct __kernel_sockaddr_storage' has,
in the regenerated source code, the __attribute__((__aligned__8)))

  $ pahole -C __kernel_sockaddr_storage ceph.o
  struct __kernel_sockaddr_storage {
          __kernel_sa_family_t       ss_family;            /*     0     2 */
          char                       __data[126];          /*     2   126 */

          /* size: 128, cachelines: 2, members: 2 */
  } __attribute__((__aligned__(8)));
  $

Which is the same as in the original DWARF:

  $ pahole -C __kernel_sockaddr_storage /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o
  struct __kernel_sockaddr_storage {
          __kernel_sa_family_t       ss_family;            /*     0     2 */
          char                       __data[126];          /*     2   126 */

          /* size: 128, cachelines: 2, members: 2 */
  } __attribute__((__aligned__(8)));
  $

Looking at the original original source code for 'struct ceph_entity_addr'
helps here, as it reads:

  include/linux/ceph/msgr.h, line 63:

  /*
   * entity_addr -- network address
   */
  struct ceph_entity_addr {
          __le32 type;
          __le32 nonce;  /* unique id for process (e.g. pid) */
          struct sockaddr_storage in_addr;
  } __attribute__ ((packed));

So the original code has no __attribute__((__aligned__(1))), so, lets look at
what the compiler generates for 'struct ceph_entity_addr':

  $ readelf -wi /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o | grep ceph_entity_addr -A7
      <193a6>   DW_AT_name        : (indirect string, offset: 0x1586): ceph_entity_addr
      <193aa>   DW_AT_byte_size   : 136
      <193ab>   DW_AT_alignment   : 1
      <193ac>   DW_AT_decl_file   : 296
      <193ae>   DW_AT_decl_line   : 63
      <193af>   DW_AT_decl_column : 8
      <193b0>   DW_AT_sibling     : <0x193e0>
   <2><193b4>: Abbrev Number: 5 (DW_TAG_member)
  $

So the natural alignment for 'struct ceph_entity_addr' ends up being the
natural alignment for 'struct __kernel_sockaddr_storage', which is 8, but
since 'struct ceph_entity_addr' was marked in the original source code as __packed__,
the compiler added the DW_AT_alignment: 1 to override that.

The heuristic in pahole, so far, took that __attribute__((__aligned__(1)))
literally:

  $ pahole -C ceph_entity_addr /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o
  struct ceph_entity_addr {
          __le32                     type;                 /*     0     4 */
          __le32                     nonce;                /*     4     4 */
          struct __kernel_sockaddr_storage in_addr __attribute__((__aligned__(1))); /*     8   128 */

          /* size: 136, cachelines: 3, members: 3 */
          /* forced alignments: 1 */
          /* last cacheline: 8 bytes */
  } __attribute__((__aligned__(1)));
  $

which ends up making the regenerated source code (with the __aligned__((1))),
generate a different layout, the __aligned__((8)) in one of its members
overrode that __aligned__((1)).

Take this into account and when faced with a structure which natural alignment
is not one and that has a DW_AT_alignment:1 to mean it really is __packed__.

Doing that makes the regenerated source code match the original structure
layouts, i.e. after the patch we get:

  $ pahole -C ceph_entity_addr /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o
  struct ceph_entity_addr {
          __le32                     type;                 /*     0     4 */
          __le32                     nonce;                /*     4     4 */
          struct __kernel_sockaddr_storage in_addr __attribute__((__aligned__(1))); /*     8   128 */

          /* size: 136, cachelines: 3, members: 3 */
          /* forced alignments: 1 */
          /* last cacheline: 8 bytes */
  } __attribute__((__packed__));
  $

And that member in 'struct ceph_connection', in the original, continues to read:

  $ pahole -C ceph_connection /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o | grep -w actual_peer_addr -B4 -A6
          char                       in_banner[30];        /*   472    30 */
          struct ceph_msg_connect out_connect;             /*   502    33 */
          /* --- cacheline 8 boundary (512 bytes) was 23 bytes ago --- */
          struct ceph_msg_connect_reply in_reply;          /*   535    26 */
          struct ceph_entity_addr actual_peer_addr __attribute__((__aligned__(1))); /*   561   136 */
          /* --- cacheline 10 boundary (640 bytes) was 57 bytes ago --- */
          struct ceph_msg_header out_hdr;                  /*   697    53 */

          /* XXX 2 bytes hole, try to pack */

          /* --- cacheline 11 boundary (704 bytes) was 48 bytes ago --- */
  $

While in the regenerated DWARF from the regenerated source code reads:

  $ pfunct --compile /home/acme/git/build/v5.1-rc4+/fs/ceph/super.o > ceph.c
  $ gcc -g -c  ceph.c
  $ pahole -C ceph_connection ceph.o | grep -w actual_peer_addr -B4 -A6
          char                       in_banner[30];        /*   472    30 */
          struct ceph_msg_connect out_connect;             /*   502    33 */
          /* --- cacheline 8 boundary (512 bytes) was 23 bytes ago --- */
          struct ceph_msg_connect_reply in_reply;          /*   535    26 */
          struct ceph_entity_addr actual_peer_addr __attribute__((__aligned__(1))); /*   561   136 */
          /* --- cacheline 10 boundary (640 bytes) was 57 bytes ago --- */
          struct ceph_msg_header out_hdr;                  /*   697    53 */

          /* XXX 2 bytes hole, try to pack */

          /* --- cacheline 11 boundary (704 bytes) was 48 bytes ago --- */
  $

I.e. it now matches.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-15 13:25:48 -03:00
Arnaldo Carvalho de Melo
f78633cfb9 core: Infer __packed__ for union struct members
I.e. check that all the structs that are embedded in a union have their natural
alignment satisfied by the size of the array they are contained in,

Before this change we ended up not marking union struct members that didn't had
natural alignment violations as __packed__ even tho they had to be to be in a
struct that didn't satisfied their natural alignment requirements, which would
violate them when said union was in an array, i.e. the second entry would have
the non __packed__ union struct member in a bad position.

E.g. Before:

  $ pahole -C ceph_osd_op /home/acme/git/build/v5.1-rc4+/net/ceph/osd_client.o
  struct ceph_osd_op {
          __le16                     op;                   /*     0     2 */
          __le32                     flags;                /*     2     4 */
          union {
                  struct {
                          __le64     offset;               /*     6     8 */
                          __le64     length;               /*    14     8 */
                          __le64     truncate_size;        /*    22     8 */
                          __le32     truncate_seq;         /*    30     4 */
                  } __attribute__((__packed__)) extent;    /*     6    28 */
                  struct {
                          __le32     name_len;             /*     6     4 */
                          __le32     value_len;            /*    10     4 */
                          __u8       cmp_op;               /*    14     1 */
                          __u8       cmp_mode;             /*    15     1 */
                  } __attribute__((__packed__)) xattr;     /*     6    10 */
                  struct {
                          __u8       class_len;            /*     6     1 */
                          __u8       method_len;           /*     7     1 */
                          __u8       argc;                 /*     8     1 */
                          __le32     indata_len;           /*     9     4 */
                  } __attribute__((__packed__)) cls;       /*     6     7 */
                  struct {
                          __le64     cookie;               /*     6     8 */
                          __le64     count;                /*    14     8 */
                  } pgls;                                  /*     6    16 */
                  struct {
                          __le64     snapid;               /*     6     8 */
                  } snap;                                  /*     6     8 */
                  struct {
                          __le64     cookie;               /*     6     8 */
                          __le64     ver;                  /*    14     8 */
                          __u8       op;                   /*    22     1 */
                          __le32     gen;                  /*    23     4 */
                  } __attribute__((__packed__)) watch;     /*     6    21 */
                  struct {
                          __le64     cookie;               /*     6     8 */
                  } notify;                                /*     6     8 */
                  struct {
                          __le64     offset;               /*     6     8 */
                          __le64     length;               /*    14     8 */
                          __le64     src_offset;           /*    22     8 */
                  } clonerange;                            /*     6    24 */
                  struct {
                          __le64     expected_object_size; /*     6     8 */
                          __le64     expected_write_size;  /*    14     8 */
                  } alloc_hint;                            /*     6    16 */
                  struct {
                          __le64     snapid;               /*     6     8 */
                          __le64     src_version;          /*    14     8 */
                          __u8       flags;                /*    22     1 */
                          __le32     src_fadvise_flags;    /*    23     4 */
                  } __attribute__((__packed__)) copy_from; /*     6    21 */
          };                                               /*     6    28 */
          __le32                     payload_len;          /*    34     4 */

          /* size: 38, cachelines: 1, members: 4 */
          /* last cacheline: 38 bytes */
  } __attribute__((__packed__));

After:

  $ pahole -C ceph_osd_op /home/acme/git/build/v5.1-rc4+/net/ceph/osd_client.o
  struct ceph_osd_op {
          __le16                     op;                   /*     0     2 */
          __le32                     flags;                /*     2     4 */
          union {
                  struct {
                          __le64     offset;               /*     6     8 */
                          __le64     length;               /*    14     8 */
                          __le64     truncate_size;        /*    22     8 */
                          __le32     truncate_seq;         /*    30     4 */
                  } __attribute__((__packed__)) extent;    /*     6    28 */
                  struct {
                          __le32     name_len;             /*     6     4 */
                          __le32     value_len;            /*    10     4 */
                          __u8       cmp_op;               /*    14     1 */
                          __u8       cmp_mode;             /*    15     1 */
                  } __attribute__((__packed__)) xattr;     /*     6    10 */
                  struct {
                          __u8       class_len;            /*     6     1 */
                          __u8       method_len;           /*     7     1 */
                          __u8       argc;                 /*     8     1 */
                          __le32     indata_len;           /*     9     4 */
                  } __attribute__((__packed__)) cls;       /*     6     7 */
                  struct {
                          __le64     cookie;               /*     6     8 */
                          __le64     count;                /*    14     8 */
                  } pgls;                                  /*     6    16 */
                  struct {
                          __le64     snapid;               /*     6     8 */
                  } snap;                                  /*     6     8 */
                  struct {
                          __le64     cookie;               /*     6     8 */
                          __le64     ver;                  /*    14     8 */
                          __u8       op;                   /*    22     1 */
                          __le32     gen;                  /*    23     4 */
                  } __attribute__((__packed__)) watch;     /*     6    21 */
                  struct {
                          __le64     cookie;               /*     6     8 */
                  } notify;                                /*     6     8 */
                  struct {
                          __le64     offset;               /*     6     8 */
                          __le64     length;               /*    14     8 */
                          __le64     src_offset;           /*    22     8 */
                  } clonerange;                            /*     6    24 */
                  struct {
                          __le64     expected_object_size; /*     6     8 */
                          __le64     expected_write_size;  /*    14     8 */
                  } alloc_hint;                            /*     6    16 */
                  struct {
                          __le64     snapid;               /*     6     8 */
                          __le64     src_version;          /*    14     8 */
                          __u8       flags;                /*    22     1 */
                          __le32     src_fadvise_flags;    /*    23     4 */
                  } __attribute__((__packed__)) copy_from; /*     6    21 */
          };                                               /*     6    28 */
          __le32                     payload_len;          /*    34     4 */

          /* size: 38, cachelines: 1, members: 4 */
          /* last cacheline: 38 bytes */
  } __attribute__((__packed__));
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 17:08:41 -03:00
Arnaldo Carvalho de Melo
75c52de9c6 core: Move packed_attribute_inferred from 'class' to 'type' class
Since we need to infer the attributes of union members too, so move
there.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 12:39:50 -03:00
Arnaldo Carvalho de Melo
1bb4527220 fprintf: Fixup const pointers
Before:

  $ pahole -C nft_ctx /home/acme/git/build/v5.1-rc4+/net/netfilter/nft_set_rbtree.o
  struct nft_ctx {
          struct net *               net;                  /*     0     8 */
          struct nft_table *         table;                /*     8     8 */
          struct nft_chain *         chain;                /*    16     8 */
          const const struct nlattr  *  * nla;             /*    24     8 */
          u32                        portid;               /*    32     4 */
          u32                        seq;                  /*    36     4 */
          u8                         family;               /*    40     1 */
          u8                         level;                /*    41     1 */
          bool                       report;               /*    42     1 */

          /* size: 48, cachelines: 1, members: 9 */
          /* padding: 5 */
          /* last cacheline: 48 bytes */
  };
  $

Original:

  struct nft_ctx {
          struct net                      *net;
          struct nft_table                *table;
          struct nft_chain                *chain;
          const struct nlattr * const     *nla;
          u32                             portid;
          u32                             seq;
          u8                              family;
          u8                              level;
          bool                            report;
  };

DWARF tags:

 <1><12c8a>: Abbrev Number: 12 (DW_TAG_structure_type)
    <12c8b>   DW_AT_name        : (indirect string, offset: 0xcc6f): nlattr
    <12c8f>   DW_AT_byte_size   : 4
    <12c93>   DW_AT_sibling     : <0x12cb2>

 <1><12cb2>: Abbrev Number: 17 (DW_TAG_const_type)
    <12cb3>   DW_AT_type        : <0x12c8a>

 <1><12cf9>: Abbrev Number: 4 (DW_TAG_pointer_type)
    <12cfa>   DW_AT_byte_size   : 8
    <12cfb>   DW_AT_type        : <0x12cb2>

 <1><12cff>: Abbrev Number: 17 (DW_TAG_const_type)
    <12d00>   DW_AT_type        : <0x12cf9>

 <1><1d54b>: Abbrev Number: 4 (DW_TAG_pointer_type)
    <1d54c>   DW_AT_byte_size   : 8
    <1d54d>   DW_AT_type        : <0x12cff>

 <2><1e52e>: Abbrev Number: 14 (DW_TAG_member)
    <1e52f>   DW_AT_name        : nla
    <1e536>   DW_AT_type        : <0x1d54b>
    <1e53a>   DW_AT_data_member_location: 24

Fixed now:

  $ pahole -C nft_ctx /home/acme/git/build/v5.1-rc4+/net/netfilter/nft_set_rbtree.o
  struct nft_ctx {
          struct net *               net;                  /*     0     8 */
          struct nft_table *         table;                /*     8     8 */
          struct nft_chain *         chain;                /*    16     8 */
          const struct nlattr  * const * nla;              /*    24     8 */
          u32                        portid;               /*    32     4 */
          u32                        seq;                  /*    36     4 */
          u8                         family;               /*    40     1 */
          u8                         level;                /*    41     1 */
          bool                       report;               /*    42     1 */

          /* size: 48, cachelines: 1, members: 9 */
          /* padding: 5 */
          /* last cacheline: 48 bytes */
  };
  $

So, one more full circled:

  $ fullcircle /home/acme/git/build/v5.1-rc4+/net/netfilter/nft_set_rbtree.o
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-12 12:07:46 -03:00
Arnaldo Carvalho de Melo
dc3d441961 core: Improve the natural alignment calculation
We need to take more than just arrays into account when figuring out the
natural alignment of struct members, looking recursively at types till
we get to basic types and pointers.

Before this patch the 'new' struct field in the 'v' union was considered
__packed__, when in fact it is not, as the natural alignment for the
'state_id' typedef is 4, so it can start at offset 36 (or 4 considering
just its container struct), see below:

  $ pahole -IC nfsd4_lock /home/acme/git/build/v5.1-rc4+/fs/nfsd/nfs4xdr.o
  /* Used at: /home/acme/git/linux/fs/nfsd/nfs4xdr.c */
  /* <1717a> /home/acme/git/linux/fs/nfsd/xdr4.h:156 */
  struct nfsd4_lock {
          u32                        lk_type;              /*     0     4 */
          u32                        lk_reclaim;           /*     4     4 */
          u64                        lk_offset;            /*     8     8 */
          u64                        lk_length;            /*    16     8 */
          u32                        lk_is_new;            /*    24     4 */

          /* XXX 4 bytes hole, try to pack */

          union {
                  struct {
                          u32        open_seqid;           /*    32     4 */
                          stateid_t  open_stateid;         /*    36    16 */
                          u32        lock_seqid;           /*    52     4 */
                          clientid_t clientid;             /*    56     8 */
                          /* --- cacheline 1 boundary (64 bytes) --- */
                          struct xdr_netobj owner;         /*    64    16 */
                  } __attribute__((__packed__)) new;       /*    32    48 */
                  struct {
                          stateid_t  lock_stateid;         /*    32    16 */
                          u32        lock_seqid;           /*    48     4 */
                  } __attribute__((__packed__)) old;       /*    32    20 */
          } v;                                             /*    32    48 */
          /* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
          union {
                  struct {
                          stateid_t  stateid;              /*    80    16 */
                  } ok;                                    /*    80    16 */
                  struct nfsd4_lock_denied denied;         /*    80    48 */
          } u;                                             /*    80    48 */

          /* size: 128, cachelines: 2, members: 7 */
          /* sum members: 124, holes: 1, sum holes: 4 */
  };
  $

Asking for -rEIC, i.e. relative offsets, expand types we can see that
stateid_t opaque type:

                struct {
                        /* typedef u32 -> __u32 */ unsigned int open_seqid;              /*     0     4 */
                        /* typedef stateid_t */ struct {
                                /* typedef u32 -> __u32 */ unsigned int si_generation;   /*     0     4 */
                                /* typedef stateid_opaque_t */ struct {
                                        /* typedef clientid_t */ struct {
                                                /* typedef u32 -> __u32 */ unsigned int   cl_boot; /*     0     4 */
                                                /* typedef u32 -> __u32 */ unsigned int   cl_id; /*     4     4 */
                                        } so_clid; /*     0     8 */
                                        /* typedef u32 -> __u32 */ unsigned int so_id;   /*     8     4 */
                                } si_opaque; /*     4    12 */
                        } open_stateid; /*     4    16 */

With the algorithm implemented in this patch we get it correctly as not
packed:

  $ pahole -IC nfsd4_lock /home/acme/git/build/v5.1-rc4+/fs/nfsd/nfs4xdr.o
  /* Used at: /home/acme/git/linux/fs/nfsd/nfs4xdr.c */
  /* <1717a> /home/acme/git/linux/fs/nfsd/xdr4.h:156 */
  struct nfsd4_lock {
          u32                        lk_type;              /*     0     4 */
          u32                        lk_reclaim;           /*     4     4 */
          u64                        lk_offset;            /*     8     8 */
          u64                        lk_length;            /*    16     8 */
          u32                        lk_is_new;            /*    24     4 */

          /* XXX 4 bytes hole, try to pack */

          union {
                  struct {
                          u32        open_seqid;           /*    32     4 */
                          stateid_t  open_stateid;         /*    36    16 */
                          u32        lock_seqid;           /*    52     4 */
                          clientid_t clientid;             /*    56     8 */
                          /* --- cacheline 1 boundary (64 bytes) --- */
                          struct xdr_netobj owner;         /*    64    16 */
                  } new;                                   /*    32    48 */
                  struct {
                          stateid_t  lock_stateid;         /*    32    16 */
                          u32        lock_seqid;           /*    48     4 */
                  } old;                                   /*    32    20 */
          } v;                                             /*    32    48 */
          /* --- cacheline 1 boundary (64 bytes) was 16 bytes ago --- */
          union {
                  struct {
                          stateid_t  stateid;              /*    80    16 */
                  } ok;                                    /*    80    16 */
                  struct nfsd4_lock_denied denied;         /*    80    48 */
          } u;                                             /*    80    48 */

          /* size: 128, cachelines: 2, members: 7 */
          /* sum members: 124, holes: 1, sum holes: 4 */
  };

Fixes: f2641ce169 ("core: Take arrays into account when inferring if a struct is packed")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-11 16:43:59 -03:00
Arnaldo Carvalho de Melo
ac32e5e908 codiff: Fix comparision of multi-cu against single-cu files
When the first arg, the old object file has multiple compile units, i.e.
multiple objects that were then linked into one, and the second just one
.o, or equivalent, i.e. a .BTF file, then codiff shouldn't try to
find the types in the single CU in each of the old CUs.

Think about a .BTF file generated from a multi-CU DWARF binary, it will
contain all the types in all of the DWARF CUs, so if we go on trying to
find all the BTF files in each of the CUs, we'll fail.

It only makes sense to go on the DWARF CUs looking for the type on the
.BTF section and then compare them.

Fixes: 6b1e43f2c1 ("codiff: When comparing against a file with just one CU don't bother finding by name")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-11 16:38:51 -03:00
Arnaldo Carvalho de Melo
f2641ce169 core: Take arrays into account when inferring if a struct is packed
Before:

  $ pahole -C qrwlock /home/acme/git/build/v5.1-rc4+/fs/ceph/dir.o
  struct qrwlock {
  	union {
  		atomic_t           cnts;                 /*     0     4 */
  		struct {
  			u8         wlocked;              /*     0     1 */
  			u8         __lstate[3];          /*     1     3 */
  		} __attribute__((__packed__));           /*     0     4 */
  	};                                               /*     0     4 */
  	arch_spinlock_t            wait_lock;            /*     4     4 */

  	/* size: 8, cachelines: 1, members: 2 */
  	/* last cacheline: 8 bytes */
  };

I.e. __lstate's class_member->byte_size is 3, causing the misinference that that
struct was packed, it is naturally aligned, we need to look at the size of the
array's entries to figure out its natural alignment:

After:

  $ pahole -C qrwlock /home/acme/git/build/v5.1-rc4+/fs/ceph/dir.o
  struct qrwlock {
  	union {
  		atomic_t           cnts;                 /*     0     4 */
  		struct {
  			u8         wlocked;              /*     0     1 */
  			u8         __lstate[3];          /*     1     3 */
  		};                                       /*     0     4 */
  	};                                               /*     0     4 */
  	arch_spinlock_t            wait_lock;            /*     4     4 */

  	/* size: 8, cachelines: 1, members: 2 */
  	/* last cacheline: 8 bytes */
  };
  $

To further test:

  $ cat packed_array_struct.c
  struct sarray {
  	short	  array[3];
  	long long first;
  } __attribute__((__packed__));

  void foo(struct sarray *s) {}
  $ gcc -g -c packed_array_struct.c
  $ pahole packed_array_struct.o
  struct sarray {
  	short int                  array[3];             /*     0     6 */
  	long long int              first;                /*     6     8 */

  	/* size: 14, cachelines: 1, members: 2 */
  	/* last cacheline: 14 bytes */
  } __attribute__((__packed__));
  $ cat packed_array_struct.c
  struct sarray {
  	short	  array[3];
  	long long first;
  };

  void foo(struct sarray *s) {}
  $ gcc -g -c packed_array_struct.c
  $ pahole packed_array_struct.o
  struct sarray {
  	short int                  array[3];             /*     0     6 */

  	/* XXX 2 bytes hole, try to pack */

  	long long int              first;                /*     8     8 */

  	/* size: 16, cachelines: 1, members: 2 */
  	/* sum members: 14, holes: 1, sum holes: 2 */
  	/* last cacheline: 16 bytes */
  };
  $

One more test:

  $ cat packed_array_struct.c
  struct sarray {
  	short     a;
  	short	  array[3];
  	long long b;
  };

  void foo(struct sarray *s) {}
  $

Before this patch:

  $ gcc -g -c packed_array_struct.c
  $ pahole packed_array_struct.o
  struct sarray {
  	short int                  a;                    /*     0     2 */
  	short int                  array[3];             /*     2     6 */
  	long long int              b;                    /*     8     8 */

  	/* size: 16, cachelines: 1, members: 3 */
  	/* last cacheline: 16 bytes */
  } __attribute__((__packed__));

After:

  $ pahole packed_array_struct.o
  struct sarray {
  	short int                  a;                    /*     0     2 */
  	short int                  array[3];             /*     2     6 */
  	long long int              b;                    /*     8     8 */

  	/* size: 16, cachelines: 1, members: 3 */
  	/* last cacheline: 16 bytes */
  };
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-11 13:25:45 -03:00
Arnaldo Carvalho de Melo
85c9936963 fprintf: Do not add explicit padding when struct has __aligned__ attr
Fixes: 13e5b9fc00 ("fprintf: Add unnamed bitfield padding at the end to rebuild original type")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-11 13:23:40 -03:00
Arnaldo Carvalho de Melo
b5e8fab596 emit: Cover void ** as a function parameter
Like in:

  static int
  cifs_setlease(struct file *file, long arg, struct file_lock **lease, void **priv)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-11 11:19:32 -03:00
Arnaldo Carvalho de Melo
28a3bc7add fprintf: Support packed enums
Check if the size is different than sizeof(int), which should be good
enough for now for both 64-bit and 32-bit targets.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-11 10:49:56 -03:00
Arnaldo Carvalho de Melo
f77a442f09 fprintf: Do not print the __aligned__ attribute if asked
I.e. honour conf_fprintf.suppress_aligned_attribute, noticed with
btfdiff, as BTF doesn't carries the alignment attribute, so can't
regenerate it, we need to suppress it so as to compare the output of
DWARF with that of the equivalent BTF.

Fixes: b42d77b0bb ("fprintf: Print __attribute__((__aligned__(N))) for structs/classes")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-10 18:34:16 -03:00
Arnaldo Carvalho de Melo
ea583dac52 fprintf: Print zero sized flat arrays as [], not [0]
To match the case when we really have just one dimension, so the
--flat-arrays should show for zero sized arrays, [], not [0]:

Noticed with btfdiff.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-10 18:25:37 -03:00
Arnaldo Carvalho de Melo
f909f13dd7 fprintf: Fixup handling of unnamed bitfields
We were only handling holes inside bitfields as a request to change the
byte_offset, which is not the case when instead of 'int foo:0;' we have
'int foo:6;' to ask for a explicit 6 bit hole inside a bitfield, like
in:

Before this patch:

  $ pahole -F btf -C kvm_mmu_page_role /home/acme/git/build/v5.1-rc4+/arch/x86/kvm/hyperv.o
  union kvm_mmu_page_role {
          u32                        word;               /*     0     4 */
          struct {
                  unsigned int       level:4;            /*     0: 0  4 */
                  unsigned int       gpte_is_8_bytes:1;  /*     0: 4  4 */
                  unsigned int       quadrant:2;         /*     0: 5  4 */
                  unsigned int       direct:1;           /*     0: 7  4 */
                  unsigned int       access:3;           /*     0: 8  4 */
                  unsigned int       invalid:1;          /*     0:11  4 */
                  unsigned int       nxe:1;              /*     0:12  4 */
                  unsigned int       cr0_wp:1;           /*     0:13  4 */
                  unsigned int       smep_andnot_wp:1;   /*     0:14  4 */
                  unsigned int       smap_andnot_wp:1;   /*     0:15  4 */
                  unsigned int       ad_disabled:1;      /*     0:16  4 */
                  unsigned int       guest_mode:1;       /*     0:17  4 */

                  /* XXX 6 bits hole, try to pack */

                  /* Force alignment to the next boundary: */
                  unsigned int       :0;

                  unsigned int       smm:8;              /*     0:24  4 */
          };                                             /*     0     4 */
  };
  $

After:

  $ pahole -F btf -C kvm_mmu_page_role /home/acme/git/build/v5.1-rc4+/arch/x86/kvm/hyperv.o
  union kvm_mmu_page_role {
          u32                        word;               /*     0     4 */
          struct {
                  unsigned int       level:4;            /*     0: 0  4 */
                  unsigned int       gpte_is_8_bytes:1;  /*     0: 4  4 */
                  unsigned int       quadrant:2;         /*     0: 5  4 */
                  unsigned int       direct:1;           /*     0: 7  4 */
                  unsigned int       access:3;           /*     0: 8  4 */
                  unsigned int       invalid:1;          /*     0:11  4 */
                  unsigned int       nxe:1;              /*     0:12  4 */
                  unsigned int       cr0_wp:1;           /*     0:13  4 */
                  unsigned int       smep_andnot_wp:1;   /*     0:14  4 */
                  unsigned int       smap_andnot_wp:1;   /*     0:15  4 */
                  unsigned int       ad_disabled:1;      /*     0:16  4 */
                  unsigned int       guest_mode:1;       /*     0:17  4 */

                  /* XXX 6 bits hole, try to pack */
                  unsigned int       :6;

                  unsigned int       smm:8;              /*     0:24  4 */
          };                                             /*     0     4 */
  };

Cc: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Yonghong Song <yhs@fb.com>
Fixes: a104eb1ea1 ("fprintf: Notice explicit bitfield alignment modifications")
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-10 18:11:55 -03:00
Arnaldo Carvalho de Melo
3247a777dc core: Infer if a struct is packed by the offsets/natural alignments
As DWARF (nor BTF) provides explicit attributes, we need to look at the
natural alignments, a byte is always alignted, etc.

This probably fails with things like __attribute__(__aligned(power-of-two)),
but with it most of the kernel data structures are full circled, i.e.
'pfunct --compile' regenerates source code from debug info that when
compiled generats debug info that end up matching the original sources.

  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	const char  *              uptr;
  	int                        refcnt;
  };

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	const char  *              uptr;                 /*     8     8 */
  	int                        refcnt;               /*    16     4 */

  	/* size: 24, cachelines: 1, members: 3 */
  	/* padding: 4 */
  	/* last cacheline: 24 bytes */
  };
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	const char  *              uptr;
  	int                        refcnt;
  } __packed;

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	const char  *              uptr;                 /*     8     8 */
  	int                        refcnt;               /*    16     4 */

  	/* size: 20, cachelines: 1, members: 3 */
  	/* last cacheline: 20 bytes */
  } __attribute__((__packed__));
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	int                        refcnt;
  	const char  *              uptr;
  };

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	int                        refcnt;               /*     8     4 */

  	/* XXX 4 bytes hole, try to pack */

  	const char  *              uptr;                 /*    16     8 */

  	/* size: 24, cachelines: 1, members: 3 */
  	/* sum members: 20, holes: 1, sum holes: 4 */
  	/* last cacheline: 24 bytes */
  };
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	int                        refcnt;
  	const char  *              uptr;
  } __packed;

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	int                        refcnt;               /*     8     4 */
  	const char  *              uptr;                 /*    12     8 */

  	/* size: 20, cachelines: 1, members: 3 */
  	/* last cacheline: 20 bytes */
  } __attribute__((__packed__));
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	const char  *              uptr;
  	unsigned char              refcnt;
  };

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	const char  *              uptr;                 /*     8     8 */
  	unsigned char              refcnt;               /*    16     1 */

  	/* size: 24, cachelines: 1, members: 3 */
  	/* padding: 7 */
  	/* last cacheline: 24 bytes */
  };
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	const char  *              uptr;
  	unsigned char              refcnt;
  } __packed;

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	const char  *              uptr;                 /*     8     8 */
  	unsigned char              refcnt;               /*    16     1 */

  	/* size: 17, cachelines: 1, members: 3 */
  	/* last cacheline: 17 bytes */
  } __attribute__((__packed__));
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	unsigned char              refcnt;
  	const char  *              uptr;
  };

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	unsigned char              refcnt;               /*     8     1 */

  	/* XXX 7 bytes hole, try to pack */

  	const char  *              uptr;                 /*    16     8 */

  	/* size: 24, cachelines: 1, members: 3 */
  	/* sum members: 17, holes: 1, sum holes: 7 */
  	/* last cacheline: 24 bytes */
  };
  $ cat a.c
  #define __packed __attribute__((__packed__))

  struct filename {
  	const char  *              name;
  	unsigned char              refcnt;
  	const char  *              uptr;
  } __packed;

  void m(struct filename *f) {}
  $ gcc -g -c a.c
  $ pahole a.o
  struct filename {
  	const char  *              name;                 /*     0     8 */
  	unsigned char              refcnt;               /*     8     1 */
  	const char  *              uptr;                 /*     9     8 */

  	/* size: 17, cachelines: 1, members: 3 */
  	/* last cacheline: 17 bytes */
  } __attribute__((__packed__));
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-10 17:41:59 -03:00
Arnaldo Carvalho de Melo
13e5b9fc00 fprintf: Add unnamed bitfield padding at the end to rebuild original type
Just like the 'struct timex' in the linux kernel UAPI, that is now
correctly reconstructed as:

  $ pahole -IC timex /home/acme/git/build/v5.0-rc2+/kernel/time/posix-clock.o | tail -32
          __kernel_long_t            ppsfreq;              /*    96     8 */
          __kernel_long_t            jitter;               /*   104     8 */
          int                        shift;                /*   112     4 */

          /* XXX 4 bytes hole, try to pack */

          __kernel_long_t            stabil;               /*   120     8 */
          /* --- cacheline 2 boundary (128 bytes) --- */
          __kernel_long_t            jitcnt;               /*   128     8 */
          __kernel_long_t            calcnt;               /*   136     8 */
          __kernel_long_t            errcnt;               /*   144     8 */
          __kernel_long_t            stbcnt;               /*   152     8 */
          int                        tai;                  /*   160     4 */

          /* Force padding: */
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;
          int                        :32;

          /* size: 208, cachelines: 4, members: 20 */
          /* sum members: 152, holes: 3, sum holes: 12 */
          /* padding: 44 */
          /* last cacheline: 16 bytes */
  };
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-10 11:15:42 -03:00
Arnaldo Carvalho de Melo
ccd67bdb20 fprintf: Print "const" for class members more early, in type__fprintf()
We want to reach array__fprintf() from here, with the class_member
name, as __tag__name() isn't handling arrays properly.

I.e. to print an array when we have its name we can't use __tag__name().

This also stops printing 0 for zero sized arrays and trows away the
extra DW_TAG_const_type that comes with zero sized arrays, where we
have:

   class_member type: DW_TAG_const_type 1
   DW_TAG_const_type 1: DW_TAG_array_type 2
   DW_TAG_array_type 2: 0 entries, type: DW_TAG_const_type 3
   DW_TAG_const_type 3: real type of the zero sized array

For instance, after this patch we get a sane reconstruction of this
type:

  $ pahole -C filename /home/acme/git/build/v5.0-rc2+/ipc/mqueue.o
  struct filename {
          const char  *              name;                 /*     0     8 */
          const char  *              uptr;                 /*     8     8 */
          int                        refcnt;               /*    16     4 */

          /* XXX 4 bytes hole, try to pack */

          struct audit_names *       aname;                /*    24     8 */
          const char                 iname[];              /*    32     0 */

          /* size: 32, cachelines: 1, members: 5 */
          /* sum members: 28, holes: 1, sum holes: 4 */
          /* last cacheline: 32 bytes */
  };
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-09 18:13:58 -03:00
Arnaldo Carvalho de Melo
b42d77b0bb fprintf: Print __attribute__((__aligned__(N))) for structs/classes
For instance:

  $ pahole -C kern_ipc_perm /home/acme/git/build/v5.0-rc2+/ipc/util.o
  struct kern_ipc_perm {
        spinlock_t                 lock;                 /*     0     4 */
        bool                       deleted;              /*     4     1 */

        /* XXX 3 bytes hole, try to pack */

        int                        id;                   /*     8     4 */
        key_t                      key;                  /*    12     4 */
        kuid_t                     uid;                  /*    16     4 */
        kgid_t                     gid;                  /*    20     4 */
        kuid_t                     cuid;                 /*    24     4 */
        kgid_t                     cgid;                 /*    28     4 */
        umode_t                    mode;                 /*    32     2 */

        /* XXX 6 bytes hole, try to pack */

        long unsigned int          seq;                  /*    40     8 */
        void *                     security;             /*    48     8 */
        struct rhash_head          khtnode;              /*    56     8 */
        /* --- cacheline 1 boundary (64 bytes) --- */
        struct callback_head       rcu __attribute__((__aligned__(8))); /*    64    16 */
        refcount_t                 refcount;             /*    80     4 */

        /* size: 128, cachelines: 2, members: 14 */
        /* sum members: 75, holes: 2, sum holes: 9 */
        /* padding: 44 */
        /* forced alignments: 1 */
  } __attribute__((__aligned__(64)));
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-09 17:03:54 -03:00
Arnaldo Carvalho de Melo
1c9c1d6bbd dwarf_loader: Store DW_AT_alignment if available in DW_TAG_{structure,union,class}_type
That is not just for DW_TAG_class_member :-)

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-09 16:57:52 -03:00
Arnaldo Carvalho de Melo
41c55858da codiff: Add --quiet option
To avoid printing anything when there are no differences to show.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-09 16:46:49 -03:00
Arnaldo Carvalho de Melo
a104eb1ea1 fprintf: Notice explicit bitfield alignment modifications
I.e. when we find that the last member has a bit_hole, i.e. it is part
of a bitfield, and the current field has a bitfield_size, i.e. it _also_
is part of a bitfield, the only explanation is that they were
artificially put in different base types, i.e. like in these fields
in the linux kernel 'struct task_struct', here reconstructed by pahole:

  $ pahole -C task_struct ~/git/build/v5.1-rc2+/kernel/sched/core.o | grep :0 -B9 -A12
          unsigned int               personality;          /*  1128     4 */
          unsigned int               sched_reset_on_fork:1; /*  1132: 0  4 */
          unsigned int               sched_contributes_to_load:1; /*  1132: 1  4 */
          unsigned int               sched_migrated:1;     /*  1132: 2  4 */
          unsigned int               sched_remote_wakeup:1; /*  1132: 3  4 */

          /* XXX 28 bits hole, try to pack */

          /* Force alignment to the next boundary: */
          unsigned int               :0;

          unsigned int               in_execve:1;          /*  1136: 0  4 */
          unsigned int               in_iowait:1;          /*  1136: 1  4 */
          unsigned int               restore_sigmask:1;    /*  1136: 2  4 */
          unsigned int               in_user_fault:1;      /*  1136: 3  4 */
          unsigned int               no_cgroup_migration:1; /*  1136: 4  4 */
          unsigned int               use_memdelay:1;       /*  1136: 5  4 */

          /* XXX 26 bits hole, try to pack */
          /* XXX 4 bytes hole, try to pack */

          long unsigned int          atomic_flags;         /*  1144     8 */
  $

This matches the original definition in the original kernel sources, and
further more, the following sequence proves that with this and DW_AT_alignment,
we can go full circle, i.e.:

1. from an object file reconstruct the source code for all the types that
   appears in function signatures, if pointers, them they will be fully defined,
   not just forward declared:

  $ pfunct --compile=sched_change_group ~/git/build/v5.1-rc2+/kernel/sched/core.o | egrep -w 'sched_change_group|task_struct {' -B10 -A5

          /* --- cacheline 3 boundary (192 bytes) --- */
          struct fpu                 fpu __attribute__((__aligned__(64))); /*   192  4160 */

          /* size: 4352, cachelines: 68, members: 21 */
          /* sum members: 4316, holes: 2, sum holes: 32 */
          /* sum bitfield members: 2 bits, bit holes: 1, sum bit holes: 30 bits */
          /* forced alignments: 1, forced holes: 1, sum forced holes: 28 */
  };

  struct task_struct {
          struct thread_info         thread_info;          /*     0    16 */

          /* XXX last struct has 4 bytes of padding */

          volatile long int          state;                /*    16     8 */
  --
          /* --- cacheline 104 boundary (6656 bytes) --- */
          struct thread_struct       thread __attribute__((__aligned__(64))); /*  6656  4352 */

          /* size: 11008, cachelines: 172, members: 207 */
          /* sum members: 10902, holes: 16, sum holes: 98 */
          /* sum bitfield members: 10 bits, bit holes: 2, sum bit holes: 54 bits */
          /* paddings: 3, sum paddings: 14 */
          /* forced alignments: 6, forced holes: 1, sum forced holes: 40 */
  };

  void sched_change_group(struct task_struct * tsk, int type)
  {
  }
  $

2. Build the regenerated skeleton function + its types:

  $ pfunct --compile=sched_change_group ~/git/build/v5.1-rc2+/kernel/sched/core.o > sched_change_group.c
  $ gcc -g -c sched_change_group.c
  $ file sched_change_group.o
  sched_change_group.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), with debug_info, not stripped
  $

3. Now lets see if the original 'struct task_struct' printed by pahole, matches
   the the output printed by pahole for the DWARF info generated for the regenerated
   'struct task_struct' source code in sched_change_group.c:

  $ pahole -C task_struct sched_change_group.o | tail

          /* --- cacheline 104 boundary (6656 bytes) --- */
          struct thread_struct       thread __attribute__((__aligned__(64))); /*  6656  4352 */

          /* size: 11008, cachelines: 172, members: 207 */
          /* sum members: 10902, holes: 16, sum holes: 98 */
          /* sum bitfield members: 10 bits, bit holes: 2, sum bit holes: 54 bits */
          /* paddings: 3, sum paddings: 14 */
          /* forced alignments: 6, forced holes: 1, sum forced holes: 40 */
  };
  $ pahole -C task_struct ~/git/build/v5.1-rc2+/kernel/sched/core.o | tail

          /* --- cacheline 104 boundary (6656 bytes) --- */
          struct thread_struct       thread __attribute__((__aligned__(64))); /*  6656  4352 */

          /* size: 11008, cachelines: 172, members: 207 */
          /* sum members: 10902, holes: 16, sum holes: 98 */
          /* sum bitfield members: 10 bits, bit holes: 2, sum bit holes: 54 bits */
          /* paddings: 3, sum paddings: 14 */
          /* forced alignments: 6, forced holes: 1, sum forced holes: 40 */
  };
  $

  Furthermore:

  $ pahole -C task_struct ~/git/build/v5.1-rc2+/kernel/sched/core.o > /tmp/original
  $ pahole -C task_struct sched_change_group.o > /tmp/regenerated
  $ diff -u /tmp/original /tmp/regenerated
  $

So one of the most complex data structures in the Linux kernel seems to be under control,
and it uses zero sized unnamed bitfields and __attribute__((aligned(N))), a DWARF5 goodie,
time to go tag v1.13!

Cc: Alexei Starovoitov <ast@fb.com>
Cc: Andrii Nakryiko <andriin@fb.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Yonghong Song <yhs@fb.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-09 15:44:42 -03:00
Arnaldo Carvalho de Melo
75f32a24c7 codiff: Improve the comparision of anonymous struct members
I.e. 'union {};', 'struct {};' members were always appearing as having
been removed, as we normally do lookup by member name, to find out if
its offset, size, type, etc changed.

For unnamed members, try a different heuristic, i.e. look for the nth
anonymous member, this way we're just trying to compare the first
unnamed member of, say, struct OLD with the first unnamed member of
struct NEW, etc.

For OLD == NEW, this works well, for OLD != NEW because some non
anonymous field got added, removed or moved around, ditto, and when the
number of unnamed fields gets decreased, then we can mix things up, and
compare the previously first in A with the previously first in B.

For the current intended use case of:

1) compile a .c file into a .o file with debugging info, say FILE.o

2) use 'pfunct --compile FILE.o > regenerated-FILE.c'

3) compile regenerated-FILE.c into regenerated-FILE.o with debugging info

4) codiff --struct FILE.o regenerated-FILE.o and find out if they match

This gets us moving forward as we'll spot differences with this algo.

For the future we can use a few more heuristics or stop using search by
name members, instead traversing both structs in tandem, spotting the
differences by comparing the fields that way.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-08 15:09:33 -03:00
Arnaldo Carvalho de Melo
6b1e43f2c1 codiff: When comparing against a file with just one CU don't bother finding by name
I.e. when we have two object files with debugging info, and one of them
jas just one CU, then compare all the CUs in the other file to this
unique CU.

Case in hand: encode BTF in a file, then the BTF info has everything in
just one "compile unit", so when looking at the types in the DWARF
originals, we should just compare its types to what is in the single BTF
"compile unit".

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-08 15:07:17 -03:00
Arnaldo Carvalho de Melo
15a754f224 core: Add nr_entries member to 'struct cus'
Will be used when considering comparing multiple CU entries in a struct
cus to the sole compile unit in a second file, like when comparing the
types in a multi-CU DWARF file like vmlinux against the combined,
deduplicated entries in that vmlinux .BTF section.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-08 11:54:06 -03:00
Arnaldo Carvalho de Melo
99750f244c pfunct: Generate a valid return type for the --compile bodies
Before:

  $ pfunct --compile examples/tcp.o > tcp.pahole.c
  $ clang --target=bpf -c -g tcp.pahole.c |& tail
  tcp.pahole.c:6154:1: warning: control reaches end of non-void function [-Wreturn-type]
  }
  ^
  tcp.pahole.c:6158:1: warning: control reaches end of non-void function [-Wreturn-type]
  }
  ^
  tcp.pahole.c:6170:1: warning: control reaches end of non-void function [-Wreturn-type]
  }
  ^
  192 warnings generated.
  $ head -6170 tcp.pahole.c | tail -3
  inline int arch_atomic_read(const atomic_t  * v)
  {
  }
  $

After:

  $ pfunct --compile examples/tcp.o > tcp.pahole.c
  $ clang --target=bpf -c -g tcp.pahole.c
  $ grep -A3 -w arch_atomic_read tcp.pahole.c
  inline int arch_atomic_read(const atomic_t  * v)
  {
	  return 0;
  }
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 16:33:43 -03:00
Arnaldo Carvalho de Melo
881aabd6fc reorganize: Introduce class__for_each_member_from_safe()
Reducing boilerplate, keeping consistent with the other member traversal
helpers.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 16:16:16 -03:00
Arnaldo Carvalho de Melo
1b2e3389f3 reorganize: Introduce class__for_each_member_reverse()
Reducing boilerplate, keeping consistent with the other member traversal
helpers.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 16:11:42 -03:00
Arnaldo Carvalho de Melo
10fef2916d reorganize: Introduce class__for_each_member_continue()
To get a lot of boilerplate behind a nice helper.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 16:07:30 -03:00
Arnaldo Carvalho de Melo
e7a56ee8cc reorganize: Introduce class__for_each_member_from()
To get a lot of boilerplate behind a nice helper.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 16:05:28 -03:00
Arnaldo Carvalho de Melo
9a79bb6ced tag: Introduce tag__is_pointer_to()
To shorten the check if a tag is a pointer to a particular type.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 15:28:55 -03:00
Arnaldo Carvalho de Melo
45ad545944 tag: Introduce tag__is_pointer()
For the usual idiom to ask if a tag is a pointer, removing a bit of
DWARFism and shortening the operation.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 15:21:55 -03:00
Arnaldo Carvalho de Melo
89ce57a02e pdwtags: Find holes in structs
The pdwtags prints all tags, so call class__find_holes() for structs so
that we don't print BFAs.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-05 11:34:33 -03:00
Arnaldo Carvalho de Melo
ce6f393bc9 fprintf: Fixup the printing of const parameters
The last problem with 'pfunct --compile' at least for tcp.o:

Before:

  $ pfunct --compile examples/tcp.o > tcp.pahole.c
  $ gcc -c tcp.pahole.c -g
  tcp.pahole.c:1808:48: error: unknown type name ‘u8const’; did you mean ‘const’?
   inline void tcp_set_ca_state(struct sock * sk, u8const ca_state)
                                                  ^~~~~~~
                                                  const
  tcp.pahole.c:5346:56: error: unknown type name ‘intconst’; did you mean ‘const’?
   inline void skb_set_tail_pointer(struct sk_buff * skb, intconst offset)
                                                          ^~~~~~~~
                                                          const
  tcp.pahole.c:5914:37: error: unknown type name ‘gfp_tconst’; did you mean ‘gfp_t’?
   inline bool gfpflags_allow_blocking(gfp_tconst gfp_flags)
                                       ^~~~~~~~~~
                                       gfp_t
  tcp.pahole.c:5926:24: error: unknown type name ‘ktime_tconst’; did you mean ‘ktime_t’?
   inline s64 ktime_to_ns(ktime_tconst kt)
                          ^~~~~~~~~~~~
                          ktime_t
  tcp.pahole.c:5939:54: warning: ‘struct timespec64const’ declared inside parameter list will not be visible outside of this definition or declaration
   inline struct timespec timespec64_to_timespec(struct timespec64const ts64)
                                                        ^~~~~~~~~~~~~~~
  tcp.pahole.c:5939:70: error: parameter 1 (‘ts64’) has incomplete type
   inline struct timespec timespec64_to_timespec(struct timespec64const ts64)
                                                 ~~~~~~~~~~~~~~~~~~~~~~~^~~~
  $

After:

  $ pfunct --compile examples/tcp.o > tcp.pahole.c
  $ gcc -c tcp.pahole.c -g

Because:

  $ grep -A2 tcp_set_ca_state tcp.pahole.c
  inline void tcp_set_ca_state(struct sock * sk, const u8 ca_state)
  {
  }
  $

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2019-04-04 21:59:23 -03:00