Commit Graph

2001 Commits

Author SHA1 Message Date
Arnaldo Carvalho de Melo a201149e18 dwarf_loader: No need to strdup() what dwarf_formstring() returns
Conversation with Mark Wieelard, elfutils developer:

  acme | ultimately dwarf_attr->valp for strings point to what? DIE memory that is always there?
  acme | I'm working on pahole and need to keep a pointer to what it returns
  acme | I was strdup()ing what dwarf_formstring() returns, tried removing the strdup() and instead point to what dwarf_formstring() returns and it worked, but I want to know for sure
   mjw | ah, yeah
   mjw | the memory/string returned by dwarf_formstring() is owned by the Dwarf. So long as the Dwarf is active (dwarf_end() hasn't been called and the underlying Elf is valid of course) you can use that string.
  acme | cool!
  acme | I'll use your explanation in the commit log message
  acme | thanks!
  acme | I'm also working on multithreading DWARF loading
   mjw | in most cases it will point directly into the .debug_str section, but it can also be the .debug_line_str section or a string embedded in the .debug_info section, etc.
   mjw | in all cases the Dwarf is responsible for keeping the memory life.

Before:

⬢[acme@toolbox pahole]$ rm -f vmlinux.btf ; perf stat -r5 pahole --btf_encode_detached vmlinux.btf vmlinux && perf stat -r5 btfdiff vmlinux vmlinux.btf

 Performance counter stats for 'pahole --btf_encode_detached vmlinux.btf vmlinux' (5 runs):

          7,802.91 msec task-clock:u              #    0.989 CPUs utilized            ( +-  0.60% )
                 0      context-switches:u        #    0.000 /sec
                 0      cpu-migrations:u          #    0.000 /sec
           871,574      page-faults:u             #  110.568 K/sec                    ( +-  0.00% )
    29,924,977,089      cycles:u                  #    3.796 GHz                      ( +-  0.60% )  (83.32%)
       455,561,473      stalled-cycles-frontend:u #    1.51% frontend cycles idle     ( +-  5.55% )  (83.33%)
     3,874,761,771      stalled-cycles-backend:u  #   12.86% backend cycles idle      ( +-  2.24% )  (83.34%)
    74,812,680,221      instructions:u            #    2.48  insn per cycle
                                                  #    0.05  stalled cycles per insn  ( +-  0.02% )  (83.34%)
    17,624,163,403      branches:u                #    2.236 G/sec                    ( +-  0.03% )  (83.34%)
       128,991,472      branch-misses:u           #    0.73% of all branches          ( +-  0.07% )  (83.33%)

            7.8861 +- 0.0471 seconds time elapsed  ( +-  0.60% )

 Performance counter stats for 'btfdiff vmlinux vmlinux.btf' (5 runs):

          6,323.23 msec task-clock:u              #    1.000 CPUs utilized            ( +-  0.97% )
                 0      context-switches:u        #    0.000 /sec
                 0      cpu-migrations:u          #    0.000 /sec
           826,233      page-faults:u             #  130.852 K/sec                    ( +-  0.00% )
    23,719,098,640      cycles:u                  #    3.756 GHz                      ( +-  0.32% )  (83.35%)
       286,636,981      stalled-cycles-frontend:u #    1.21% frontend cycles idle     ( +-  2.52% )  (83.34%)
     2,821,674,085      stalled-cycles-backend:u  #   11.91% backend cycles idle      ( +-  1.20% )  (83.28%)
    64,095,069,092      instructions:u            #    2.70  insn per cycle
                                                  #    0.04  stalled cycles per insn  ( +-  0.03% )  (83.35%)
    15,398,500,941      branches:u                #    2.439 G/sec                    ( +-  0.02% )  (83.35%)
        80,187,703      branch-misses:u           #    0.52% of all branches          ( +-  0.32% )  (83.34%)

            6.3233 +- 0.0613 seconds time elapsed  ( +-  0.97% )

⬢[acme@toolbox pahole]$

After:

 static struct dwarf_off_ref attr_type(Dwarf_Die *die, uint32_t attr_name)
⬢[acme@toolbox pahole]$ rm -f vmlinux.btf ; perf stat -r5 pahole --btf_encode_detached vmlinux.btf vmlinux && perf stat -r5 btfdiff vmlinux vmlinux.btf

 Performance counter stats for 'pahole --btf_encode_detached vmlinux.btf vmlinux' (5 runs):

          7,008.59 msec task-clock:u              #    0.977 CPUs utilized            ( +-  1.03% )
                 0      context-switches:u        #    0.000 /sec
                 0      cpu-migrations:u          #    0.000 /sec
           796,469      page-faults:u             #  111.073 K/sec                    ( +-  0.00% )
    28,167,752,342      cycles:u                  #    3.928 GHz                      ( +-  0.26% )  (83.32%)
       377,704,478      stalled-cycles-frontend:u #    1.35% frontend cycles idle     ( +-  0.96% )  (83.34%)
     3,758,855,221      stalled-cycles-backend:u  #   13.43% backend cycles idle      ( +-  1.68% )  (83.34%)
    72,453,367,989      instructions:u            #    2.59  insn per cycle
                                                  #    0.05  stalled cycles per insn  ( +-  0.03% )  (83.33%)
    17,110,081,987      branches:u                #    2.386 G/sec                    ( +-  0.02% )  (83.34%)
       116,081,751      branch-misses:u           #    0.68% of all branches          ( +-  0.32% )  (83.33%)

            7.1731 +- 0.0724 seconds time elapsed  ( +-  1.01% )

 Performance counter stats for 'btfdiff vmlinux vmlinux.btf' (5 runs):

          5,768.59 msec task-clock:u              #    1.014 CPUs utilized            ( +-  0.45% )
                 0      context-switches:u        #    0.000 /sec
                 0      cpu-migrations:u          #    0.000 /sec
           751,092      page-faults:u             #  132.237 K/sec                    ( +-  0.00% )
    21,623,439,905      cycles:u                  #    3.807 GHz                      ( +-  0.46% )  (83.34%)
       221,665,165      stalled-cycles-frontend:u #    1.02% frontend cycles idle     ( +-  1.55% )  (83.30%)
     2,860,640,878      stalled-cycles-backend:u  #   13.10% backend cycles idle      ( +-  2.03% )  (83.32%)
    61,757,937,981      instructions:u            #    2.83  insn per cycle
                                                  #    0.04  stalled cycles per insn  ( +-  0.01% )  (83.37%)
    14,873,361,434      branches:u                #    2.619 G/sec                    ( +-  0.02% )  (83.36%)
        65,356,868      branch-misses:u           #    0.44% of all branches          ( +-  0.07% )  (83.35%)

            5.6884 +- 0.0282 seconds time elapsed  ( +-  0.50% )

⬢[acme@toolbox pahole]$

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 6b7f1b72f9 core: No need for debug_fmt_ops->variable_name() anymore
Since we store a char pointer string in variable->name, we can use it
directly.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 9d0e3ab9a2 pahole: function__name() doesn't need a 'struct cu *' argument
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo a7d789a4f8 core: Make variable->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo b5694280ec core: Make label->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo e974d1b240 pahole: class_member_filter__new() doesn't need a 'struct cu *' argument
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 0275e8d249 pahole: class_member_filter__parse() doesn't need a 'struct cu *' argument
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 90183e8e4d pahole: tag__real_sizeof() doesn't need a 'struct cu *' argument
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 5cb9192738 pahole: Rename tag__fprintf_hexdump_value() to instance__fprintf_hexdump_value()
As it acts only on an instance, doesn't need neither a 'struct tag' nor
a 'struct cu'.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 75c769a900 pahole: enumerations__lookup_entry_from_value() doesn't need to return a CU anymore
As it will not be used in the caller.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 1edca26552 pahole: enumeration__lookup_entry_from_value() doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo f8d98eff75 core: Ditch unused enumeration__prefix_len() method
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 5cc365164a core: Ditch unused enumeration__prefix() method
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo e18c60d793 pahole: enumeration__lookup_value() doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 4b877c8e67 pahole: enumeration__lookup_enumerator() doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 3ff11828fe core: enumeration__emit_definitions() doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 0947d6e795 core: enumeration__fprintf() doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo bb22f5bb0a core: Make enumeration__max_entry_name_len() static
As it is not used outside where it is defined.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo dc83336171 core: enumeration__max_entry_name_len() doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo ec1667c76d core: Make enumeration__calc_prefix() static
As it is not used outside where it is defined.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 45ec63ed20 core: enumeration__calc_prefix doesn't need a 'cu' argument
With the conversion of ->name members to plain char strings, no need
to use 'cu' to get the old string_t index and find the per-cu string
table.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 2fae84e2f7 btf_encoder: btf_encoder__add_enum_type() doesn't need a 'cu' argument, ditch it
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 96243fdd79 core: enumerator__name() doesn't need a 'cu' argument, ditch it
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo ee5c12893b core: Ditch dwarves__active_loader, unused
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 713239bc00 core: Make enumerator->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

For the enumerator->name case we get the bonus of removing the last user
of dwarves__active_loader in the btf_encoder class.

This covers unions, enums, structs and classes.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 7721cc17ac emit: type__emit_fwd_decl() isn't used outside emit.c, make it static
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 9aa5db7acd emit: type__emit_fwd_decl() doesn't need a cu arg
Due to previous simplifications in the FOO__name() methods.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo c46f91ef6f emit: type_emissions__find_definition() doesn't need a cu arg
Due to previous simplifications in the FOO__name() methods.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo c127d25daf core: class__name() doesn't need a cu arg
Now that namespace->name is a real char string.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 00e8c5fe21 core: type__name() doesn't need a cu arg
Now that namespace->name is a real char string.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo b99c4008ac core: Make namespace->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

For the namespace->name case we get the bonus of removing another
user of dwarves__active_loader.

This covers unions, enums, structs and classes.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 379a73c6eb core: Make class_member->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

For the class_member->name case we get the bonus of removing another
user of dwarves__active_loader.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo 3280cb4176 core: Make parameter->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

For the parameter->name case we get the bonus of removing a user of
dwarves__active_loader.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo f009162fd1 core: Make base_type->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

For the base_type->name case we get the bonus of removing some more
functions related base types and a user of dwarves__active_loader.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:46 -03:00
Arnaldo Carvalho de Melo e2ee753fa7 pahole: Disable incomplete CTF encoder
It was never fully functional and now its getting in the way of further
improvements for loading/encoding BTF and loading DWARF.

Eventually we can use Oracle's libctf library to add an encoder.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-08-12 09:39:44 -03:00
Arnaldo Carvalho de Melo 0d13bc50ee core: Ditch unused cu__find_struct_by_sname()
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:17 -03:00
Arnaldo Carvalho de Melo 46f3f37241 core: Convert cu__find_base_type_by_sname_and_size to search for a string
It looked for an index into a string table, a string_t, but since for
multithreading we'd be growing the string table while looking up stuff,
we'd be looking at realloc'ed memory, so lets move to stable char
pointer strings.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo f84e8777ea core: Convert cu__find_enumeration_by_sname_and_size to search for a string
It looked for an index into a string table, a string_t, but since for
multithreading we'd be growing the string table while looking up stuff,
we'd be looking at realloc'ed memory, so lets move to stable char
pointer strings.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo a16c82f711 dwarf_loader: Rename strings_t 'name' to 'sname' to clarify usage
We're transitioning to plain strings, so lets pave the way for that
step by step.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 82e5b5101a core: Make function->name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

For the function->name case we get the bonus of removing the need of a
debug_fmt_ops->function() callback receiving the 'cu', just access the
string directly.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 4f73cac853 core: Make function->linkage_name a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo a93160df53 dwarf_loader: Make dwarf_tag->decl_file a real string
For the threaded code we want to access strings in tags at the same time
that the string table may grow in another thread making the previous
pointer invalid, so, to avoid excessive locking, use plain strings.

The way the tools work will either consume the just produced CU straight
away or keep just one copy of each data structure when we keep all CUs
in memory, so lets try stopping using strings_t for strings.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo a3fcbcacf7 pahole: Allow specifying the number of threads to use while loading files
Using the same command line option as 'make' and 'ninja': -j.
Unfortunately the argp parser in glibc expects a '=' for single letter
args, so it is:

  $ pahole -j=10

or, like with 'make':

  $ make --jobs=10

This is unwired at the moment, probably I'll reorder the patch, but for
testing, add it first.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo d70b2562ee pahole: Make '-j' available for use as number of jobs (threads)
The '-j' option, to generate detached BTF files wasn't published in any
tagged release, so leave it as just --btf_encode_detached and make '-j'
available for specifying the number of jobs/threads, as is the case with
'make -j', 'ninja -j', etc.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 41a283c65d core: Protect cus->cus with a mutex
Paving the way for having multiple threads creating CUs and possibly
adding them to cus->cus.

The mutex will also be used to ask elfutils-libdwfl to read the next CU,
after a thread finishes reading one.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 972065482a core: Make 'struct cus' opaque, only visible in dwarves.c
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 3895b29060 core: Introduce helper to return number of cu entries in a 'struct cus'
Provide a helper so that we can make 'struct cus' opaque.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 874e750fb8 core: Move cus__find_pair() from codiff to the core
To avoid touching 'struct cus' internal state outside dwarves.c

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo bf74fc1fcf core: Introduce helper to return if there is no cu entries in a 'struct cus'
Provide a helper so that we can make 'struct cus' opaque.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00
Arnaldo Carvalho de Melo 7020f92143 core: Prepare cus__find_cu_by_name() for locking
By having just one exit point.

Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2021-07-28 15:18:16 -03:00