PR 24445 ("dwz multifile index not written to index cache") exposed the
fact that we are not doing things right when we generate an index for an
object file that has is linked to a dwz file. The same happens whether
the index is generated with the intent of populating the index cache or
using the save gdb-index command.
The problem can be observed when running these tests with the
cc-with-dwz-m board:
FAIL: gdb.base/index-cache.exp: test_cache_enabled_hit: check index-cache stats
FAIL: gdb.dwarf2/gdb-index.exp: index used
FAIL: gdb.dwarf2/gdb-index.exp: index used after symbol reloading
When generating the index for such file and inspecting the CU list of the
resulting index (with readelf --debug-dump=gdb_index), we can see something
like:
CU table:
[ 0] 0x0 - 0xb9
[ 1] 0x0 - 0x44
This is supposed to be a sorted list of the ranges of all CUs in the
file this index represents, so already having some overlap is a red
flag. It turns out that we save the ranges of CUs coming from both the
main file and the dwz file in the same index.
After digging a little bit, it became quite obvious that the index in
the main file should only list the CUs present in the main file, and a
separate index should be generated for the dwz file, listing the CUs
present in that file.
First, that's what happens if you run dwz on a file that already has a
GDB index embedded. Second, dwarf2read.c has code to read an index from
a dwz file. The index in the dwz file is actually required to be
present, if the main file has an index.
So this patch changes write_psymtabs_to_index to generate an index for
the dwz file, if present. That index only contains a CU list, just like
what the dwz tool does when processing a file that already contains an
index.
Some notes about the implementation:
- The file management (creating a temp file, make sure it's
close/removed on error - in the right order) is a bit heavy in
write_psymtabs_to_index, and I needed to add a third file. I factored
this pattern in a separate class, index_wip_file.
- It became a bit tedious to keep the call to assert_file_size in
write_psymtabs_to_index, write_gdbindex would have had to return two
sizes. Instead, I moved the calls to assert_file_size where the file
is written. The downside is that we lose the filename at this point,
but it was only used for the very improbable case of ftell failing, so
I think it's not a problem.
- The actual writing of the index file is factored out to
write_gdbindex_1, so it can be re-used for both index files.
- While the "save gdb-index" command will now write two .gdb-index
files, this patch does not update the gdb-add-index.sh script, this
will come in a later patch.
gdb/ChangeLog:
YYYY-MM-DD Simon Marchi <simon.marchi@efficios.com>
PR gdb/24445
* dwarf-index-write.h (write_psymtabs_to_index): Add
dwz_basename parameter.
* dwarf-index-write.c (write_gdbindex): Move file writing to
write_gdbindex_1. Change return type void.
(assert_file_size): Move up, remove filename parameter.
(write_gdbindex_1): New function.
(write_debug_names): Change return type to void, call
assert_file_size.
(struct index_wip_file): New struct.
(write_psymtabs_to_index): Add dwz_basename parameter. Move
file logic to index_wip_file. Write index for dwz file if
needed.
(save_gdb_index_command): Pass basename of dwz file, if present.
* dwarf-index-cache.c (index_cache::store): Obtain and pass
build-id of dwz file, if present.
* dwarf2read.c (struct dwz_file): Move to dwarf2read.h.
(dwarf2_get_dwz_file): Likewise.
* dwarf2read.h (struct dwz_file): Move from dwarf2read.c.
(dwarf2_get_dwz_file): Likewise.
gdb/testsuite/ChangeLog:
YYYY-MM-DD Tom de Vries <tdevries@suse.de>
PR gdb/24445
* gdb.dwarf2/gdb-index.exp (add_gdb_index): Update dwz file with
generated index.
This commit applies all changes made after running the gdb/copyright.py
script.
Note that one file was flagged by the script, due to an invalid
copyright header
(gdb/unittests/basic_string_view/element_access/char/empty.cc).
As the file was copied from GCC's libstdc++-v3 testsuite, this commit
leaves this file untouched for the time being; a patch to fix the header
was sent to gcc-patches first.
gdb/ChangeLog:
Update copyright year range in all GDB files.
New in v3:
- Remove things related to the dwarf-5 format.
- Fix compilation on mingw (scoped_mmap.c).
GDB can generate indexes for DWARF debug information, which, when
integrated in the original binary, can speed up loading object files.
This can be done using the gdb-add-index script or directly by the
linker itself. However, not many people know about this. And even
among those who do, because it requires additional steps, I don't know a
lot of people who actually go through that trouble.
To help make using the DWARF index more transparent, this patch
introduces a DWARF index cache. When enabled, loading an index-less
binary in GDB will automatically save an index file in ~/.cache/gdb.
When loading that same object file again, the index file will be looked
up and used to load the DWARF index. You therefore get the benefit of
the DWARF index without having to do additional manual steps or
modifying your build system. When an index section is already present
in the file, GDB will prefer that one over looking up the cache.
When doing my edit-compile-debug cycle, I often debug multiple times the
same build, so the cache helps reducing the load time of the debug
sessions after the first one.
- The saved index file is exactly the same as the output of the "save
gdb-index" command. It is therefore the exact same content that would
be found in the .gdb_index or .debug_names section. We just leave it
as a standalone file instead of merging it in the binary.
- The cache is just a directory with files named after the object
file's build-id. It is not possible to save/load the index for an
object file without build-id in the cache.
- The cache uses the gdb index format. The problem with the dwarf-5
format is that we can generate an addendum to the .debug_str section
that you're supposed to integrate to the original binary. This
complicates a little bit loading the data from the cached index files,
so I would leave this for later.
- The size taken up by ~/.cache/gdb is not limited. I was thinking we
could add configurable limit (like ccache does), but that would come
after. Also, maybe a command to flush the cache.
- The cache is disabled by default. I think once it's been out there
and tested for a while, it could be turned on by default, so that
everybody can enjoy it.
- The code was made to follow the XDG specification: if the
XDG_CACHE_HOME environment variable, it is used, otherwise it falls
back to ~/.cache/gdb. It is possible to change it using "set
index-cache directory". On other OSes than GNU/Linux, ~/.cache may
not be the best place to put such data. On macOS it should probably
default to ~/Library/Caches/... On Windows, %LocalAppData%/... I
don't intend to do this part, but further patches are welcome.
- I think that we need to be careful that multiple instances of GDB
don't interfere with each other (not far fetched at all if you run GDB
in some automated script) and the cache is always coherent (either the
file is not found, or it is found and entirely valid). Writing the
file directly to its final location seems like a recipe for failure.
One GDB could read a file in the index while it is being written by
another GDB. To mitigate this, I made write_psymtabs_to_index write
to temporary files and rename them once it's done. Two GDB instances
writing the index for the same file should not step on each other's
toes (the last file to be renamed will stay). A GDB looking up a file
will only see a complete file or no file. Also, if GDB crashes while
generating the index file, it will leave a work-in-progress file, but
it won't be picked up by other instances looking up in the cache.
gdb/ChangeLog:
* common/pathstuff.h (get_standard_cache_dir): New.
* common/pathstuff.c (get_standard_cache_dir): New.
* build-id.h (build_id_to_string): New.
* dwarf-index-common.h (INDEX4_SUFFIX, INDEX5_SUFFIX,
DEBUG_STR_SUFFIX): Move to here.
* dwarf-index-write.c (INDEX4_SUFFIX, INDEX5_SUFFIX,
DEBUG_STR_SUFFIX): Move from there.
(write_psymtabs_to_index): Make non-static, add basename
parameter. Write to temporary files, rename when done.
(save_gdb_index_command): Adjust call to
write_psymtabs_to_index.
* dwarf2read.h (dwarf2_per_objfile) <index_cache_res>: New
field.
* dwarf2read.c (dwz_file) <index_cache_res>: New field.
(get_gdb_index_contents_from_cache): New.
(get_gdb_index_contents_from_cache_dwz): New.
(dwarf2_initialize_objfile): Read index from cache.
(dwarf2_build_psymtabs): Save to index.
* dwarf-index-cache.h: New file.
* dwarf-index-cache.c: New file.
* dwarf-index-write.h: New file.
gdb/testsuite/ChangeLog:
* boards/index-cache-gdb.exp: New file.
* gdb.dwarf2/index-cache.exp: New file.
* gdb.dwarf2/index-cache.c: New file.
* gdb.base/maint.exp: Check if we are using the index cache.
Joel Brobecker
Simon Marchi
Tom Tromey
Simon Marchi