When performing a "fat" LTO the compiler has a whole mess of codegen
units that it links together. To do this it needs to select one module
as a "base" module and then link everything else into this module.
Previously LTO passes assume that there's at least one module in-memory
to link into, but nowadays that's not always true! With incremental
compilation modules may actually largely be cached and it may be
possible that there's no in-memory modules to work with.
This commit updates the logic of the LTO backend to handle modules a bit
more uniformly during a fat LTO. This commit immediately splits them
into two lists, one serialized and one in-memory. The in-memory list is
then searched for the largest module and failing that we simply
deserialize the first serialized module and link into that. This
refactoring avoids juggling three lists, two of which are serialized
modules and one of which is half serialized and half in-memory.
Closes#63349
This commit removes the crates.io dependency of `rustc-demangle` from
`rustc_codegen_llvm`. This crate is actually already pulled in to part
of the `librustc_driver` build and with the upcoming pipelining
implementation in Cargo it causes build issues if `rustc-demangle` is
left to its own devices.
This is not currently required, but once pipelining is enabled for
rustc's own build it will be required to build correctly.
Now that procedural macros no longer link transitively to libsyntax,
this shouldn't be needed any more! This commit is an experiment in
removing all dynamic libraries from rustc except for librustc_driver
itself. Let's see how far we can get with that!
Implement another internal lints
cc #49509
This adds ~~two~~ one internal lint~~s~~:
1. LINT_PASS_IMPL_WITHOUT_MACRO: Make sure, that the `{declare,impl}_lint_pass` macro is used to implement lint passes. cc #59669
2. ~~USAGE_OF_TYCTXT_AND_SPAN_ARGS: item 2 on the list in #49509~~
~~With 2. I wasn't sure, if this lint should be applied everywhere. That means a careful review of 0955835 would be great. Also 73fb9b4 allows this lint on some functions. Should I also apply this lint there?~~
TODO (not directly relevant for review):
- [ ] https://github.com/rust-lang/rust/pull/59316#discussion_r280186517 (not sure yet, if this works or how to query for `rustc_private`, since it's not in [`Features`](https://doc.rust-lang.org/nightly/nightly-rustc/syntax/feature_gate/struct.Features.html) 🤔 cc @eddyb)
- [x] https://github.com/rust-lang/rust/pull/61735#discussion_r292389870
- [x] Check explicitly for the `{declare,impl}_lint_pass!` macros
r? @oli-obk
This commit separates metadata encoding (`tcx.encode_metadata`) from the
creation of the metadata module (which is now handled by
`write_compressed_metadata`, formerly `write_metadata`).
The metadata encoding now occurs slightly earlier in the pipeline, at
the very start of code generation within `start_codegen`.
Metadata *writing* still occurs near the end of compilation; that will
be moved forward in subsequent commits.
To implement pipelining, Cargo needs to know when metadata generation is
finished. This commit adds code to do that. Unfortunately, metadata file
writing currently occurs very late during compilation, so pipelining
won't produce a speed-up. Moving metadata file writing earlier will be a
follow-up.
The change involves splitting the existing `Emitter::emit` method in
two: `Emitter::emit_diagnostic` and `Emitter::emit_directive`.
The JSON directives look like this:
```
{"directive":"metadata file written: liba.rmeta"}
```
The functionality is behind the `-Z emit-directives` option, and also
requires `--error-format=json`.
Stabilize Range*::contains.
Closes https://github.com/rust-lang/rust/issues/32311. There's also a bit of rustfmt on range.rs thrown in for good measure (I forgot to turn off format-on-save in VSCode).
`codegen_allocator` and `write_metadata` mutate the underlying LLVM module. As
such, it makes sense for these two functions to receive a mutable reference to
the module (as opposed to an immutable one).
Stabilize slice_sort_by_cached_key
I was going to ask on the tracking issue (https://github.com/rust-lang/rust/issues/34447), but decided to just send this and hope for an FCP here. The method was added last March by https://github.com/rust-lang/rust/pull/48639.
Signature: https://doc.rust-lang.org/std/primitive.slice.html#method.sort_by_cached_key
```rust
impl [T] {
pub fn sort_by_cached_key<K, F>(&mut self, f: F)
where F: FnMut(&T) -> K, K: Ord;
}
```
That's an identical signature to the existing `sort_by_key`, so I think the questions are just naming, implementation, and the usual "do we want this?".
The implementation seems to have proven its use in rustc at least, which many uses: https://github.com/rust-lang/rust/search?l=Rust&q=sort_by_cached_key
(I'm asking because it's exactly what I just needed the other day:
```rust
all_positions.sort_by_cached_key(|&n|
data::CITIES.iter()
.map(|x| *metric_closure.get_edge(n, x.pos).unwrap())
.sum::<usize>()
);
```
since caching that key is a pretty obviously good idea.)
Closes#34447
Currently the compiler will produce an error if both incremental
compilation and full fat LTO is requested. With recent changes and the
advent of incremental ThinLTO, however, all the hard work is already
done for us and it's actually not too bad to remove this error!
This commit updates the codegen backend to allow incremental full fat
LTO. The semantics are that the input modules to LTO are all produce
incrementally, but the final LTO step is always done unconditionally
regardless of whether the inputs changed or not. The only real
incremental win we could have here is if zero of the input modules
changed, but that's so rare it's unlikely to be worthwhile to implement
such a code path.
cc #57968
cc rust-lang/cargo#6643
csmoe
Mark Rousskov
Guanqun Lu
Alex Crichton
Vadim Petrochenkov
Mazdak Farrokhzad
Jeremy Stucki
Chris Gregory
flip1995
Matthew Jasper
Eduard-Mihai Burtescu
Mark Mansi
Nicholas Nethercote
bjorn3
Yuki OKUSHI
Wesley Wiser
Simonas Kazlauskas
kennytm
Steven Malis
John Kåre Alsaker
Gabriela Alexandra Moldovan
Taiki Endo
Scott McMurray