Remove interior mutability from TraitDef by turning fields into queries
This PR gets rid of anything `std::cell` in `TraitDef` by
- moving the global list of trait impls from `TraitDef` into a query,
- moving the list of trait impls relevent for some self-type from `TraitDef` into a query
- moving the specialization graph of trait impls into a query, and
- moving `TraitDef::object_safety` into a query.
I really like how querifying things not only helps with incremental compilation and on-demand, but also just plain makes the code cleaner `:)`
There are also some smaller fixes in the PR. Commits can be reviewed separately.
r? @eddyb or @nikomatsakis
This commit adds a new `-Z` flag to the compiler for use when bootstrapping the
compiler itself. We want to be able to use crates.io crates, but we also want
the usage of such crates to be as ergonomic as possible! To that end compiler
crates are a little tricky in that the crates.io crates are not annotated as
unstable, nor do they expect to pull in unstable dependencies.
To cover all these situations it's intended that the compiler will forever now
bootstrap with `-Z force-unstable-if-unmarked`. This flags serves a dual purpose
of forcing crates.io crates to themselves be unstable while also allowing them
to use other "unstable" crates.io crates. This should mean that adding a
dependency to compiler no longer requires upstream modification with
unstable/staged_api attributes for inclusion!
The new setup is as follows. There is a pipeline of MIR passes that each
run **per def-id** to optimize a particular function. You are intended
to request MIR at whatever stage you need it. At the moment, there is
only one stage you can request:
- `optimized_mir(def_id)`
This yields the final product. Internally, it pulls the MIR for the
given def-id through a series of steps. Right now, these are still using
an "interned ref-cell" but they are intended to "steal" from one
another:
- `mir_build` -- performs the initial construction for local MIR
- `mir_pass_set` -- performs a suite of optimizations and transformations
- `mir_pass` -- an individual optimization within a suite
So, to construct the optimized MIR, we invoke:
mir_pass_set((MIR_OPTIMIZED, def_id))
which will build up the final MIR.
Overall goal: reduce the amount of context a mir pass needs so that it
resembles a query.
- The hooks are no longer "threaded down" to the pass, but rather run
automatically from the top-level (we also thread down the current pass
number, so that the files are sorted better).
- The hook now receives a *single* callback, rather than a callback per-MIR.
- The traits are no longer lifetime parameters, which moved to the
methods -- given that we required
`for<'tcx>` objecs, there wasn't much point to that.
- Several passes now store a `String` instead of a `&'l str` (again, no
point).
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.
The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
Instead of requesting the region maps for the entire crate, request for
a given item etc. Several bits of code were modified to take
`&RegionMaps` as input (e.g., the `resolve_regions_and_report_errors()`
function). I am not totally happy with this setup -- I *think* I'd
rather have the region maps be part of typeck tables -- but at least the
`RegionMaps` works in a "parallel" way to `FreeRegionMap`, so it's not
too bad. Given that I expect a lot of this code to go away with NLL, I
didn't want to invest *too* much energy tweaking it.
this avoids parsing item attributes on each call to `item_attrs`, which takes
off 33% (!) of translation time and 50% (!) of trans-item collection time.
ICH: Replace old, transitive metadata hashing with direct hashing approach.
This PR replaces the old crate metadata hashing strategy with a new one that directly (but stably) hashes all values we encode into the metadata. Previously we would track what data got accessed during metadata encoding and then hash the input nodes (HIR and upstream metadata) that were transitively reachable from the accessed data. While this strategy was sound, it had two major downsides:
1. It was susceptible to generating false positives, i.e. some input node might have changed without actually affecting the content of the metadata. That metadata entry would still show up as changed.
2. It was susceptible to quadratic blow-up when many metadata nodes shared the same input nodes, which would then get hashed over and over again.
The new method does not have these disadvantages and it's also a first step towards caching more intermediate results in the compiler.
Metadata hashing/cross-crate incremental compilation is still kept behind the `-Zincremental-cc` flag even after this PR. Once the new method has proven itself with more tests, we can remove the flag and enable cross-crate support by default again.
r? @nikomatsakis
cc @rust-lang/compiler
remove unnecessary tasks
Remove various unnecessary tasks. All of these are "always execute" tasks that don't do any writes to tracked state (or else an assert would trigger, anyhow). In some cases, they issue lints or errors, but we''ll deal with that -- and anyway side-effects outside of a task don't cause problems for anything that I can see.
The one non-trivial refactoring here is the borrowck conversion, which adds the requirement to go from a `DefId` to a `BodyId`. I tried to make a useful helper here.
r? @eddyb
cc #40746
cc @cramertj @michaelwoerister
Instead of collecting all potential inputs to some metadata entry and
hashing those, we directly hash the values we are storing in metadata.
This is more accurate and doesn't suffer from quadratic blow-up when
many entries have the same dependencies.
Handle DefPath hashing centrally as part of DefPathTable (+ save work during SVH calculation)
In almost all cases where we construct a `DefPath`, we just hash it and throw it away again immediately.
With this PR, the compiler will immediately compute and store the hash for each `DefPath` as it is allocated. This way we
+ can get rid of any subsequent `DefPath` hash caching (e.g. the `DefPathHashes`),
+ don't need to allocate a transient `Vec` for holding the `DefPath` (although I'm always surprised how little these small, dynamic allocations seem to hurt performance), and
+ we don't hash `DefPath` prefixes over and over again.
That last part is because we construct the hash for `prefix::foo` by hashing `(hash(prefix), foo)` instead of hashing every component of prefix.
The last commit of this PR is pretty neat, I think:
```
The SVH (Strict Version Hash) of a crate is currently computed
by hashing the ICHes (Incremental Computation Hashes) of the
crate's HIR. This is fine, expect that for incr. comp. we compute
two ICH values for each HIR item, one for the complete item and
one that just includes the item's interface. The two hashes are
are needed for dependency tracking but if we are compiling
non-incrementally and just need the ICH values for the SVH,
one of them is enough, giving us the opportunity to save some
work in this case.
```
r? @nikomatsakis
This PR depends on https://github.com/rust-lang/rust/pull/40878 to be merged first (you can ignore the first commit for reviewing, that's just https://github.com/rust-lang/rust/pull/40878).
On demandify reachability
cc https://github.com/rust-lang/rust/issues/40746
I tried following this guidance from #40746:
> The following tasks currently execute before a tcx is built, but they could be easily converted into queries that are requested after tcx is built. The main reason they are the way they are was to avoid a gratuitious refcell (but using the refcell map seems fine)...
but the result of moving `region_maps` out of `TyCtxt` and into a query caused a lot of churn, and seems like it could potentially result in a rather large performance hit, since it means a dep-graph lookup on every use of `region_maps` (rather than just a field access). Possibly `TyCtxt` could store a `RefCell<Option<RegionMap>>` internally and use that to prevent repeat lookups, but that feels like it's duplicating the work of the dep-graph. @nikomatsakis What did you have in mind for this?
Remove internal liblog
This commit deletes the internal liblog in favor of the implementation that
lives on crates.io. Similarly it's also setting a convention for adding crates
to the compiler. The main restriction right now is that we want compiler
implementation details to be unreachable from normal Rust code (e.g. requires a
feature), and by default everything in the sysroot is reachable via `extern
crate`.
The proposal here is to require that crates pulled in have these lines in their
`src/lib.rs`:
#![cfg_attr(rustbuild, feature(staged_api, rustc_private))]
#![cfg_attr(rustbuild, unstable(feature = "rustc_private", issue = "27812"))]
This'll mean that by default they're not using these attributes but when
compiled as part of the compiler they do a few things:
* Mark themselves as entirely unstable via the `staged_api` feature and the
`#![unstable]` attribute.
* Allow usage of other unstable crates via `feature(rustc_private)` which is
required if the crate relies on any other crates to compile (other than std).
This commit deletes the internal liblog in favor of the implementation that
lives on crates.io. Similarly it's also setting a convention for adding crates
to the compiler. The main restriction right now is that we want compiler
implementation details to be unreachable from normal Rust code (e.g. requires a
feature), and by default everything in the sysroot is reachable via `extern
crate`.
The proposal here is to require that crates pulled in have these lines in their
`src/lib.rs`:
#![cfg_attr(rustbuild, feature(staged_api, rustc_private))]
#![cfg_attr(rustbuild, unstable(feature = "rustc_private", issue = "27812"))]
This'll mean that by default they're not using these attributes but when
compiled as part of the compiler they do a few things:
* Mark themselves as entirely unstable via the `staged_api` feature and the
`#![unstable]` attribute.
* Allow usage of other unstable crates via `feature(rustc_private)` which is
required if the crate relies on any other crates to compile (other than std).
On-demandify associated item retrieval
Part of #40614.
I also started converting `adt_def`, but I decided to open a PR with just this bit first to make sure I'm going about this correctly.
r? @nikomatsakis
Some preparations for directly computing the ICH of crate-metadata.
This PR contains some small fixes in preparation for direct metadata hashing. It mostly just moves stuff into places where it will be needed (making the module structure slightly cleaner along the way) and it fixes some omissions in the MIR region eraser.
r? @nikomatsakis
Marco Castelluccio
bors
Michael Woerister
est31
Alex Crichton
Niko Matsakis
whitequark
Taylor Cramer
Ariel Ben-Yehuda
Ariel Ben-Yehuda
Eduard-Mihai Burtescu
Tim Neumann
Corey Farwell
Simonas Kazlauskas
Jeffrey Seyfried
Nick Cameron