Prevent query cycles in the MIR inliner
r? `@eddyb` `@wesleywiser`
cc `@rust-lang/wg-mir-opt`
The general design is that we have a new query that is run on the `validated_mir` instead of on the `optimized_mir`. That query is forced before going into the optimization pipeline, so as to not try to read from a stolen MIR.
The query should not be cached cross crate, as you should never call it for items from other crates. By its very design calls into other crates can never cause query cycles.
This is a pessimistic approach to inlining, since we strictly have more calls in the `validated_mir` than we have in `optimized_mir`, but that's not a problem imo.
Various ABI refactorings
This includes changes to the rust abi and various refactorings that will hopefully make it easier to use the abi handling infrastructure of rustc in cg_clif. There are several refactorings that I haven't done. I am opening this draft PR to check that I haven't broken any non x86_64 architectures.
r? `@ghost`
Due to macro expansion, its possible to end up with two distinct
`ExpnId`s that have the same `ExpnData` contents. This violates the
contract of `HashStable`, since two unequal `ExpnId`s will end up with
equal `Fingerprint`s.
This commit adds a `disambiguator` field to `ExpnData`, which is used to
force two otherwise-equivalent `ExpnData`s to be distinct.
Abi::ScalarPair is only ever used for types that don't have a stable
layout anyway so this doesn't break any FFI. It does however reduce the
amount of special casing on the abi outside of the code responsible for
abi specific adjustments to the pass mode.
Improve diagnostics when closure doesn't meet trait bound
Improves the diagnostics when closure doesn't meet trait bound by modifying `TypeckResuts::closure_kind_origins` such that `hir::Place` is used instead of `Symbol`. Using `hir::Place` to describe which capture influenced the decision of selecting a trait a closure satisfies to (Fn/FnMut/FnOnce, Copy) allows us to show precise path in the diagnostics when `capture_disjoint_field` feature is enabled.
Closes rust-lang/project-rfc-2229/issues/21
r? ```@nikomatsakis```
Remove DepKind::CrateMetadata and pre-allocation of DepNodes
Remove much of the special-case handling around crate metadata
dependency tracking by replacing `DepKind::CrateMetadata` and the
pre-allocation of corresponding `DepNodes` with on-demand invocation
of the `crate_hash` query.
Properly handle `SyntaxContext` of dummy spans in incr comp
Fixes#80336
Due to macro expansion, we may end up with spans with an invalid
location and non-root `SyntaxContext`. This commits preserves the
`SyntaxContext` of such spans in the incremental cache, and ensures
that we always hash the `SyntaxContext` when computing the `Fingerprint`
of a `Span`
Previously, we would discard the `SyntaxContext` during serialization to
the incremental cache, causing the span's `Fingerprint` to change across
compilation sessions.
Rework diagnostics for wrong number of generic args (fixes#66228 and #71924)
This PR reworks the `wrong number of {} arguments` message, so that it provides more details and contextual hints.
Fixes#80336
Due to macro expansion, we may end up with spans with an invalid
location and non-root `SyntaxContext`. This commits preserves the
`SyntaxContext` of such spans in the incremental cache, and ensures
that we always hash the `SyntaxContext` when computing the `Fingerprint`
of a `Span`
Previously, we would discard the `SyntaxContext` during serialization to
the incremental cache, causing the span's `Fingerprint` to change across
compilation sessions.
Separate out a `hir::Impl` struct
This makes it possible to pass the `Impl` directly to functions, instead
of having to pass each of the many fields one at a time. It also
simplifies matches in many cases.
See `rustc_save_analysis::dump_visitor::process_impl` or `rustdoc::clean::clean_impl` for a good example of how this makes `impl`s easier to work with.
r? `@petrochenkov` maybe?
This makes it possible to pass the `Impl` directly to functions, instead
of having to pass each of the many fields one at a time. It also
simplifies matches in many cases.
Turn type inhabitedness into a query to fix `exhaustive_patterns` perf
We measured in https://github.com/rust-lang/rust/pull/79394 that enabling the [`exhaustive_patterns` feature](https://github.com/rust-lang/rust/issues/51085) causes significant perf degradation. It was conjectured that the culprit is type inhabitedness checking, and [I hypothesized](https://github.com/rust-lang/rust/pull/79394#issuecomment-733861149) that turning this computation into a query would solve most of the problem.
This PR turns `tcx.is_ty_uninhabited_from` into a query, and I measured a 25% perf gain on the benchmark that stress-tests `exhaustiveness_patterns`. This more than compensates for the 30% perf hit I measured [when creating it](https://github.com/rust-lang/rustc-perf/pull/801). We'll have to measure enabling the feature again, but I suspect this fixes the perf regression entirely.
I'd like a perf run on this PR obviously.
I made small atomic commits to help reviewing. The first one is just me discovering the "revisions" feature of the testing framework.
I believe there's a push to move things out of `rustc_middle` because it's huge. I guess `inhabitedness/mod.rs` could be moved out, but it's quite small. `DefIdForest` might be movable somewhere too. I don't know what the policy is for that.
Ping `@camelid` since you were interested in following along
`@rustbot` modify labels: +A-exhaustiveness-checking
Since `DefIdForest` contains 0 or 1 elements the large majority of the
time, by allocating only in the >1 case we avoid almost all allocations,
compared to `Arc<SmallVec<[DefId;1]>>`. This shaves off 0.2% on the
benchmark that stresses uninhabitedness checking.
Remove much of the special-case handling around crate metadata
dependency tracking by replacing `DepKind::CrateMetadata` and the
pre-allocation of corresponding `DepNodes` with on-demand invocation
of the `crate_hash` query.
Make CTFE able to check for UB...
... by not doing any optimizations on the `const fn` MIR used in CTFE. This means we duplicate all `const fn`'s MIR now, once for CTFE, once for runtime. This PR is for checking the perf effect, so we have some data when talking about https://github.com/rust-lang/const-eval/blob/master/rfcs/0000-const-ub.md
To do this, we now have two queries for obtaining mir: `optimized_mir` and `mir_for_ctfe`. It is now illegal to invoke `optimized_mir` to obtain the MIR of a const/static item's initializer, an array length, an inline const expression or an enum discriminant initializer. For `const fn`, both `optimized_mir` and `mir_for_ctfe` work, the former returning the MIR that LLVM should use if the function is called at runtime. Similarly it is illegal to invoke `mir_for_ctfe` on regular functions.
This is all checked via appropriate assertions and I don't think it is easy to get wrong, as there should be no `mir_for_ctfe` calls outside the const evaluator or metadata encoding. Almost all rustc devs should keep using `optimized_mir` (or `instance_mir` for that matter).
Serialize incr comp structures to file via fixed-size buffer
Reduce a large memory spike that happens during serialization by writing
the incr comp structures to file by way of a fixed-size buffer, rather
than an unbounded vector.
Effort was made to keep the instruction count close to that of the
previous implementation. However, buffered writing to a file inherently
has more overhead than writing to a vector, because each write may
result in a handleable error. To reduce this overhead, arrangements are
made so that each LEB128-encoded integer can be written to the buffer
with only one capacity and error check. Higher-level optimizations in
which entire composite structures can be written with one capacity and
error check are possible, but would require much more work.
The performance is mostly on par with the previous implementation, with
small to moderate instruction count regressions. The memory reduction is
significant, however, so it seems like a worth-while trade-off.
Reduce a large memory spike that happens during serialization by writing
the incr comp structures to file by way of a fixed-size buffer, rather
than an unbounded vector.
Effort was made to keep the instruction count close to that of the
previous implementation. However, buffered writing to a file inherently
has more overhead than writing to a vector, because each write may
result in a handleable error. To reduce this overhead, arrangements are
made so that each LEB128-encoded integer can be written to the buffer
with only one capacity and error check. Higher-level optimizations in
which entire composite structures can be written with one capacity and
error check are possible, but would require much more work.
The performance is mostly on par with the previous implementation, with
small to moderate instruction count regressions. The memory reduction is
significant, however, so it seems like a worth-while trade-off.
Access query (DepKind) metadata through fields
This refactors the access to query definition metadata (attributes such as eval always, anon, has_params) and loading/forcing functions to generate a number of structs, instead of matching on the DepKind enum. This makes access to the fields cheaper to compile. Using a struct means that finding the metadata for a given query is just an offset away; previously the match may have been compiled to a jump table but likely not completely inlined as we expect here.
A previous attempt explored a similar strategy, but using trait objects in #78314 that proved less effective, likely due to higher overheads due to forcing dynamic calls and poorer cache utilization (all metadata is fairly densely packed with this PR).
Add check for `[T;N]`/`usize` mismatch in astconv
Helps clarify the issue in #80506
by adding a specific check for mismatches between [T;N] and usize.
r? `@lcnr`
use PlaceRef more consistently instead of loosely coupled local+projection
Instead of working directly with the `projections` array, use `iter_projections` and `last_projection`. This avoids having to construct new `PlaceRef` from the pieces everywhere.
I only did this for a few files, to see how people think about this. If y'all are happy with this, I'll open an E-mentor issue to complete this. I grepped for `Place::ty_from` to find the places that need adjusting -- this could miss some, but I am not sure what else to grep for.
Add `#[track_caller]` to `bug!` and `register_renamed`
Before:
```
thread 'rustc' panicked at 'compiler/rustc_lint/src/context.rs:267:18: invalid lint renaming of broken_intra_doc_links to rustdoc::broken_intra_doc_links', compiler/rustc_middle/src/util/bug.rs:34:26
```
After:
```
thread 'rustc' panicked at 'src/librustdoc/core.rs:455:24: invalid lint renaming of broken_intra_doc_links to rustdoc::broken_intra_doc_links', compiler/rustc_middle/src/util/bug.rs:35:26
```
The reason I added it to `register_renamed` too is that any panic in
that function will be the caller's fault.
Before:
```
thread 'rustc' panicked at 'compiler/rustc_lint/src/context.rs:267:18: invalid lint renaming of broken_intra_doc_links to rustdoc::broken_intra_doc_links', compiler/rustc_middle/src/util/bug.rs:34:26
```
After:
```
thread 'rustc' panicked at 'src/librustdoc/core.rs:455:24: invalid lint renaming of broken_intra_doc_links to rustdoc::broken_intra_doc_links', compiler/rustc_middle/src/util/bug.rs:35:26
```
The reason I added it to `register_renamed` too is that any panic in
that function will be the caller's fault.
Document `InferTy` & co.
I finally figured out what `TyVid` means! The name is quite opaque, so I
decided to document it and related types.
I don't know that much about `InferTy` & co., but I was able to *infer*
( :) ) from the names and what I know generally about type inference to
add some basic documentation.
Rollup of 11 pull requests
Successful merges:
- #79662 (Move some more code out of CodegenBackend::{codegen_crate,link})
- #79815 (Update RELEASES.md for 1.49.0)
- #80284 (Suggest fn ptr rather than fn item and suggest to use `Fn` trait bounds rather than the unique closure type in E0121)
- #80331 (Add more comments to trait queries)
- #80344 (use matches!() macro in more places)
- #80353 (BTreeMap: test split_off (and append) more thoroughly)
- #80362 (Document rustc_macros on nightly-rustc)
- #80399 (Remove FIXME in rustc_privacy)
- #80408 (Sync rustc_codegen_cranelift)
- #80411 (rustc_span: Remove `Symbol::with`)
- #80434 (bootstrap: put the component name in the tarball temp dir path)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Rename rustc_middle::lint::LintSource
Rename [`rustc_middle::lint::LintSource`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/lint/enum.LintSource.html) to `rustc_middle::lint::LintLevelSource`.
This enum represents the source of a *lint level*, not a lint. This should improve code readability.
Update: Also documents `rustc_middle::lint::LevelSource` to clarify.
Add module-level docs to rustc_middle::ty
I thought it would be nice to point out `Ty` and `TyCtxt` on the module page, and link out to the [rustc-dev-guide chapter](https://rustc-dev-guide.rust-lang.org/ty.html).
docs: Edit rustc_middle::ty::query::on_disk_cache
Expand abbreviations for "incremental compliation".
Also added the word "to" to the description of CacheEncoder.
rustc_query_system: explicitly register reused dep nodes
Register nodes that we've reused from the previous session explicitly
with `OnDiskCache`. Previously, we relied on this happening as a side
effect of accessing the nodes in the `PreviousDepGraph`. For the sake of
performance and avoiding unintended side effects, register explictily.
Edit rustc_middle docs
Re-word doc comment for rustc_middle::hir::place::Projection.
Also adds:
- Missing end stop punctuation, and
- Documentation links to `rustc_middle::mir::Place`.
Re-word doc comment for rustc_middle::hir::place::Projection.
Also adds:
- Missing end stop punctuation, and
- Documentation links to `rustc_middle::mir::Place`.
Make BoundRegion have a kind of BoungRegionKind
Split from #76814
Also includes making `replace_escaping_bound_vars` only return `T`
Going to r? `@lcnr`
Feel free to reassign
const_evaluatable_checked: fix occurs check
fixes#79615
this is kind of a hack because we use `TypeRelation` for both the `Generalizer` and the `ConstInferUnifier` but i am not sure if there is a useful way to disentangle this without unnecessarily duplicating some code.
The error in the added test is kind of unavoidable until we erase the unused substs of `ConstKind::Unevaluated`. We talked a bit about this in the cg lazy norm meeting (https://rust-lang.zulipchat.com/#narrow/stream/260443-project-const-generics/topic/lazy_normalization_consts)
Register nodes that we've reused from the previous session explicitly
with `OnDiskCache`. Previously, we relied on this happening as a side
effect of accessing the nodes in the `PreviousDepGraph`. For the sake of
performance and avoiding unintended side effects, register explictily.
Move binder for dyn to each list item
This essentially changes `ty::Binder<&'tcx List<ExistentialTraitRef>>` to `&'tcx List<ty::Binder<ExistentialTraitRef>>`.
This is a first step in moving the `dyn Trait` representation closer to Chalk, which we've talked about in `@rust-lang/wg-traits.`
r? `@nikomatsakis`
consider assignments of union field of ManuallyDrop type safe
Assigning to `Copy` union fields is safe because that assignment will never drop anything. However, with https://github.com/rust-lang/rust/pull/77547, unions may also have `ManuallyDrop` fields, and their assignments are currently still unsafe. That seems unnecessary though, as assigning `ManuallyDrop` does not drop anything either, and is thus safe even for union fields.
I assume this will at least require FCP.
[mir-opt] Allow debuginfo to be generated for a constant or a Place
Prior to this commit, debuginfo was always generated by mapping a name
to a Place. This has the side-effect that `SimplifyLocals` cannot remove
locals that are only used for debuginfo because their other uses have
been const-propagated.
To allow these locals to be removed, we now allow debuginfo to point to
a constant value. The `ConstProp` pass detects when debuginfo points to
a local with a known constant value and replaces it with the value. This
allows the later `SimplifyLocals` pass to remove the local.
Create `rustc_type_ir`
Decided to start small 😄
This PR creates a `rustc_type_ir` crate as part of the WG-Traits plan to create a shared type library.
~~There already exists a `rustc_ty` crate, so I named the new crate `rustc_ty_library`. However I think it would make sense to rename the current `rustc_ty` to something else (e.g. `rustc_ty_passes`) to free the name for this new crate.~~
r? `@jackh726`
Capture precise paths in THIR and MIR
This PR allows THIR and MIR to use the result of the new capture analysis to actually capture precise paths
To achieve we:
- Writeback min capture results to TypeckResults
- Move handling upvars to PlaceBuilder in mir_build
- Lower precise paths in THIR build by reading min_captures
- Search for ancestors in min_capture when trying to build a MIR place which starts off of an upvar
Closes: https://github.com/rust-lang/project-rfc-2229/issues/10
Partly implements: rust-lang/project-rfc-2229#18
Work that remains (not in this PR):
- [ ] [Known bugs when feature gate is enabled](https://github.com/rust-lang/project-rfc-2229/projects/1?card_filter_query=label%3Abug)
- [ ] Use min_capure_map for
- [ ] Liveness analysis
- [ ] rustc_mir/interpret/validity.rs
- [ ] regionck
- [ ] rust-lang/project-rfc-2229#8
- [ ] remove closure_captures and upvar_capture_map
r? `@ghost`
Fixes#79890
Previously, we just copied a `RawDefId` from the 'old' map to the 'new'
map. However, the `RawDefId` for a given `DefPathHash` may be different
in the current compilation session. Using `def_path_hash_to_def_id`
ensures that the `RawDefId` we use is valid in the current session.
- Use closure_min_capture maps to capture precise paths
- PlaceBuilder now searches for ancestors in min_capture list
- Add API to `Ty` to allow access to the n-th element in a
tuple in O(1) time.
Co-authored-by: Roxane Fruytier <roxane.fruytier@hotmail.com>
Properly re-use def path hash in incremental mode
Fixes#79661
In incremental compilation mode, we update a `DefPathHash -> DefId`
mapping every time we create a `DepNode` for a foreign `DefId`.
This mapping is written out to the on-disk incremental cache, and is
read by the next compilation session to allow us to lazily decode
`DefId`s.
When we decode a `DepNode` from the current incremental cache, we need
to ensure that any previously-recorded `DefPathHash -> DefId` mapping
gets recorded in the new mapping that we write out. However, PR #74967
didn't do this in all cases, leading to us being unable to decode a
`DefPathHash` in certain circumstances.
This PR refactors some of the code around `DepNode` deserialization to
prevent this kind of mistake from happening again.
Prior to this commit, debuginfo was always generated by mapping a name
to a Place. This has the side-effect that `SimplifyLocals` cannot remove
locals that are only used for debuginfo because their other uses have
been const-propagated.
To allow these locals to be removed, we now allow debuginfo to point to
a constant value. The `ConstProp` pass detects when debuginfo points to
a local with a known constant value and replaces it with the value. This
allows the later `SimplifyLocals` pass to remove the local.
- Derive TypeFoldable on `hir::place::Place` and associated
structs, to them to be written into typeck results.
Co-authored-by: Jennifer Wills <wills.jenniferg@gmail.com>
Co-authored-by: Logan Mosier <logmosier@gmail.com>
extend `WithOptConstParam` docs, move rustdoc test
This should hopefully make things a bit clearer, feel free to comment on anything which can still be improved.
cc `@ecstatic-morse` `@nikomatsakis` `@RalfJung`
check the recursion limit when finding a struct's tail
fixes#79437
This does a `delay_span_bug` (via `ty_error_with_message`) rather than emit a new error message, under the assumption that there will be an error elsewhere (even if the type isn't infinitely recursive, just deeper than the recursion limit, this appears to be the case).
Fixes#79661
In incremental compilation mode, we update a `DefPathHash -> DefId`
mapping every time we create a `DepNode` for a foreign `DefId`.
This mapping is written out to the on-disk incremental cache, and is
read by the next compilation session to allow us to lazily decode
`DefId`s.
When we decode a `DepNode` from the current incremental cache, we need
to ensure that any previously-recorded `DefPathHash -> DefId` mapping
gets recorded in the new mapping that we write out. However, PR #74967
didn't do this in all cases, leading to us being unable to decode a
`DefPathHash` in certain circumstances.
This PR refactors some of the code around `DepNode` deserialization to
prevent this kind of mistake from happening again.
bjorn3
bors
Caleb Zulawski
Aaron Hill
Jonas Schievink
oli
Camille GILLOT
pierwill
Ömer Sinan Ağacan
LeSeulArtichaut
Tomasz Miąsko
Ashley Mannix
Jack Huey
Jack Huey
Mara Bos
Bastian Kauschke
Chris Pardy
Yuki Okushi
LingMan
Jacob Pratt
Dylan DPC
Joshua Nelson
Nadrieril
Tyson Nottingham
Santiago Pastorino
Patryk Wychowaniec
Caio
oli
kadmin
Ralf Jung
Matthias Krüger
Roxane
Rémy Rakic
Camelid
pierwill
PankajChaudhary5
Guillaume Gomez
Dániel Buga
Wesley Wiser
bstrie
Aman Arora
Tunahan Karlibas
SNCPlay42