Commit Graph

63 Commits

Author SHA1 Message Date
bors 2a55274e0c Auto merge of #82999 - cuviper:rustc-rayon-0.3.1, r=Mark-Simulacrum
Update to rustc-rayon 0.3.1

This pulls in rust-lang/rustc-rayon#8 to fix #81425. (h/t `@ammaraskar)`

That revealed weak constraints on `rustc_arena::DropArena`, because its
`DropType` was holding type-erased raw pointers to generic `T`. We can
implement `Send` for `DropType` (under `cfg(parallel_compiler)`) by
requiring all `T: Send` before they're type-erased.
2021-03-15 08:49:25 +00:00
bors e7e1dc158c Auto merge of #83007 - Aaron1011:incr-verify-default, r=Mark-Simulacrum
Turn `-Z incremental-verify-ich` on by default

Issue #82920 showed that the kind of bugs caught by this flag have
soundness implications.
2021-03-13 17:52:22 +00:00
Aaron Hill 7d7c81a114
Always run `incremental_verify_ich` when re-computing query results
Issue #82920 showed that the kind of bugs caught by this flag have
soundness implications.

This causes performance regressions of up to 15.2% during incremental
compilation, but this is necessary to catch miscompilations caused by
bugs in query implementations.
2021-03-13 12:00:38 -05:00
Tyson Nottingham adcbe49b16 rustc_query_system: simplify QueryCache::iter
Minor cleanup to reduce a small amount of complexity and code bloat.
Reduces the number of mono items in rustc_query_impl by 15%.
2021-03-12 17:34:14 -08:00
Josh Stone f7e75a2124 Update to rustc-rayon 0.3.1
This pulls in rust-lang/rustc-rayon#8 to fix #81425. (h/t @ammaraskar)

That revealed weak constraints on `rustc_arena::DropArena`, because its
`DropType` was holding type-erased raw pointers to generic `T`. We can
implement `Send` for `DropType` (under `cfg(parallel_compiler)`) by
requiring all `T: Send` before they're type-erased.
2021-03-10 17:53:35 -08:00
Camille GILLOT 903f65f215 Simplify hashing. 2021-02-21 12:22:22 +01:00
Camille GILLOT a87de890fd Move print_query_stack to rustc_query_system. 2021-02-20 23:40:56 +01:00
Camille GILLOT c26d965714 Move report_cycle to rustc_query_system.
The call to `ty::print::with_forced_impl_filename_line`
is done when constructing the description,
at the construction of the QueryStackFrame.
2021-02-20 23:36:31 +01:00
Camille GILLOT 3897395787 Move Query to rustc_query_system.
Rename it to QueryStackFrame and document a bit.
2021-02-20 22:53:47 +01:00
Camille GILLOT 0144d6a3b7 Do not hold query key in Query. 2021-02-20 22:53:46 +01:00
Camille GILLOT f96e960ccf Access the session directly from DepContext. 2021-02-20 22:53:46 +01:00
Camille GILLOT b27266fdb2 Use a QueryContext for try_mark_green. 2021-02-19 17:51:56 +01:00
Camille GILLOT ea3d465c95 Move try_load_from_on_disk_cache to the QueryContext. 2021-02-19 17:51:55 +01:00
Camille GILLOT 49c1b07a9e Decouple QueryContext from DepContext. 2021-02-19 17:51:49 +01:00
Camille GILLOT 6f04883023 Remove QueryAccessors::to_dep_node. 2021-02-19 17:51:49 +01:00
Camille GILLOT 211b05aef3 Don't require a QueryContext to access the DepGraph. 2021-02-19 17:51:49 +01:00
Eduard-Mihai Burtescu 6165d1cc72 Print -Ztime-passes (and misc stats/logs) on stderr, not stdout. 2021-02-18 14:13:38 +02:00
Tomasz Miąsko db36db2e81 Inline try_get_cached 2021-02-16 00:00:00 +00:00
bors d1206f950f Auto merge of #81855 - cjgillot:ensure-cache, r=oli-obk
Check the result cache before the DepGraph when ensuring queries

Split out of https://github.com/rust-lang/rust/pull/70951

Calling `ensure` on already forced queries is a common operation.
Looking at the results cache first is faster than checking the DepGraph for a green node.
2021-02-15 12:11:59 +00:00
klensy 93c8ebe022 bumped smallvec deps 2021-02-14 18:03:11 +03:00
Camille GILLOT 3fc8ed68e9 Check query cache before calling into the query engine. 2021-02-13 21:14:58 +01:00
Camille GILLOT 280a2866d5 Drop the cache lock earlier. 2021-02-13 21:14:58 +01:00
Camille GILLOT 15b0bc6b83 Separate the query cache from the query state. 2021-02-13 21:14:58 +01:00
Camille GILLOT 9f46259a75 Return a Result for query cache. 2021-02-13 21:14:58 +01:00
Camille GILLOT 8684e9e47d Merge {get,ensure}_query. 2021-02-13 21:14:57 +01:00
bors 097bc6a84f Auto merge of #81892 - jyn514:no-inline, r=cjgillot
[experiment] remove `#[inline]` from rustc_query_system::plumbing

These functions have a ton of generic parameters and are instantiated
over and over again. Hopefully this will reduce binary bloat and speed
up bootstrapping times.

r? `@cjgillot`
2021-02-09 18:37:33 +00:00
Mark Rousskov f564d7abba Switch query descriptions to just String
In practice we never used the borrowed variant anyway.
2021-02-08 17:20:41 -05:00
Joshua Nelson 4f77a1afc2 [experiment] remove `#[inline]` from rustc_query_system::plumbing
These functions have a ton of generic parameters and are instantiated
over and over again. Hopefully this will reduce binary bloat and speed
up bootstrapping times.
2021-02-08 14:57:15 -05:00
bors a8f7075532 Auto merge of #80692 - Aaron1011:feature/query-result-debug, r=estebank
Enforce that query results implement Debug

Currently, we require that query keys implement `Debug`, but we do not do the same for query values. This can make incremental compilation bugs difficult to debug - there isn't a good place to print out the result loaded from disk.

This PR adds `Debug` bounds to several query-related functions, allowing us to debug-print the query value when an 'unstable fingerprint' error occurs. This required adding `#[derive(Debug)]` to a fairly large number of types - hopefully, this doesn't have much of an impact on compiler bootstrapping times.
2021-01-26 05:47:23 +00:00
bors c5a96fb797 Auto merge of #80957 - tgnottingham:direct_serialize_depgraph, r=michaelwoerister
Serialize dependency graph directly from DepGraph

Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-19 19:36:41 +00:00
Aaron Hill 056fbbf7ee
Undo assertion change 2021-01-16 21:37:53 -05:00
Aaron Hill 6417760632
Run fmt 2021-01-16 17:53:02 -05:00
Aaron Hill 93ab705655
Print result on unstable fingerprint error 2021-01-16 17:53:02 -05:00
Aaron Hill 7afb32557d
Enforce that query results implement Debug 2021-01-16 17:53:02 -05:00
LingMan a56bffb4f9 Use Option::map_or instead of `.map(..).unwrap_or(..)` 2021-01-14 19:23:59 +01:00
Tyson Nottingham 09067db8a0 Serialize dependency graph directly from DepGraph
Reduce memory usage by serializing dep graph directly from `DepGraph`,
rather than copying it into `SerializedDepGraph` and serializing that.
2021-01-12 22:20:29 -08:00
Joshua Nelson 0215b3a456 Don't mark `force_query_with_job` as `inline(always)`
It's rather large, and using `inline(always)` forces it to be recompiled
in each calling crate.
2021-01-08 18:38:33 -05:00
Camille GILLOT 016ea6b319 Use a side-table of consts instead of matching on the DepKind enum. 2021-01-08 17:48:02 +01:00
Camille GILLOT d1220fdedf Simplify DepNodeParams. 2021-01-08 17:29:49 +01:00
Tyson Nottingham 03eb75f759 rustc_query_system: avoid race condition when using edge_count 2020-12-22 14:12:57 -08:00
Tyson Nottingham 22ed75158b rustc_query_system: add more comments for dependency graph 2020-12-22 14:12:57 -08:00
Tyson Nottingham d6b2aaed7d rustc_query_system: rename intern_node to intern_new_node 2020-12-22 14:12:57 -08:00
Tyson Nottingham 712fcae13a rustc_query_system: remove inline annotation from edge_count
This isn't called frequently enough to justify inlining.
2020-12-22 14:12:57 -08:00
Tyson Nottingham 4f76266295 rustc_query_system: minor cleanup
Remove effectively unused parameter and delete out of date comment.
2020-12-22 14:12:57 -08:00
Tyson Nottingham dd1ab840d2 rustc_query_system: use more space-efficient edges representation
Use single vector of edges rather than per-node vector. There is a small
hit to instruction counts (< 0.5%), but the memory savings make up for it.
2020-12-22 14:12:57 -08:00
Tyson Nottingham ea47269f5f rustc_query_system: share previous graph edges with current graph
Reduce memory consumption by sharing the previous dependency graph's
edges with the current graph when it is known to be valid to do so. It
is known to be valid whenever we mark a node green because all of its
dependencies were green. It is *not* known to be valid when we mark a
node green because we re-executed its query and its result was the same
as in the previous compilation session. In that case, the dependency set
might have changed (we don't try to determine whether or not it changed
and whether or not we can share).
2020-12-22 14:12:57 -08:00
Tyson Nottingham f6d6b0c96d rustc_query_system: share previous graph data with current graph
Reduce memory consumption by taking advantage of red/green algorithm
properties to share the previous dependency graph's node data with the
current graph instead of storing node data redundantly. Red nodes can
share the `DepNode`, and green nodes can share the `DepNode` and
`Fingerprint`. Edges will be shared when possible in a later change.
2020-12-22 14:12:57 -08:00
Tyson Nottingham 7795801902 rustc_query_system: explicitly register reused dep nodes
Register nodes that we've reused from the previous session explicitly
with `OnDiskCache`. Previously, we relied on this happening as a side
effect of accessing the nodes in the `PreviousDepGraph`. For the sake of
performance and avoiding unintended side effects, register explictily.
2020-12-18 18:53:12 -08:00
Aaron Hill 3918b82993
Use `def_path_hash_to_def_id` when re-using a `RawDefId`
Fixes #79890

Previously, we just copied a `RawDefId` from the 'old' map to the 'new'
map. However, the `RawDefId` for a given `DefPathHash` may be different
in the current compilation session. Using `def_path_hash_to_def_id`
ensures that the `RawDefId` we use is valid in the current session.
2020-12-10 16:04:19 -05:00
Aaron Hill c2946402ff
Properly re-use def path hash in incremental mode
Fixes #79661

In incremental compilation mode, we update a `DefPathHash -> DefId`
mapping every time we create a `DepNode` for a foreign `DefId`.
This mapping is written out to the on-disk incremental cache, and is
read by the next compilation session to allow us to lazily decode
`DefId`s.

When we decode a `DepNode` from the current incremental cache, we need
to ensure that any previously-recorded `DefPathHash -> DefId` mapping
gets recorded in the new mapping that we write out. However, PR #74967
didn't do this in all cases, leading to us being unable to decode a
`DefPathHash` in certain circumstances.

This PR refactors some of the code around `DepNode` deserialization to
prevent this kind of mistake from happening again.
2020-12-04 22:16:40 -05:00