rustc: Forbid interpolated tokens in the HIR
Right now the HIR contains raw `syntax::ast::Attribute` structure but nowadays
these can contain arbitrary tokens. One variant of the `Token` enum is an
"interpolated" token which basically means to shove all the tokens for a
nonterminal in this position. A "nonterminal" in this case is roughly analagous
to a macro argument:
macro_rules! foo {
($a:expr) => {
// $a is a nonterminal as an expression
}
}
Currently nonterminals contain namely items and expressions, and this poses a
problem for incremental compilation! With incremental we want a stable hash of
all HIR items, but this means we may transitively need a stable hash *of the
entire AST*, which is certainly not stable w/ node ids and whatnot. Hence today
there's a "bug" where the "stable hash" of an AST is just the raw hash value of
the AST, and this only arises with interpolated nonterminals. The downside of
this approach, however, is that a bunch of errors get spewed out during
compilation about how this isn't a great idea.
This PR is focused at fixing these warnings, basically deleting them from the
compiler. The implementation here is to alter attributes as they're lowered from
the AST to HIR, expanding all nonterminals in-place as we see them. This code
for expanding a nonterminal to a token stream already exists for the
`proc_macro` crate, so we basically just reuse the same implementation there.
After this PR it's considered a bug to have an `Interpolated` token and hence
the stable hash implementation simply uses `bug!` in this location.
Closes#40946
hide internal types/traits from std docs via new #[doc(masked)] attribute
Fixes#43701 (hopefully for good this time)
This PR introduces a new parameter to the `#[doc]` attribute that rustdoc looks for on `extern crate` statements. When it sees `#[doc(masked)]` on such a statement, it hides traits and types from that crate from appearing in either the "Trait Implementations" section of many type pages, or the "Implementors" section of trait pages. This is then applied to the `libc`/`rand`/`compiler_builtins` imports in libstd to prevent those crates from creating broken links in the std docs.
Like in #43348, this also introduces a feature gate, `doc_masked`, that controls the use of this parameter.
To view the std docs generated with this change, head to https://tonberry.quietmisdreavus.net/std-43701/std/index.html.
In <https://github.com/rust-lang/rust/pull/42998>, we added an
uninstantiable type for the internal `UNICODE_VERSION` value,
`UnicodeVersion`, but it was not made public to the outside of the
crate, resulting in the value becoming less useful. Here we make the
type accessible from the outside.
Also add a run-pass test to make sure the type and value can be accessed
as intended.
Right now the HIR contains raw `syntax::ast::Attribute` structure but nowadays
these can contain arbitrary tokens. One variant of the `Token` enum is an
"interpolated" token which basically means to shove all the tokens for a
nonterminal in this position. A "nonterminal" in this case is roughly analagous
to a macro argument:
macro_rules! foo {
($a:expr) => {
// $a is a nonterminal as an expression
}
}
Currently nonterminals contain namely items and expressions, and this poses a
problem for incremental compilation! With incremental we want a stable hash of
all HIR items, but this means we may transitively need a stable hash *of the
entire AST*, which is certainly not stable w/ node ids and whatnot. Hence today
there's a "bug" where the "stable hash" of an AST is just the raw hash value of
the AST, and this only arises with interpolated nonterminals. The downside of
this approach, however, is that a bunch of errors get spewed out during
compilation about how this isn't a great idea.
This PR is focused at fixing these warnings, basically deleting them from the
compiler. The implementation here is to alter attributes as they're lowered from
the AST to HIR, expanding all nonterminals in-place as we see them. This code
for expanding a nonterminal to a token stream already exists for the
`proc_macro` crate, so we basically just reuse the same implementation there.
After this PR it's considered a bug to have an `Interpolated` token and hence
the stable hash implementation simply uses `bug!` in this location.
Closes#40946
rustbuild: with --no-fail-fast, report the specific commands that failed
I'm not sure this is the most elegant way of doing it, I'm still a bit of a rust noob. I tried `Vec<Command>` and keeping `Cell` instead of `RefCell` but couldn't fight my way past the borrow errors, this was the first arrangement that I could make work.
rfold is the reverse version of fold.
Fold allows iterators to implement a different (non-resumable) internal
iteration when it is more efficient than the external iteration
implemented through the next method. (Common examples are VecDeque and
.chain()).
Introduce rfold() so that the same customization is available for
reverse iteration. This is achieved by both adding the method, and by
having the Rev<I> adaptor connect Rev::rfold -> I::fold, Rev::fold -> I::rfold.
Adaptors are things that take iterators and adapt them into other
iterators. With this definition, fold is just a usual method, because it
doesn't normally make an iterator.
Replace str's transmute() calls with pointer casts
After the following conversation in #rust-lang:
```
[14:43:50] <Ixrec> TIL the implementation of from_utf_unchecked is literally just "mem::transmute(x)"
[14:43:59] <Ixrec> no wonder people keep saying transmute is overpowered
[15:15:30] <eddyb> Ixrec: it should be a pointer cast lol
[15:15:46] <eddyb> unless it doesn't let you
[16:50:34] <Ixrec> https://play.rust-lang.org/?gist=d1e6b629ad9ec1baf64ce261c63845e6&version=stable seems like it does let me
[16:52:35] <eddyb> Ixrec: yeah that's the preferred impl
[16:52:46] <eddyb> Ixrec: it just wasn't in 1.0
[16:52:50] <eddyb> IIRC
[16:53:00] <eddyb> (something something fat pointers)
```
Since I already wrote half of the preferred impls in the playground, might as well make an actual PR.
Document thread builder panics for nul bytes in thread names
This seems to have been undocumented. Mention this where the name is set
(Builder::name) and where the panic could happen (Builder::spawn).
Thread::new is private and I think the builder is the only user where
this matters. A short comment was added to "document" Thread::new too.
Add proper help line for `-C inline threshold`
Looks like someone accidentally some words when adding this.
This also remove a period on a different help line for consistency, as no options have a period.
Add Cow<str> -> Box<Error> impls.
Considering how impls exist for `String` and `&str`, it makes sense to also add an impl for `Cow<str>` as well.
This would allow converting `String::from_utf8_lossy` directly into a `Box<Error>` or `io::Error` without having to add an extra `into_ownd()`.
incr.comp.: Compute fingerprint for all query results.
This PR enables query result fingerprinting in incremental mode. This is an essential piece of infrastructure for red/green tracking. We don't do anything with the fingerprints yet but merging the infrastructure should protect it from bit-rotting and will make it easier to start measuring its performance impact (and thus let us determine if we should switch to a faster hashing algorithm rather sooner than later).
Note, this PR also includes the changes from https://github.com/rust-lang/rust/pull/43887 which I'm therefore closing. No need to re-review the first commit though.
r? @nikomatsakis
- Don't hash traits in scope as part of HIR hashing any more.
- Some queries returned DefIndexes from other crates.
- Provide a generic way of stably hashing maps (not used everywhere yet).
This makes sure that we don't introduce strange cases where we have
nodes outside the query system that could break red/green tracking
and it will allow to keep red/green neatly encapsulated within the
DepGraph implementation.
