expand: Stop normalizing `NtIdent`s before passing them to built-in macros
Built-in macros should be able to deal with `NtIdents` in the input by themselves like any other parser code.
You can't imagine how bad mutable AST visitors are, *especially* if they are modifying tokens.
This is one step towards removing token visiting from the visitor infrastructure (https://github.com/rust-lang/rust/pull/77271 also works in this direction.)
Optimize `IntRange::from_pat`, then shrink `ParamEnv`
Resolves#77058.
r? `@Mark-Simulacrum`
cc `@vandenheuvel`
Looking at the output of `perf report` for #76244, the hot instructions seemed to be around the call to `pat_constructor` in `IntRange::from_pat`. I carried out an obvious optimization, but it actually made the instruction count higher (see #77075). However, it seems to have mitigated whatever was causing the pipeline stalls, so when combined with #76244, it's a net win.
As you can see below, the regression in #76244 seems to have originated from something measured by `stalled-cycles-backend`. I'll try to collect some finer-grained stats to see if I can isolate it. I wish I had a better idea of what was going on here. I'd like to prevent the regression from reappearing in the future due to small changes in unrelated code.
<details>
<summary>Current `master`:</summary>
```
Performance counter stats for 'cargo +baseline-stage1 check':
2,275.67 msec task-clock:u # 0.998 CPUs utilized
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
49,826 page-faults:u # 0.022 M/sec
5,117,221,678 cycles:u # 2.249 GHz
299,655,943 stalled-cycles-frontend:u # 5.86% frontend cycles idle
2,284,213,395 stalled-cycles-backend:u # 44.64% backend cycles idle
8,051,871,959 instructions:u # 1.57 insn per cycle
# 0.28 stalled cycles per insn
1,359,589,402 branches:u # 597.447 M/sec
7,359,347 branch-misses:u # 0.54% of all branches
2.281030026 seconds time elapsed
2.108197000 seconds user
0.164183000 seconds sys
```
</details>
<details>
<summary>Shrink `ParamEnv` without changing `IntRange::from_pat`:</summary>
```
Performance counter stats for 'cargo +perf-stage1 check':
2,751.79 msec task-clock:u # 0.996 CPUs utilized
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
50,103 page-faults:u # 0.018 M/sec
6,260,590,019 cycles:u # 2.275 GHz
317,355,920 stalled-cycles-frontend:u # 5.07% frontend cycles idle
3,397,743,582 stalled-cycles-backend:u # 54.27% backend cycles idle
8,276,224,367 instructions:u # 1.32 insn per cycle
# 0.41 stalled cycles per insn
1,370,453,386 branches:u # 498.023 M/sec
7,281,031 branch-misses:u # 0.53% of all branches
2.763265838 seconds time elapsed
2.544578000 seconds user
0.204548000 seconds sys
```
</details>
<details>
<summary>Shrink `ParamEnv` and change `IntRange::from_pat`: </summary>
```
Performance counter stats for 'cargo +perf-stage1 check':
2,295.57 msec task-clock:u # 0.996 CPUs utilized
0 context-switches:u # 0.000 K/sec
0 cpu-migrations:u # 0.000 K/sec
49,959 page-faults:u # 0.022 M/sec
5,151,407,066 cycles:u # 2.244 GHz
324,517,829 stalled-cycles-frontend:u # 6.30% frontend cycles idle
2,301,671,001 stalled-cycles-backend:u # 44.68% backend cycles idle
8,130,868,329 instructions:u # 1.58 insn per cycle
# 0.28 stalled cycles per insn
1,356,618,512 branches:u # 590.972 M/sec
7,323,800 branch-misses:u # 0.54% of all branches
2.304509653 seconds time elapsed
2.128090000 seconds user
0.163909000 seconds sys
```
</details>
Makes progress towards #43081
In PR #73084, we started recursively expanded nonterminals during the
pretty-print/reparse check, allowing them to be properly compared
against the reparsed tokenstream.
Unfortunately, the recursive logic in that PR only handles the case
where a nonterminal appears inside a `TokenTree::Delimited`. If a
nonterminal appears directly in the expanded tokens of another
nonterminal, the inner nonterminal will not be expanded.
This PR fixes the recursive expansion of nonterminals, ensuring that
they are expanded wherever they occur.
This changes the behavior from *not* building for host whenever an
explicit target is specified. I find this much less confusing.
You can still disable host steps by passing an explicit empty list for
host.
Fixes#76990.
Rollup of 7 pull requests
Successful merges:
- #76454 (UI to unit test for those using Cell/RefCell/UnsafeCell)
- #76474 (Add option to pass a custom codegen backend from a driver)
- #76711 (diag: improve closure/generic parameter mismatch)
- #77170 (Remove `#[rustc_allow_const_fn_ptr]` and add `#![feature(const_fn_fn_ptr_basics)]`)
- #77194 (Add doc alias for iterator fold)
- #77288 (fix building libstd for Miri on macOS)
- #77295 (Update unstable-book: Fix ABNF in inline assembly docs)
Failed merges:
r? `@ghost`
Add doc alias for iterator fold
fold is known in python and javascript as reduce,
not sure about inject but it was written in doc there.
This was my first confusion when coming into rust, I somehow cannot find where is reduce, sometimes I still forget that it is known as `fold`.
Remove `#[rustc_allow_const_fn_ptr]` and add `#![feature(const_fn_fn_ptr_basics)]`
`rustc_allow_const_fn_ptr` was a hack to work around the lack of an escape hatch for the "min `const fn`" checks in const-stable functions. Now that we have co-opted `allow_internal_unstable` for this purpose, we no longer need a bespoke attribute.
Now this functionality is gated under `const_fn_fn_ptr_basics` (how concise!), and `#[allow_internal_unstable(const_fn_fn_ptr_basics)]` replaces `#[rustc_allow_const_fn_ptr]`. `const_fn_fn_ptr_basics` allows function pointer types to appear in the arguments and locals of a `const fn` as well as function pointer casts to be performed inside a `const fn`. Both of these were allowed in constants and statics already. Notably, this does **not** allow users to invoke function pointers in a const context. Presumably, we will use a nicer name for that (`const_fn_ptr`?).
r? @oli-obk
diag: improve closure/generic parameter mismatch
Fixes#51154.
This PR improves the diagnostic when a type parameter is expected and a closure is found, noting that each closure has a distinct type and therefore could not always match the caller-chosen type of the parameter.
r? @estebank
Add option to pass a custom codegen backend from a driver
This allows the driver to pass information to the codegen backend. For example the headcrab debugger may in the future want to use cg_clif to JIT code to be injected in the debuggee. This would PR make it possible to tell cg_clif which symbol can be found at which address and to tell it to inject the JITed code into the right process.
This PR may also help with https://github.com/rust-lang/miri/pull/1540 by allowing miri to provide a codegen backend that only emits metadata and doesn't perform any codegen.
cc @nbaksalyar (headcrab)
cc @RalfJung (miri)
UI to unit test for those using Cell/RefCell/UnsafeCell
Helps with #76268.
I'm working on all files using `Cell` and moving them to unit tests when possible.
r? @matklad
[mir-opt] Introduce a new flag to enable experimental/unsound mir opts
This implements part of https://github.com/rust-lang/compiler-team/issues/319. The exact name of this flag was not decided as part of that MCP and some people expressed that it should include "unsound" in some way.
I've chosen to use `enable-experimental-unsound-mir-opts` as the name. While long, I don't think that matters too much as really it will only be used by some mir-opt tests. If you object or have a better name, please leave a comment!
r? `@oli-obk`
cc `@rust-lang/wg-mir-opt` `@RalfJung`
Previously, `config.config` was always hardcoded as `"config.toml"`.
I thought that it was being overridden with the actual value later, but
it turns out `flags.config` was being completely discarded. This keeps
`config.config` in sync with `flags.config`.
Add missing definitions required by the sparc-unknown-linux-gnu target
This PR adds a few missing definitions required by sparc-unknown-linux-target which were discovered during build tests.
Deduplicate and generalize some (de/)serializer impls
I noticed this while implementing #77227 and getting a "not implemented for [T; 16]" error. There's likely more things we can deduplicate in this file, but I didn't need any additional changes.
This fixes https://github.com/rust-lang/wg-cargo-std-aware/issues/53
Now users will be able to do:
```
cargo build -Zbuild-std=core -Zbuild-std-features=compiler-builtins-mem
```
and correctly get the Rust implemenations for `memcpy` and friends.
Signed-off-by: Joe Richey <joerichey@google.com>
LVI hardening tests
Mitigating the speculative execution LVI attack against SGX enclaves requires compiler changes (i.e., adding lfences). This pull requests adds various tests to check if this happens correctly.
Reopen standard file descriptors when they are missing on Unix
The syscalls returning a new file descriptors generally return lowest-numbered
file descriptor not currently opened, without any exceptions for those
corresponding to stdin, sdout, or stderr.
Previously when any of standard file descriptors has been closed before starting
the application, operations on std::io::{stderr,stdin,stdout} were likely to
either succeed while being performed on unrelated file descriptor, or fail with
EBADF which is silently ignored.
Avoid the issue by using /dev/null as a replacement when the standard file
descriptors are missing.
The implementation is based on the one found in musl. It was selected among a
few others on the basis of the lowest overhead in the case when all descriptors
are already present (measured on GNU/Linux).
Closes#57728.
Closes#46981.
Closes#60447.
Benefits:
* Makes applications robust in the absence of standard file descriptors.
* Upholds IntoRawFd / FromRawFd safety contract (which was broken previously).
Drawbacks:
* Additional syscall during startup.
* The standard descriptors might have been closed intentionally.
* Requires /dev/null.
Alternatives:
* Check if stdin, stdout, stderr are opened and provide no-op substitutes in std::io::{stdin,stdout,stderr} without reopening them directly.
* Leave the status quo, expect robust applications to reopen them manually.
