Link compiler test documentation to rustc-guide
Update the compiler test documentation to point to the relevant rustc-guide page.
This commit is contained in:
Tom Tromey
parent
a77dfcc79f
commit
8a76656435
@ -1,186 +1,4 @@
|
||||
# Compiler Test Documentation
|
||||
|
||||
In the Rust project, we use a special set of commands embedded in
|
||||
comments to test the Rust compiler. There are two groups of commands:
|
||||
|
||||
1. Header commands
|
||||
2. Error info commands
|
||||
|
||||
Both types of commands are inside comments, but header commands should
|
||||
be in a comment before any code.
|
||||
|
||||
## Summary of Error Info Commands
|
||||
|
||||
Error commands specify something about certain lines of the
|
||||
program. They tell the test what kind of error and what message you
|
||||
are expecting.
|
||||
|
||||
* `~`: Associates the following error level and message with the
|
||||
current line
|
||||
* `~|`: Associates the following error level and message with the same
|
||||
line as the previous comment
|
||||
* `~^`: Associates the following error level and message with the
|
||||
previous line. Each caret (`^`) that you add adds a line to this, so
|
||||
`~^^^^^^^` is seven lines up.
|
||||
|
||||
The error levels that you can have are:
|
||||
|
||||
1. `ERROR`
|
||||
2. `WARNING`
|
||||
3. `NOTE`
|
||||
4. `HELP` and `SUGGESTION`*
|
||||
|
||||
\* **Note**: `SUGGESTION` must follow immediately after `HELP`.
|
||||
|
||||
## Summary of Header Commands
|
||||
|
||||
Header commands specify something about the entire test file as a
|
||||
whole. They are normally put right after the copyright comment, e.g.:
|
||||
|
||||
```Rust
|
||||
// Copyright blah blah blah
|
||||
// except according to those terms.
|
||||
|
||||
// ignore-test This doesn't actually work
|
||||
```
|
||||
|
||||
### Ignoring tests
|
||||
|
||||
These are used to ignore the test in some situations, which means the test won't
|
||||
be compiled or run.
|
||||
|
||||
* `ignore-X` where `X` is a target detail or stage will ignore the test accordingly (see below)
|
||||
* `ignore-pretty` will not compile the pretty-printed test (this is done to test the pretty-printer, but might not always work)
|
||||
* `ignore-test` always ignores the test
|
||||
* `ignore-lldb` and `ignore-gdb` will skip a debuginfo test on that debugger.
|
||||
|
||||
`only-X` is the opposite. The test will run only when `X` matches.
|
||||
|
||||
Some examples of `X` in `ignore-X`:
|
||||
|
||||
* Architecture: `aarch64`, `arm`, `asmjs`, `mips`, `wasm32`, `x86_64`, `x86`, ...
|
||||
* OS: `android`, `emscripten`, `freebsd`, `ios`, `linux`, `macos`, `windows`, ...
|
||||
* Environment (fourth word of the target triple): `gnu`, `msvc`, `musl`.
|
||||
* Pointer width: `32bit`, `64bit`.
|
||||
* Stage: `stage0`, `stage1`, `stage2`.
|
||||
|
||||
### Other Header Commands
|
||||
|
||||
* `min-{gdb,lldb}-version`
|
||||
* `min-llvm-version`
|
||||
* `compile-pass` for UI tests, indicates that the test is supposed
|
||||
to compile, as opposed to the default where the test is supposed to error out.
|
||||
* `compile-flags` passes extra command-line args to the compiler,
|
||||
e.g. `compile-flags -g` which forces debuginfo to be enabled.
|
||||
* `should-fail` indicates that the test should fail; used for "meta testing",
|
||||
where we test the compiletest program itself to check that it will generate
|
||||
errors in appropriate scenarios. This header is ignored for pretty-printer tests.
|
||||
* `gate-test-X` where `X` is a feature marks the test as "gate test" for feature X.
|
||||
Such tests are supposed to ensure that the compiler errors when usage of a gated
|
||||
feature is attempted without the proper `#![feature(X)]` tag.
|
||||
Each unstable lang feature is required to have a gate test.
|
||||
|
||||
## Revisions
|
||||
|
||||
Certain classes of tests support "revisions" (as of the time of this
|
||||
writing, this includes run-pass, compile-fail, run-fail, and
|
||||
incremental, though incremental tests are somewhat
|
||||
different). Revisions allow a single test file to be used for multiple
|
||||
tests. This is done by adding a special header at the top of the file:
|
||||
|
||||
```
|
||||
// revisions: foo bar baz
|
||||
```
|
||||
|
||||
This will result in the test being compiled (and tested) three times,
|
||||
once with `--cfg foo`, once with `--cfg bar`, and once with `--cfg
|
||||
baz`. You can therefore use `#[cfg(foo)]` etc within the test to tweak
|
||||
each of these results.
|
||||
|
||||
You can also customize headers and expected error messages to a particular
|
||||
revision. To do this, add `[foo]` (or `bar`, `baz`, etc) after the `//`
|
||||
comment, like so:
|
||||
|
||||
```
|
||||
// A flag to pass in only for cfg `foo`:
|
||||
//[foo]compile-flags: -Z verbose
|
||||
|
||||
#[cfg(foo)]
|
||||
fn test_foo() {
|
||||
let x: usize = 32_u32; //[foo]~ ERROR mismatched types
|
||||
}
|
||||
```
|
||||
|
||||
Note that not all headers have meaning when customized to a revision.
|
||||
For example, the `ignore-test` header (and all "ignore" headers)
|
||||
currently only apply to the test as a whole, not to particular
|
||||
revisions. The only headers that are intended to really work when
|
||||
customized to a revision are error patterns and compiler flags.
|
||||
|
||||
## Guide to the UI Tests
|
||||
|
||||
The UI tests are intended to capture the compiler's complete output,
|
||||
so that we can test all aspects of the presentation. They work by
|
||||
compiling a file (e.g., `ui/hello_world/main.rs`), capturing the output,
|
||||
and then applying some normalization (see below). This normalized
|
||||
result is then compared against reference files named
|
||||
`ui/hello_world/main.stderr` and `ui/hello_world/main.stdout`. If either of
|
||||
those files doesn't exist, the output must be empty. If the test run
|
||||
fails, we will print out the current output, but it is also saved in
|
||||
`build/<target-triple>/test/ui/hello_world/main.stdout` (this path is
|
||||
printed as part of the test failure message), so you can run `diff` and
|
||||
so forth.
|
||||
|
||||
Normally, the test-runner checks that UI tests fail compilation. If you want
|
||||
to do a UI test for code that *compiles* (e.g. to test warnings, or if you
|
||||
have a collection of tests, only some of which error out), you can use the
|
||||
`// compile-pass` header command to have the test runner instead
|
||||
check that the test compiles successfully.
|
||||
|
||||
### Editing and updating the reference files
|
||||
|
||||
If you have changed the compiler's output intentionally, or you are
|
||||
making a new test, you can pass `--bless` to the command you used to
|
||||
run the tests. This will then copy over the files
|
||||
from the build directory and use them as the new reference.
|
||||
|
||||
### Normalization
|
||||
|
||||
The normalization applied is aimed at eliminating output difference
|
||||
between platforms, mainly about filenames:
|
||||
|
||||
- the test directory is replaced with `$DIR`
|
||||
- all backslashes (`\`) are converted to forward slashes (`/`) (for Windows)
|
||||
- all CR LF newlines are converted to LF
|
||||
|
||||
Sometimes these built-in normalizations are not enough. In such cases, you
|
||||
may provide custom normalization rules using the header commands, e.g.
|
||||
|
||||
```
|
||||
// normalize-stdout-test: "foo" -> "bar"
|
||||
// normalize-stderr-32bit: "fn\(\) \(32 bits\)" -> "fn\(\) \($$PTR bits\)"
|
||||
// normalize-stderr-64bit: "fn\(\) \(64 bits\)" -> "fn\(\) \($$PTR bits\)"
|
||||
```
|
||||
|
||||
This tells the test, on 32-bit platforms, whenever the compiler writes
|
||||
`fn() (32 bits)` to stderr, it should be normalized to read `fn() ($PTR bits)`
|
||||
instead. Similar for 64-bit. The replacement is performed by regexes using
|
||||
default regex flavor provided by `regex` crate.
|
||||
|
||||
The corresponding reference file will use the normalized output to test both
|
||||
32-bit and 64-bit platforms:
|
||||
|
||||
```
|
||||
...
|
||||
|
|
||||
= note: source type: fn() ($PTR bits)
|
||||
= note: target type: u16 (16 bits)
|
||||
...
|
||||
```
|
||||
|
||||
Please see `ui/transmute/main.rs` and `.stderr` for a concrete usage example.
|
||||
|
||||
Besides `normalize-stderr-32bit` and `-64bit`, one may use any target
|
||||
information or stage supported by `ignore-X` here as well (e.g.
|
||||
`normalize-stderr-windows` or simply `normalize-stderr-test` for unconditional
|
||||
replacement).
|
||||
Documentation the compiler testing framework has moved to
|
||||
[the rustc guide](https://rust-lang-nursery.github.io/rustc-guide/tests/intro.html).
|
||||
|
||||
Loading…
Reference in New Issue
Block a user