Miscellaneous changes for CI, Docker and compiletest.
This PR contains 7 independent commits that improves interaction with CI, Docker and compiletest.
1. a4e5c91cb8 — Forces a newline every 100 dots when testing in quiet mode. Prevents spurious timeouts when abusing the CI to test Android jobs.
2. 1b5aaf22e8 — Use vault.centos.org for dist-powerpc64le-linux, see #45744.
3. 33400fbbcd — Modify `src/ci/docker/run.sh` so that the docker images can be run from Docker Toolbox for Windows on Windows 7. I haven't checked the behavior of the newer Docker for Windows on Windows 10. Also, "can run" does not mean all the test can pass successfully (the UDP tests failed last time I checked)
4. d517668a08 — Don't emit a real warning the linker segfault, which affects UI tests like https://github.com/rust-lang/rust/pull/45489#issuecomment-340134944. Log it instead.
5. 51e2247948 — During run-pass, trim the output if stdout/stderr exceeds 416 KB (top 160 KB + bottom 256 KB). This is an attempt to avoid spurious failures like https://github.com/rust-lang/rust/pull/45384#issuecomment-341755788
6. 9cfdabaf3c — Force `gem update --system` before deploy. This is an attempt to prevent spurious error #44159.
7. eee10cc482 — Tries to print the crash log on macOS on failure. This is an attempt to debug #45230.
This commit removes the `rand` crate from the standard library facade as
well as the `__rand` module in the standard library. Neither of these
were used in any meaningful way in the standard library itself. The only
need for randomness in libstd is to initialize the thread-local keys of
a `HashMap`, and that unconditionally used `OsRng` defined in the
standard library anyway.
The cruft of the `rand` crate and the extra `rand` support in the
standard library makes libstd slightly more difficult to port to new
platforms, namely WebAssembly which doesn't have any randomness at all
(without interfacing with JS). The purpose of this commit is to clarify
and streamline randomness in libstd, focusing on how it's only required
in one location, hashmap seeds.
Note that the `rand` crate out of tree has almost always been a drop-in
replacement for the `rand` crate in-tree, so any usage (accidental or
purposeful) of the crate in-tree should switch to the `rand` crate on
crates.io. This then also has the further benefit of avoiding
duplication (mostly) between the two crates!
This affects regular code generation as well as constant evaluation in trans,
but not the HIR constant evaluator because that one returns an error for
overflowing casts and NaN-to-int casts. That error is conservatively
correct and we should be careful to not accept more code in constant
expressions.
The changes to code generation are guarded by a new -Z flag, to be able
to evaluate the performance impact. The trans constant evaluation changes
are unconditional because they have no run time impact and don't affect
type checking either.
These tools have been corrected in their upstream repo's, and the
submodules have been updated here to reflect that. I also had to update
Cargo to match what the RLS is expecting.
The tool states for `rustfmt` and `rls` where both changed from "Broken"
to "Testing" in this commit, thus enabling testing and distribution
again.
Make last structs indexes definitions use newtype_index macro
This PR makes the last two index structs not using newtype_index macro to use it and also fixes this https://github.com/rust-lang/rust/issues/45763 issue.
This is intended to prevent the spurious OOM error from
run-pass/rustc-rust-log.rs, by skipping the output in the middle when the
size is over 416 KB, so that the log output will not be overwhelmed.
Bump to 1.23 and update bootstrap
This commit updates the bootstrap compiler, bumps the version to 1.23, updates
Cargo, updates books, and updates crates.io dependencies
Don't emit the same compiler diagnostic twice.
This PR makes the compiler filter out diagnostic messages that have already been emitted during the same compilation session.
update jobserver version to work around macos bug
Update `jobserver` crate to fixrust-lang/cargo#4643, a panic which can't happen according to `libc::poll`'s man page but was nevertheless reported on macOS 10.9 and 10.10.
r? @alexcrichton
Reactivate clippy in toolstate.toml
The `Cargo.lock` changes are b/c clippy is not its own workspace anymore, but is part of the main workspace
r? @Manishearth
rustc: Remove `used_mut_nodes` from `TyCtxt`
This updates the borrowck query to return a result, and this result is then used
to incrementally check for unused mutable nodes given sets of all the used
mutable nodes.
Closes#42384
Add x86_64-unknown-linux-gnux32 target
This adds X32 ABI support for Linux on X86_64. Let's package and dist it so we can star testing libc, libstd, etc.
Fixes https://github.com/rust-lang/rfcs/issues/1339
This updates the borrowck query to return a result, and this result is then used
to incrementally check for unused mutable nodes given sets of all the used
mutable nodes.
Closes#42384
Initial support for `..=` syntax
#28237
This PR adds `..=` as a synonym for `...` in patterns and expressions.
Since `...` in expressions was never stable, we now issue a warning.
cc @durka
r? @aturon
rustbuild: Update `cmake` dependency
Should help suppress some warnings from various repos as `cmake` in the newest
version disables warnings by default.
bump gcc for bootstrap
On Windows, the gcc crate would send /Wall to msvc, which would cause
builds to get flooded with warnings, exploding compile times from one
hour to more than 72! The gcc crate version 0.3.54 changes this behavior
to send /W4 instead, which greatly cuts down on cl.exe flooding the
command prompt window with warnings.
Individualize feature gates for const fn invocation
This PR changes the meaning of `#![feature(const_fn)]` so it is only required to declare a const fn but not to call one. Based on discussion at #24111. I was hoping we could have an FCP here in order to move that conversation forward.
This sets the stage for future stabilization of the constness of several functions in the standard library (listed below), so could someone please tag the lang team for review.
- `std::cell`
- `Cell::new`
- `RefCell::new`
- `UnsafeCell::new`
- `std::mem`
- `size_of`
- `align_of`
- `std::ptr`
- `null`
- `null_mut`
- `std::sync`
- `atomic`
- `Atomic{Bool,Ptr,Isize,Usize}::new`
- `once`
- `Once::new`
- primitives
- `{integer}::min_value`
- `{integer}::max_value`
Some other functions are const but they are also unstable or hidden, e.g. `Unique::new` so they don't have to be considered at this time.
After this stabilization, the following `*_INIT` constants in the standard library can be deprecated. I wasn't sure whether to include those deprecations in the current PR.
- `std::sync`
- `atomic`
- `ATOMIC_{BOOL,ISIZE,USIZE}_INIT`
- `once`
- `ONCE_INIT`
On Windows, the gcc crate would send /Wall to msvc, which would cause
builds to get flooded with warnings, exploding compile times from one
hour to more than 72! The gcc crate version 0.3.54 changes this behavior
to send /W4 instead, which greatly cuts down on cl.exe flooding the
command prompt window with warnings.
The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.
This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.
Hopefully this can help speed up local builds by doing fewer copies all over the
place!
use gcc::Build rather than deprecated gcc::Config
I did `cargo update -p gcc` to upgrade only this package. Is there further process that should be follwoed when updating a build dependency from crates.io?
r? @alexcrichton
Fixes#43973
Update rls
And expose the `CFG_VERSION` env var to tools so they can determine the version of Rust.
This gets the RLS back on master and so completes the PR dance for the generators PR.
r? @alexcrichton
Initial diagnostic API for proc-macros.
This commit introduces the ability to create and emit `Diagnostic` structures from proc-macros, allowing for proc-macro authors to emit warning, error, note, and help messages just like the compiler does.
The API is somewhat based on the diagnostic API already present in `rustc` with several changes that improve usability. The entry point into the diagnostic API is a new `Diagnostic` type which is primarily created through new `error`, `warning`, `help`, and `note` methods on `Span`. The `Diagnostic` type records the diagnostic level, message, and optional `Span` for the top-level diagnostic and contains a `Vec` of all of the child diagnostics. Child diagnostics can be added through builder methods on `Diagnostic`.
A typical use of the API may look like:
```rust
let token = parse_token();
let val = parse_val();
val.span
.error(format!("expected A but found {}", val))
.span_note(token.span, "because of this token")
.help("consider using a different token")
.emit();
```
cc @jseyfried @nrc @dtolnay @alexcrichton
This commit introduces the ability to create and emit `Diagnostic`
structures from proc-macros, allowing for proc-macro authors to emit
warning, error, note, and help messages just like the compiler does.
post-rebase: Do not put "(Ast)" suffix in error msg unless passed `-Z borrowck-mir`.
(But unconditionally include "(Mir)" suffix for mir-borrowck errors.)
Cargo pulls in libc from crates.io for a number of dependencies, but
0.2.27 is too old to work properly with Solaris. In particular, it
needs the change to make Solaris' PTHREAD_PROCESS_PRIVATE a 16-bit
integer.
APFloat: Rewrite It In Rust and use it for deterministic floating-point CTFE.
As part of the CTFE initiative, we're forced to find a solution for floating-point operations.
By design, IEEE-754 does not explicitly define everything in a deterministic manner, and there is some variability between platforms, at the very least (e.g. NaN payloads).
If types are to evaluate constant expressions involving type (or in the future, const) generics, that evaluation needs to be *fully deterministic*, even across `rustc` host platforms.
That is, if `[T; T::X]` was used in a cross-compiled library, and the evaluation of `T::X` executed a floating-point operation, that operation has to be reproducible on *any other host*, only knowing `T` and the definition of the `X` associated const (as either AST or HIR).
Failure to uphold those rules allows an associated type (e.g. `<Foo as Iterator>::Item`) to be seen as two (or more) different types, depending on the current host, and such type safety violations typically allow writing of a `transmute` in safe code, given enough generics.
The options considered by @rust-lang/compiler were:
1. Ban floating-point operations in generic const-evaluation contexts
2. Emulate floating-point operations in an uniformly deterministic fashion
The former option may seem appealing at first, but floating-point operations *are allowed today*, so they can't be banned wholesale, a distinction has to be made between the code that already works, and future generic contexts. *Moreover*, every computation that succeeded *has to be cached*, otherwise the generic case can be reproduced without any generics. IMO there are too many ways it can go wrong, and a single violation can be enough for an unsoundness hole.
Not to mention we may end up really wanting floating-point operations *anyway*, in CTFE.
I went with the latter option, and seeing how LLVM *already* has a library for this exact purpose (as it needs to perform optimizations independently of host floating-point capabilities), i.e. `APFloat`, that was what I ended up basing this PR on.
But having been burned by the low reusability of bindings that link to LLVM, and because I would *rather* the floating-point operations to be wrong than not deterministic or not memory-safe (`APFloat` does far more pointer juggling than I'm comfortable with), I decided to RIIR.
This way, we have a guarantee of *no* `unsafe` code, a bit more control over the where native floating-point might accidentally be involved, and non-LLVM backends can share it.
I've also ported all the testcases over, *before* any functionality, to catch any mistakes.
Currently the PR replaces all CTFE operations to go through `apfloat::ieee::{Single,Double}`, keeping only the bits of the `f32` / `f64` memory representation in between operations.
Converting from a string also double-checks that `core::num` and `apfloat` agree on the interpretation of a floating-point number literal, in case either of them has any bugs left around.
r? @nikomatsakis
f? @nagisa @est31
<hr/>
Huge thanks to @edef1c for first demoing usable `APFloat` bindings and to @chandlerc for fielding my questions on IRC about `APFloat` peculiarities (also upstreaming some bugfixes).
Run translation and LLVM in parallel when compiling with multiple CGUs
This is still a work in progress but the bulk of the implementation is done, so I thought it would be good to get it in front of more eyes.
This PR makes the compiler start running LLVM while translation is still in progress, effectively allowing for more parallelism towards the end of the compilation pipeline. It also allows the main thread to switch between either translation or running LLVM, which allows to reduce peak memory usage since not all LLVM module have to be kept in memory until linking. This is especially good for incr. comp. but it works just as well when running with `-Ccodegen-units=N`.
In order to help tuning and debugging the work scheduler, the PR adds the `-Ztrans-time-graph` flag which spits out html files that show how work packages where scheduled:
(red is translation, green is llvm)
One side effect here is that `-Ztime-passes` might show something not quite correct because trans and LLVM are not strictly separated anymore. I plan to have some special handling there that will try to produce useful output.
One open question is how to determine whether the trans-thread should switch to intermediate LLVM processing.
TODO:
- [x] Restore `-Z time-passes` output for LLVM.
- [x] Update documentation, esp. for work package scheduling.
- [x] Tune the scheduling algorithm.
cc @alexcrichton @rust-lang/compiler
Three small fixes for save-analysis
First commit does some naive deduplication of macro uses. We end up with lots of duplication here because of the weird way we get this data (we extract a use for every span generated by a macro use).
Second commit is basically a typo fix.
Third commit is a bit interesting, it partially reverts a change from #40939 where temporary variables in format! (and thus println!) got a span with the primary pointing at the value stored into the temporary (e.g., `x` in `println!("...", x)`). If `format!` had a definition it should point at the temporary in the macro def, but since it is built-in, that is not possible (for now), so `DUMMY_SP` is the best we can do (using the span in the callee really breaks save-analysis because it thinks `x` is a definition as well as a reference).
There aren't a test for this stuff because: the deduplication is filtered by any of the users of save-analysis, so it is purely an efficiency change. I couldn't actually find an example for the second commit that we have any machinery to test, and the third commit is tested by the RLS, so there will be a test once I update the RLS version and and uncomment the previously failing tests).
r? @jseyfried
Rework Rustbuild to an eagerly compiling approach
This introduces a new dependency on `serde`; I don't believe that's a problem since bootstrap is compiled with nightly/beta always so proc macros are available. Compile times are slightly longer -- about 2-3x (30 seconds vs. 10 seconds). I don't think this is too big a problem, especially since recompiling bootstrap is somewhat rare. I think we can remove the dependency on Serde if necessary, though, so let me know.
r? @alexcrichton
Update the `cargo` submodule
Notably pull in an update to the `jobserver` crate to have Cargo set the
`CARGO_MAKEFLAGS` environment variable instead of the `MAKEFLAGS` environment
variable.
cc https://github.com/rust-lang/rust/issues/42635
Notably pull in an update to the `jobserver` crate to have Cargo set the
`CARGO_MAKEFLAGS` environment variable instead of the `MAKEFLAGS` environment
variable.
Switch to rust-lang-nursery/compiler-builtins
This commit migrates the in-tree `libcompiler_builtins` to the upstream version
at https://github.com/rust-lang-nursery/compiler-builtins. The upstream version
has a number of intrinsics written in Rust and serves as an in-progress rewrite
of compiler-rt into Rust. Additionally it also contains all the existing
intrinsics defined in `libcompiler_builtins` for 128-bit integers.
It's been the intention since the beginning to make this transition but
previously it just lacked the manpower to get done. As this PR likely shows it
wasn't a trivial integration! Some highlight changes are:
* The PR rust-lang-nursery/compiler-builtins#166 contains a number of fixes
across platforms and also some refactorings to make the intrinsics easier to
read. The additional testing added there also fixed a number of integration
issues when pulling the repository into this tree.
* LTO with the compiler-builtins crate was fixed to link in the entire crate
after the LTO process as these intrinsics are excluded from LTO.
* Treatment of hidden symbols was updated as previously the
`#![compiler_builtins]` crate would mark all symbol *imports* as hidden
whereas it was only intended to mark *exports* as hidden.
rustc: Implement the #[global_allocator] attribute
This PR is an implementation of [RFC 1974] which specifies a new method of
defining a global allocator for a program. This obsoletes the old
`#![allocator]` attribute and also removes support for it.
[RFC 1974]: https://github.com/rust-lang/rfcs/pull/1974
The new `#[global_allocator]` attribute solves many issues encountered with the
`#![allocator]` attribute such as composition and restrictions on the crate
graph itself. The compiler now has much more control over the ABI of the
allocator and how it's implemented, allowing much more freedom in terms of how
this feature is implemented.
cc #27389
This PR is an implementation of [RFC 1974] which specifies a new method of
defining a global allocator for a program. This obsoletes the old
`#![allocator]` attribute and also removes support for it.
[RFC 1974]: https://github.com/rust-lang/rfcs/pull/197
The new `#[global_allocator]` attribute solves many issues encountered with the
`#![allocator]` attribute such as composition and restrictions on the crate
graph itself. The compiler now has much more control over the ABI of the
allocator and how it's implemented, allowing much more freedom in terms of how
this feature is implemented.
cc #27389
This commit migrates the in-tree `libcompiler_builtins` to the upstream version
at https://github.com/rust-lang-nursery/compiler-builtins. The upstream version
has a number of intrinsics written in Rust and serves as an in-progress rewrite
of compiler-rt into Rust. Additionally it also contains all the existing
intrinsics defined in `libcompiler_builtins` for 128-bit integers.
It's been the intention since the beginning to make this transition but
previously it just lacked the manpower to get done. As this PR likely shows it
wasn't a trivial integration! Some highlight changes are:
* The PR rust-lang-nursery/compiler-builtins#166 contains a number of fixes
across platforms and also some refactorings to make the intrinsics easier to
read. The additional testing added there also fixed a number of integration
issues when pulling the repository into this tree.
* LTO with the compiler-builtins crate was fixed to link in the entire crate
after the LTO process as these intrinsics are excluded from LTO.
* Treatment of hidden symbols was updated as previously the
`#![compiler_builtins]` crate would mark all symbol *imports* as hidden
whereas it was only intended to mark *exports* as hidden.
When writing LLVM IR output demangled fn name in comments
`--emit=llvm-ir` looks like this now:
```
; <alloc::vec::Vec<T> as core::ops::index::IndexMut<core::ops::range::RangeFull>>::index_mut
; Function Attrs: inlinehint uwtable
define internal { i8*, i64 } @"_ZN106_$LT$alloc..vec..Vec$LT$T$GT$$u20$as$u20$core..ops..index..IndexMut$LT$core..ops..range..RangeFull$GT$$GT$9index_mut17h7f7b576609f30262E"(%"alloc::vec::Vec<u8>"* dereferenceable(24)) unnamed_addr #0 {
start:
...
```
cc https://github.com/integer32llc/rust-playground/issues/15
Integrate jobserver support to parallel codegen
This commit integrates the `jobserver` crate into the compiler. The crate was
previously integrated in to Cargo as part of rust-lang/cargo#4110. The purpose
here is to two-fold:
* Primarily the compiler can cooperate with Cargo on parallelism. When you run
`cargo build -j4` then this'll make sure that the entire build process between
Cargo/rustc won't use more than 4 cores, whereas today you'd get 4 rustc
instances which may all try to spawn lots of threads.
* Secondarily rustc/Cargo can now integrate with a foreign GNU `make` jobserver.
This means that if you call cargo/rustc from `make` or another
jobserver-compatible implementation it'll use foreign parallelism settings
instead of creating new ones locally.
As the number of parallel codegen instances in the compiler continues to grow
over time with the advent of incremental compilation it's expected that this'll
become more of a problem, so this is intended to nip concurrent concerns in the
bud by having all the tools to cooperate!
Note that while rustc has support for itself creating a jobserver it's far more
likely that rustc will always use the jobserver configured by Cargo. Cargo today
will now set a jobserver unconditionally for rustc to use.
This commit integrates the `jobserver` crate into the compiler. The crate was
previously integrated in to Cargo as part of rust-lang/cargo#4110. The purpose
here is to two-fold:
* Primarily the compiler can cooperate with Cargo on parallelism. When you run
`cargo build -j4` then this'll make sure that the entire build process between
Cargo/rustc won't use more than 4 cores, whereas today you'd get 4 rustc
instances which may all try to spawn lots of threads.
* Secondarily rustc/Cargo can now integrate with a foreign GNU `make` jobserver.
This means that if you call cargo/rustc from `make` or another
jobserver-compatible implementation it'll use foreign parallelism settings
instead of creating new ones locally.
As the number of parallel codegen instances in the compiler continues to grow
over time with the advent of incremental compilation it's expected that this'll
become more of a problem, so this is intended to nip concurrent concerns in the
bud by having all the tools to cooperate!
Note that while rustc has support for itself creating a jobserver it's far more
likely that rustc will always use the jobserver configured by Cargo. Cargo today
will now set a jobserver unconditionally for rustc to use.
This commit deletes the in-tree `getopts` crate in favor of the crates.io-based
`getopts` crate. The main difference here is with a new builder-style API, but
otherwise everything else remains relatively standard.
Update cargo/rls submodules and dependencies
Brings in a few regression fixes on the Cargo side, updates the rls to work
with the newer Cargo, and also updates other crates.io dependencies to pull in
various bug fixes and such.
Implement lazy loading of external crates' sources. Fixes#38875Fixes#38875. This is a follow-up to #42507. When a (now correctly translated) span from an external crate is referenced in a error, warning or info message, we still don't have the source code being referenced.
Since stuffing the source in the serialized metadata of an rlib is extremely wasteful, the following scheme has been implemented:
* File maps now contain a source hash that gets serialized as well.
* When a span is rendered in a message, the source hash in the corresponding file map(s) is used to try and load the source from the corresponding file on disk. If the file is not found or the hashes don't match, the failed attempt is recorded (and not retried).
* The machinery fetching source lines from file maps is augmented to use the lazily loaded external source as a secondary fallback for file maps belonging to external crates.
This required a small change to the expected stderr of one UI test (it now renders a span, where previously was none).
Further work can be done based on this - some of the machinery previously used to hide external spans is possibly obsolete and the hashing code can be reused in different places as well.
r? @eddyb
Brings in a few regression fixes on the Cargo side, updates the rls to work
with the newer Cargo, and also updates other crates.io dependencies to pull in
various bug fixes and such.
Autogenerate stubs and SUMMARY.md in the unstable book
Removes a speed bump in compiler development by autogenerating stubs for features in the unstable book. See #42454 for discussion.
The PR contains three commits, separated in order to make review easy:
* The first commit converts the tidy tool from a binary crate to a crate that contains both a library and a binary. In the second commit, we'll use the tidy library
* The second and main commit introduces autogeneration of SUMMARY.md and feature stub files
* The third commit turns off the tidy lint that checks for features without a stub, and removes the stub files. A separate commit due to the large number of files touched
Members of the doc team who wish to document some features can either do this (where `$rustsrc` is the root of the rust repo git checkout):
1. cd to `$rustsrc/src/tools/unstable-book-gen` and then do `cargo run $rustsrc/src $rustsrc/src/doc/unstable-book` to put the stubs into the unstable book
2. cd to `$rustsrc` and run `git ls-files --others --exclude-standard` to list the newly added stubs
3. choose a file to edit, then `git add` it and `git commit`
4. afterwards, remove all changes by the tool by doing `git --reset hard` and `git clean -f`
Or they can do this:
1. remove the comment marker in `src/tools/tidy/src/unstable_book.rs` line 122
2. run `./x.py test src/tools/tidy` to list the unstable features which only have stubs
3. revert the change in 1
3. document one of the chosen unstable features
The changes done by this PR also allow for further development:
* tidy obtains information about tracking issues. We can now forbid differing tracking issues between differing `#![unstable]` annotations. I haven't done this but plan to in a future PR
* we now have a general framework for generating stuff for the unstable book at build time. Further changes can autogenerate a list of the API a given library feature exposes.
The old way to simply click through the documentation after it has been uploaded to rust-lang.org works as well.
r? @nagisa
Fixes#42454
Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
The rationale is that BOM stripping is needed for lazy source loading
for external crates, and duplication can be avoided by moving the
corresponding functionality to libsyntax_pos.
Update rust-installer for Windows executable mode
It now marks a few whitelisted extensions as executable in the tarball,
so Windows packages can be extracted on other platforms and directly
execute install.sh.
It also includes a fix for the chmod on bulk dirs, so now the html docs
won't be marked executable en masse.
Fixes#42121
r? @alexcrichton
It now marks a few whitelisted extensions as executable in the tarball,
so Windows packages can be extracted on other platforms and directly
execute install.sh.
It also includes a fix for the chmod on bulk dirs, so now the html docs
won't be marked executable en masse.
Fixes#42121
r? @alexcrichton
This avoids double compiled Cargo. Hopefully this would speed up (extended) compilation for ~10m.
Notes: when updating Cargo submodule, the replacement version may also need to be updated.
Make only rustc_trans depend on rustc_llvm
With these changes, only rustc_trans depends directly on rustc_llvm (and no crate gained a new dependency on trans). This means changing LLVM doesn't rebuild librustc or rustc_metadata, only rustc_trans, rustc_driver and the rustc executable
Also, rustc_driver technically doesn't know about LLVM any more (of course, it still handles a ton of options that conceptually refer to LLVM, but it delegates their implementation to trans).
What I *didn't* implement was merging most or all of rustc_llvm into rustc_trans. I ran into a nasty bug, which was probably just a silly typo somewhere but I probably won't have the time to figure it out in the next week or two. I opened #41699 for that step.
Fixes#41473
This does not actually improve build times, since it still depends
on rustc_trans, but is better layering and fits the multi-backend
future slightly better.
Consequently, session creation can no longer initialize LLVM.
The few places that use the compiler without going through
rustc_driver/CompilerCalls thus need to be careful to manually
initialize LLVM (via rustc_trans!) immediately after session
creation.
This means librustc is not rebuilt when LLVM changes.
Move the code for loading metadata from rlibs and dylibs from
rustc_metadata into rustc_trans, and introduce a trait to avoid
introducing a direct dependency on rustc_trans.
This means rustc_metadata is no longer rebuilt when LLVM changes.
When -Z profile is passed, the GCDAProfiling LLVM pass is added
to the pipeline, which uses debug information to instrument the IR.
After compiling with -Z profile, the $(OUT_DIR)/$(CRATE_NAME).gcno
file is created, containing initial profiling information.
After running the program built, the $(OUT_DIR)/$(CRATE_NAME).gcda
file is created, containing branch counters.
The created *.gcno and *.gcda files can be processed using
the "llvm-cov gcov" and "lcov" tools. The profiling data LLVM
generates does not faithfully follow the GCC's format for *.gcno
and *.gcda files, and so it will probably not work with other tools
(such as gcov itself) that consume these files.
use diff crate for compile-fail test diagnostics #41474
Hello!
This fixes#41474
We were using a custom implementation to dump the differences between expected and actual outputs of compile-fail tests.
I removed this internal implementation and added `diff` crate as a new dependency to `compile-fail`.
Again, huge thanks to @nikomatsakis for guiding.
Currently our slowest test suite on android, run-pass, takes over 5 times longer
than the x86_64 component (~400 -> ~2200s). Typically QEMU emulation does indeed
add overhead, but not 5x for this kind of workload. One of the slowest parts of
the Android process is that *compilation* happens serially. Tests themselves
need to run single-threaded on the emulator (due to how the test harness works)
and this forces the compiles themselves to be single threaded.
Now Travis gives us more than one core per machine, so it'd be much better if we
could take advantage of them! The emulator itself is still fundamentally
single-threaded, but we should see a nice speedup by sending binaries for it to
run much more quickly.
It turns out that we've already got all the tools to do this in-tree. The
qemu-test-{server,client} that are in use for the ARM Linux testing are a
perfect match for the Android emulator. This commit migrates the custom adb
management code in compiletest/rustbuild to the same qemu-test-{server,client}
implementation that ARM Linux uses.
This allows us to lift the parallelism restriction on the compiletest test
suites, namely run-pass. Consequently although we'll still basically run the
tests themselves in single threaded mode we'll be able to compile all of them in
parallel, keeping the pipeline much more full and using more cores for the work
at hand. Additionally the architecture here should be a bit speedier as it
should have less overhead than adb which is a whole new process on both the host
and the emulator!
Locally on an 8 core machine I've seen the run-pass test suite speed up from
taking nearly an hour to only taking 6 minutes. I don't think we'll see quite a
drastic speedup on Travis but I'm hoping this change can place the Android tests
well below 2 hours instead of just above 2 hours.
Because the client/server here are now repurposed for more than just QEMU,
they've been renamed to `remote-test-{server,client}`.
Note that this PR does not currently modify how debuginfo tests are executed on
Android. While parallelizable it wouldn't be quite as easy, so that's left to
another day. Thankfully that test suite is much smaller than the run-pass test
suite.
As a final fix I discovered that the ARM and Android test suites were actually
running all library unit tests (e.g. stdtest, coretest, etc) twice. I've
corrected that to only run tests once which should also give a nice boost in
overall cycle time here.
Prior to this commit, the contents of the Unstable Book were assumed to
be unstable features. This commit moves features into 'language features'
or 'library features' subsections. It also moves the 'linker_flavor'
compiler flag into a new 'Compiler Flags' subsection.
Even though it was helpful, I removed the tidy check that
cross-references the SUMMARY.md links with the Unstable Book directory
contents just because it would be difficult to maintain.
Relevant PR: https://github.com/rust-lang/rust/issues/41142.
Add a tidy lint that checks for...
* Unstable Book sections with no corresponding SUMMARY.md links
* unstable features that don't have Unstable Book sections
* Unstable Book sections that don't have corresponding unstable features
Marc-Antoine Perennou
Oliver Schneider
bors
kennytm
Nick Cameron
Alex Crichton
Robin Kruppe
Dustin Speckhals
Santiago Pastorino
Wonwoo Choi
Igor Matuszewski
Manish Goregaokar
Michael Woerister
leonardo.yvens
QuietMisdreavus
Corey Farwell
Alex Burka
bjorn3
Vadim Petrochenkov
Marco A L Barbosa
steveklabnik
est31
Badel2
Tamir Duberstein
Tim Neumann
Alex Burka
Ralf Jung
Guillaume Gomez
Sergio Benitez
Tatsuyuki Ishi
Mark Simulacrum
Felix S. Klock II
Danek Duvall
Eduard-Mihai Burtescu
Michael Woerister
Simon Sapin
Stepan Koltsov
Jeffrey Seyfried
Murarth
Inokentiy Babushkin
Marco Castelluccio
Brian Anderson
Josh Stone
Liran Ringel
ubsan
Niko Matsakis
whitequark
Cengiz Can
Jessica Hamilton
