to not shadow the system installed LLD when linking with LLD.
Before:
- `-C linker=lld -Z linker-flavor=ld.lld` uses rustc's LLD
- It's not possible to use a system installed LLD that's named `lld`
With this commit:
- `-C linker=rust-lld -Z linker-flavor=ld.lld` uses rustc's LLD
- `-C linker=lld -Z linker-flavor=ld.lld` uses the system installed LLD
Do not build LLVM tools for any of the tools
None of the tools in the list should need LLVM tools themselves as far as I can
tell; if this is incorrect, we can re-enable the tool building later.
The primary reason for doing this is that rust-central-station uses the
BuildManifest tool and building LLVM there is not cached: it takes ~1.5
hours on the 2 core machine. This commit should make nightlies and
stable releases much faster.
Followup to https://github.com/rust-lang/rust/pull/51459, r? @kennytm
I'm mostly relying on CI to test this so probably don't roll it up; I'm not sure how to (and not particularly inclined to) wait for multiple hours to test this locally. I imagine that the failures should be fairly obvious when/if encountered.
None of the tools in the list should need LLVM tools themselves as far as I can
tell; if this is incorrect, we can re-enable the tool building later.
The primary reason for doing this is that rust-central-station uses the
BuildManifest tool and building LLVM there is not cached: it takes ~1.5
hours on the 2 core machine. This commit should make nightlies and
stable releases much faster.
build: add llvm-tools to manifest
This commit expands on a previous commit to build llvm-tools as a rustup component. It causes the llvm-tools component to be built if the extended step is active. It also adds llvm-tools to the build manifest so rustup can find it.
I tested this as far as I could, but had to hack `build-manifest/src/main.rs` a bit as it is not supported on MacOS. The main change I am not sure about is this line:
```rust
self.package("llvm-tools", &mut manifest.pkg, TARGETS);
```
There are numerous calls to `self.package()`, and I'm not sure if `TARGETS`, `HOSTS`, or `["*"]` is appropriate for llvm-tools.
Otherwise I mostly copied the example set by `rustfmt-preview`.
This commit expands on a previous commit to build llvm-tools as a rustup
component. It causes the llvm-tools component to be built if the
extended step is active. It also adds llvm-tools to the build manifest
so rustup can find it.
ship LLVM tools with the toolchain
this PR adds llvm-{nm,objcopy,objdump,size} to the rustc sysroot (right next to LLD)
this slightly increases the size of the rustc component. I measured these numbers on x86_64 Linux:
- rustc-1.27.0-dev-x86_64-unknown-linux-gnu.tar.gz 180M -> 193M (+7%)
- rustc-1.27.0-dev-x86_64-unknown-linux-gnu.tar.xz 129M -> 137M (+6%)
r? @alexcrichton
cc #49584
Since they are unlikely to fail and are almost never going to fail
except with bootstrap changes (which would be tested locally anyway) it
makes sense to run these tests close to last.
rustbuild: generate full list of dependencies for metadata
Previously, we didn't send --features to our cargo metadata invocations,
and thus missed some dependencies that we enable through the --features
mechanism.
Previously, we didn't send --features to our cargo metadata invocations,
and thus missed some dependencies that we enable through the --features
mechanism.
Switch to bootstrapping from 1.27
It's possible the Float trait could be removed from core, but I couldn't tell whether it was intended to be removed or not. @SimonSapin may be able to comment more here; we can presumably also do that in a follow up PR as this one is already quite large.
In #49289, rustc was changed to emit metadata for binaries, which made
it so that the librustc.rmeta file created when compiling librustc was
overwritten by the rustc-main compilation. This commit renames the
rustc-main binary to avoid this problem.
https://github.com/rust-lang/cargo/issues/5524 has also been filed to
see if Cargo can learn to warn on this situation instead of leaving it
for the user to debug.
Ensure libraries built in stage0 have unique metadata
Issue #50786 shows a case with local rebuild where the libraries built
by stage0 had the same suffix as stage0's own, and were accidentally
loaded by that stage0 rustc when compiling `librustc_trans`.
Now we set `__CARGO_DEFAULT_LIB_METADATA` to "bootstrap" during stage0,
rather than the release channel like usual, so the library suffix will
always be completely distinct from the stage0 compiler.
Issue #50786 shows a case with local rebuild where the libraries built
by stage0 had the same suffix as stage0's own, and were accidentally
loaded by that stage0 rustc when compiling `librustc_trans`.
Now we set `__CARGO_DEFAULT_LIB_METADATA` to "bootstrap" during stage0,
rather than the release channel like usual, so the library suffix will
always be completely distinct from the stage0 compiler.
Use the correct crt*.o files when linking musl targets.
This is supposed to support optionally using the system copy of musl
libc instead of the included one if supported. This currently only
affects the start files, which is enough to allow building rustc on musl
targets.
Most of the changes are analogous to crt-static.
Excluding the start files is something musl based distributions usually patch into their copy of rustc:
- https://github.com/alpinelinux/aports/blob/eb064c8/community/rust/musl-fix-linux_musl_base.patch
- https://github.com/voidlinux/void-packages/blob/77400fc/srcpkgs/rust/patches/link-musl-dynamically.patch
For third-party distributions that not yet carry those patches it would be nice if it was supported without the need to patch upstream sources.
## Reasons
### What breaks?
Some start files were missed when originally writing the logic to swap in musl start files (gcc comes with its own start files, which are suppressed by -nostdlib, but not manually included later on). This caused #36710, which also affects rustc with the internal llvm copy or any other system libraries that need crtbegin/crtend.
### How is it fixed?
The system linker already has all the logic to decide which start files to include, so we can just defer to it (except of course if it doesn't target musl).
### Why is it optional?
In #40113 it was first tried to remove the start files, which broke compiling musl-targeting static binaries with a glibc-targeting compiler. This is why it eventually landed without removing the start files. Being an option side-steps the issue.
### Why are the start files still installed?
This has the nice side-effect, that the produced rust-std-* binaries can still be used by on a glibc-targeting system with a rustc built against glibc.
## Does it work?
With the following build script (using [musl-cross-make](https://github.com/richfelker/musl-cross-make)): https://shadowice.org/~mixi/rust-musl/build.sh, I was able to cross-compile a musl-host musl-targeting rustc on a glibc-based system. The resulting binaries are at https://shadowice.org/~mixi/rust-musl/binaries/. This also requires #50103 and #50104 (which are also applied to the branch the build script uses).
./x.py test should be able to run individual tests
Allows user to be able to run individual tests by specifying filename i.e `./x.py test src/test/run-pass/foo.rs`
Fixes#48483
Currently on CI we predominately compile LLVM with the default system compiler
which means gcc on Linux, some version of Clang on OSX, MSVC on Windows, and
gcc on MinGW. This commit switches Linux, OSX, and Windows to all use Clang
6.0.0 to build LLVM (aka the C/C++ compiler as part of the bootstrap). This
looks to generate faster code according to #49879 which translates to a faster
rustc (as LLVM internally is faster)
The major changes here were to the containers that build Linux releases,
basically adding a new step that uses the previous gcc 4.8 compiler to compile
the next Clang 6.0.0 compiler. Otherwise the OSX and Windows scripts have been
updated to download precompiled versions of Clang 6 and configure the build to
use them.
Note that `cc` was updated here to fix using `clang-cl` with `cc-rs` on MSVC, as
well as an update to `sccache` on Windows which was needed to correctly work
with `clang-cl`. Finally the MinGW compiler is entirely left out here
intentionally as it's currently thought that Clang can't generate C++ code for
MinGW and we need to use gcc, but this should be verified eventually.
trim and pass relative test paths as test-args
use collect for getting test_args
move suite_path to ShouldRun and make Option
append existing args to test_args
use enum for PathSet
handle Suites differently from Paths
Error out if part of test suite but not file
refactor, make requested changes
Add some groundwork for cross-language LTO.
Implements part of #49879:
- Adds a `-Z cross-lang-lto` flag to rustc
- Makes sure that bitcode is embedded in object files if the flag is set.
This should already allow for using cross language LTO for staticlibs (where one has to invoke the linker manually anyway). However, `rustc` will not try to enable LTO for its own linker invocations yet.
r? @alexcrichton
Pass a test directory to rustfmt
Another attempt to fix the rustfmt tests. `RUSTFMT_TEST_DIR` is consumed by Rustfmt in the latext commit (thus the Rustfmt update) because we need a place to create temp files that won't be read-only.
r? @alexcrichton
rustbuild: Allow quick testing of libstd and libcore at stage0
This PR implemented two features:
1. Added a `--no-doc` flag to allow testing a crate *without* doc tests. In this mode, we don't need to build rustdoc, and thus we can skip building the stage2 compiler. (Ideally stage0 test should use the bootstrap rustdoc, but I don't want to mess up the core builder logic here)
2. Moved all libcore tests externally and added a tidy test to ensure we don't accidentally add `#[test]` into libcore.
After this PR, one could run `./x.py test --stage 0 --no-doc src/libstd` to test `libstd` without building the compiler, thus enables us to quickly test new library features.
Misc tweaks
This:
- ~~Add explicit dependencies on `getops`~~
- Fixes the libtest-json test when `RUST_BACKTRACE=1` is set
- ~~Sets `opt-level` to `3`~~
- Removes the use of `staged_api` from `rustc_plugin`
- ~~Enables the Windows Error Reporting dialog when running rustc during bootstrapping~~
- Disables Windows Error Reporting dialog when running compiletest tests
- Enables backtraces when running rustc during bootstrapping
- ~~Removes the `librustc` dependency on `libtest`~~
- Triggers JIT debugging on Windows if rustc panics during bootstrapping
r? @alexcrichton
Add "the Rustc book"
This PR introduces a new book into the documentation, "The rustc book". We already have books for Cargo, and for Rustdoc, rustc should have some too. This book is focused on *users* of rustc, and provides a nice place to write documentation for users.
I haven't put content here, but plan on scaffolding it out very soon, and wanted this PR open for a few discussions first. One of those is "what exactly should said TOC be?" I plan on having a proposed one up tomorrow, but figured I'd let people know to start thinking about it now.
The big one is that we also will want to put https://github.com/rust-lang-nursery/rustc-guide in-tree as well, and the naming is... tough. I'm proposing:
* doc.rust-lang.org/rustc is "The Rustc book", to mirror the other tools' books.
* doc.rust-lang.org/rustc-contribution is "The Rustc contribution guide", and contains that book
@nikomatsakis et al, any thoughts on this? I'm not attached to it in particular, but had to put something together to get this discussion going. I think mirroring the other tools is a good idea for this work, but am not sure where exactly that leaves yours.
Fixes https://github.com/rust-docs/team/issues/11
Disable auto-detection of libxml2 when compiling llvm.
This broke cross-compiling rustc with internal llvm (with both the host and target being executable on the machine), because llvm's build system detected libxml2 on the host, therefore auto-enabled libxml2 support, but wouldn't compile as the target didn't have libxml2 installed.
Add src/test/ui regression testing for NLL
This PR changes `x.py test` so that when you are running the `ui` test suite, it will also always run `compiletest` in the new `--compare-mode=nll`, which just double-checks that when running under the experimental NLL mode, the output matches the `<source-name>.nll.stderr` file, if present.
In order to reduce the chance of a developer revolt in response to this change, this PR also includes some changes to make the `--compare-mode=nll` more user-friendly:
1. It now generates nll-specific .stamp files, and uses them (so that repeated runs can reuse previously cached results).
2. Each line of terminal output distinguishes whether we are running under `--compare-mode=nll` by printing with the prefix `[ui (nll)]` instead of just the prefix `[ui]`.
Subtask of rust-lang/rust#48879
Add rustc_trans to x.py check
r? @Mark-Simulacrum
I looked at `bootstrap/compile.rs` and `bootstrap/check.rs` to try to work out which steps were appropriate, but I'm sure I've overlooked some details here, so it's worth checking carefully I've got all the steps right (e.g. I wasn't sure whether we want to build LLVM if necessary with `x.py check`, though I thought it was probably better to than to not).
From a quick test, it seems to be working, though.
Only emit save-analysis data for `cargo build` tasks
Previously, we were emittinng analysis data for all tasks, including `doc`. That meant we got two sets of save-analysis data, one from the normal build and one from the docs. That means indexing with the RLS took twice as long and made downloads larger and build times longer.
cc https://github.com/rust-lang-nursery/rls/issues/826
r? @Mark-Simulacrum
Previously, we were emittinng analysis data for all tasks, including `doc`. That meant we got two sets of save-analysis data, one from the normal build and one from the docs. That means indexing with the RLS took twice as long and made downloads larger and build times longer.
cc https://github.com/rust-lang-nursery/rls/issues/826
rustbuild: allow building tools with debuginfo
Debugging information for the extended tools is currently disabled for
concerns about the size. This patch adds `--enable-debuginfo-tools` to
let one opt into having that debuginfo.
This is useful for debugging the tools in distro packages. We always
strip debuginfo into separate packages anyway, so the extra size is not
a concern in regular use.
Add docs for the test crate with the std docs
If the compiler docs aren't going to include the test crate then it may as well be included with std.
Fixes#49388
Debugging information for the extended tools is currently disabled for
concerns about the size. This patch adds `--enable-debuginfo-tools` to
let one opt into having that debuginfo.
This is useful for debugging the tools in distro packages. We always
strip debuginfo into separate packages anyway, so the extra size is not
a concern in regular use.
rustbuild: canonicalize prefix `install_sh`
Testing:
```
$ git diff
diff --git a/config.toml.example b/config.toml.example
index 9dd3002506..b47bc490cd 100644
--- a/config.toml.example
+++ b/config.toml.example
@@ -196,7 +196,7 @@
[install]
# Instead of installing to /usr/local, install to this path instead.
-#prefix = "/usr/local"
+prefix = "install-prefix"
# Where to install system configuration files
# If this is a relative path, it will get installed in `prefix` above
$ mkdir install-prefix
$ ./x.py install -i --stage 0 --config config.toml.example
...
$ ls install-prefix/
bin lib share
```
Closes#36989.
r? @Mark-Simulacrum
Testing:
```
$ git diff
diff --git a/config.toml.example b/config.toml.example
index 9dd3002506..b47bc490cd 100644
--- a/config.toml.example
+++ b/config.toml.example
@@ -196,7 +196,7 @@
[install]
# Instead of installing to /usr/local, install to this path instead.
-#prefix = "/usr/local"
+prefix = "install-prefix"
# Where to install system configuration files
# If this is a relative path, it will get installed in `prefix` above
$ mkdir install-prefix
$ ./x.py install -i --stage 0 --config config.toml.example
...
$ ls install-prefix/
bin lib share
```
Closes#36989.
configure.py --tools should set a list instead of a string
Currently the --tools option does not work because it is setting a string value
for 'build.tools'. It should be a list of strings instead.
Jorge Aparicio
bors
Mark Simulacrum
Niels Sascha Reedijk
Brad Campbell
Steven Laabs
Nikita Popov
kennytm
est31
Oliver Schneider
Collins Abitekaniza
Johannes Nixdorf
Santiago Pastorino
steveklabnik
John Kåre Alsaker
David Wood
Jonathan A. Kollasch
Alex Crichton
Irina Popa
Josh Stone
Eric Huss
Oliver Schneider
Collins Abitekaniza
Nick Cameron
Michael Woerister
Harm Berntsen
Vadim Petrochenkov
varkor
Felix S. Klock II
Guillaume Gomez
Fabio B
Chris Coulson
Tamir Duberstein
Dan Callaghan
Oliver Middleton