Discovered in #45529 it looks like cross-module TLS imports aren't quite working
today, especially with `hidden` visibility which mostly comes up with multiple
codegen units. As a result this completely disables compiling with ThinLTO and
multiple codegen units on MIPS when bootstrapping.
cc #45654, the tracking issue for this
* SDK tools is upgraded to 27.0.0.
- Refactored to use `sdkmanager`/`avdmanager` instead of the deprecated
`android` tool.
* The Java version used by Android SDK is downgraded to OpenJDK-8, in order
to download the SDK through HTTPS.
* NDK is upgrade to r15c.
- Dropped support for android-9 (2.3 / Gingerbread), the minimal
supported version is now android-14 (4.0 / Ice Cream Sandwich).
- Changed the default Android compiler from GCC to clang.
- For details of change introduced by NDK r15, see
https://github.com/android-ndk/ndk/wiki/Changelog-r15.
Return 0 as an exit status when no subcommand is given to bootstrap
Running `./x.py` emits usage and error messages when no subcommand is given:
```
Usage: x.py <subcommand> [options] [<paths>...]
Subcommands:
build Compile either the compiler or libraries
test Build and run some test suites
bench Build and run some benchmarks
doc Build documentation
clean Clean out build directories
dist Build distribution artifacts
install Install distribution artifacts
To learn more about a subcommand, run `./x.py <subcommand> -h`
failed to run: /home/topecongiro/rust/build/bootstrap/debug/bootstrap
```
IMHO the last line is unnecessary. This PR removes it by changing the return code of `bootstrap` to 0 when no sub command is given.
configure.py: fix --disable-option-checking and extra config paths
- indexing 'option-checking' out of `known_args` had a type error
- when option checking is disabled, don't error on duplicate args, just take the last
- add config.toml stubs for datadir, infodir, and localstatedir (which were already accepted, but broken)
---
This fixes a regression from 1.21 to beta, when the configure script was rewritten in python.
... specifically `datadir`, `infodir`, and `localstatedir`. These were
already accepted by `configure.py`, but it didn't have any place to put
the values.
Getting the value of this argument needs another level of indexing,
as `known_args` are stored in `{dict}[list](opt, value)` form.
Also, when option-checking is disabled, let this bypass the check that
options are only passed once, and just apply the last value.
rustbuild: Fix `no output generated` error for next bootstrap cargo.
Due to rust-lang/cargo#4570, a `*.dll.lib` file is uplifted when building dynamic libraries on Windows. The current bootstrap code does not understand files with multiple extensions, and will instead assume `xxxx.dll` is the file name. This caused a `no output generated` error because it tries to search for `xxxx.dll-hash.lib` inside the `deps/` folder, while it should check for `xxxx-hash.dll.lib` instead.
This PR is blocking #45285, see https://github.com/rust-lang/rust/pull/45285#issuecomment-338454149 and the rest of the comments for detail.
bootstrap: Add openssl configuration for sparc64-unknown-linux-gnu
Hi!
This adds the target missing mapping for sparc64-unknown-linux-gnu.
See: https://github.com/rust-lang/rust/issues/45456
Thanks
Add Stylo and WebRender to src/tools/cargotest
This is a subset of Servo that takes relatively less time to compile and does not use unstable Rust features.
Remove deprecated `collections` crate.
The real `collections` was merged with `alloc`, this facade was introduced [in this PR](https://github.com/rust-lang/rust/pull/42720) to give `#[no_std]` users time to adapt. This was done at least two cycles ago, now we can consider removing it for good.
Due to rust-lang/cargo#4570, a `*.dll.lib` file is uplifted when building
dynamic libraries on Windows. The current bootstrap code does not
understand files with multiple extensions, and will instead assume
`xxxx.dll` is the file name. This caused a `no output generated` error
because it tries to search for `xxxx.dll-hash.lib` inside the `deps/`
folder, while it should check for `xxxx-hash.dll.lib` instead.
This PR is blocking #45285 (Bump to 1.23 and update bootstrap).
Fix rustbuild --color conflict when building on Travis outside of Docker
When trying to build rust on Travis without using `stamp` or `docker`, both `RUSTC_COLOR=1` and `TRAVIS=true` will separately pass `--color always` to the command line. This causes the build to fail due to "*Option 'color' given more than once*".
In this PR, the `RUSTC_COLOR=1` will not be passed in the CI environment.
rustbuild: Build stage 1 error index generator at stage 0
At stage 1 rustdoc is built at stage 0 so the error index generator should be as well.
This fixes `x.py --stage 1 doc` as rustdoc doesn't even build at stage 1.
It was broken by #44605.
r? @alexcrichton
rustbuild: Compile rustc with ThinLTO
This commit enables ThinLTO for the compiler as well as multiple codegen units.
This is intended to get the benefits of parallel codegen while also avoiding
any major loss of perf. Finally this commit is also intended as further testing
for #45320 and shaking out bugs.
rustbuild: Don't try to build rustdoc API docs with compiler docs
rustdoc is built separately to rustc now so the docs would need to be
generated separately as well. Also rustdoc doesn't build at stage 1
which prevented the compiler docs being built at stage 1.
Fixes: #44629
At stage 1 rustdoc is built at stage 0 so the error index generator should be as well.
This fixes `x.py --stage 1 doc` as rustdoc doesn't even build at stage 1.
This commit enables ThinLTO for the compiler as well as multiple codegen units.
This is intended to get the benefits of parallel codegen while also avoiding
any major loss of perf. Finally this commit is also intended as further testing
for #45320 and shaking out bugs.
Make sure to clear out the stageN-{rustc,std,tools} directories.
We copy built tool binaries into a dedicated directory to avoid deleting them,
stageN-tools-bin. These aren't ever cleared out by code, since there should be
no reason to do so, and we'll simply overwrite them as necessary.
When clearing out the stageN-{std,rustc,tools} directories, make sure to delete
both Cargo directories -- per-target and build scripts. This ensures that
changing libstd doesn't cause problems due to build scripts not being rebuilt,
even though they should be.
Fixes https://github.com/rust-lang/rust/issues/44739.
rustbuild: fix dist in debug mode
In debug mode, the artifacts are placed in "debug", so don't hardcode "release" and use our helper to get the right directory name
Allow passing a path with tilde to the configure script
Currently `./configure --local-rust-root=~/.cargo --enable-local-rebuild` fails with
```
Exception: no cargo executable found at `~/.cargo//bin/cargo`
```
Remove --enable-rustbuild config option from an example
`--enable-rustbuild` is no longer recognized by the configure script. Also I think we should use `./configure` and `./x.py` instead of `configure` and `../x.py`.
We copy built tool binaries into a dedicated directory to avoid deleting
them, stageN-tools-bin. These aren't ever cleared out by code, since
there should be no reason to do so, and we'll simply overwrite them as
necessary.
When clearing out the stageN-{std,rustc,tools} directories, make sure to
delete both Cargo directories -- per-target and build scripts. This
ensures that changing libstd doesn't cause problems due to build scripts
not being rebuilt, even though they should be.
While the `config.toml.example` comments say "we automatically check the
version by default," we actually didn't. That check was badly out of
date, only allowing 3.5, 3.6, or 3.7. This it now updated to the new
3.9 minimum requirement, and truly enabled by default.
This commit enables configuring the RLS/rustfmt tools to the "broken" state and
actually get it past CI. The main changes here were to update all dist-related
code to handle the situation where the RLS isn't available. This in turn
involved a homegrown preprocessor-like-function to edit the configuration files
we pass to the various combined installer tools.
Add x86_64-unknown-linux-gnux32 target
This adds X32 ABI support for Linux on X86_64. Let's package and dist it so we can star testing libc, libstd, etc.
Fixes https://github.com/rust-lang/rfcs/issues/1339
Enable building clippy in CI
r? @alexcrichton
As discussed at Rustfest. Measured additional time is 4 minutes on my machine if no dependencies are shared with other tools. In reality most dependencies are shared (especially the slow to compile ones like serde).
cc @Manishearth
Does not run clippy's test suite, since
a) it is nontrivial in the rustc build system
b) it breaks more frequently but the breakage is negligible
If clippy breaks, the procedure to follow is documented under https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#external-dependencies
rustdoc is built separately to rustc now so the docs would need to be
generated separately as well. Also rustdoc doesn't build at stage 1
which prevented the compiler docs being built at stage 1.
rustbuild: Prevent spurious rebuilds of the RLS
The RLS currently is rebuilt every time you test it because the `OPENSSL_DIR`
env var is changing, which is in turn caused by an accidental omission of
`prepare_tool_cargo` when testing the RLS.
incr.comp.: Bring back output of -Zincremental-info.
This got kind lost during the transition to red/green.
I also switched back from `eprintln!()` to `println!()` since the former never actually produced any output. I suspect this has to do with `libterm` somehow monopolizing `stderr`.
r? @nikomatsakis
Fix PEP8 style issues in bootstrap code
This fixes PEP8 style issues (other than line-length) in the bootstrap Python code.
The most important fix is in the `set` function where the code was indented with 6 spaces instead of 4.
let rustdoc print the crate version into docs
This PR adds a new unstable flag to rustdoc, `--crate-version`, which when present will add a new entry to the sidebar of the root module, printing the given version number:
Closes#24336
(The WIP status is because i don't want to merge this until i can get the std docs to use it, which i need help from rustbuild people to make sure i get right.)
The RLS currently is rebuilt every time you test it because the `OPENSSL_DIR`
env var is changing, which is in turn caused by an accidental omission of
`prepare_tool_cargo` when testing the RLS.
1. Add -f flag to curl, so when the server returns 403 or 500 it will fail
immediately.
2. Moved the checksum part into the retry loop, assuming checksum failure
is due to broken download that can be fixed by downloading again.
Fix path to x.py in bootstrap/configure.py script
We may see a help message in the end of the output of the ./configure script:
```
$ ./configure
configure: processing command line
configure:
configure: build.configure-args := []
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python ./src/bootstrap/x.py --help`
configure:
```
but the `x.py` script is actually in the rust root directory and
executing of such help string will give us error:
```
$ python ./src/bootstrap/x.py --help
python: can't open file './src/bootstrap/x.py': [Errno 2] No such file
or directory
```
This patch fixes path to the x.py script in the output of the ./configure
We may see a help message in the end of the output of the ./configure script:
$ ./configure
configure: processing command line
configure:
configure: build.configure-args := []
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python ./src/bootstrap/x.py --help`
configure:
but the x.py script is actually in the rust root directory and
executing of such help string will give us error:
$ python ./src/bootstrap/x.py --help
python: can't open file './src/bootstrap/x.py': [Errno 2] No such file
or directory
This patch fixes path to the x.py script in the output of the ./configure
Use identity operator `is` when comparing to None
This is very minor, but idiomatic Python code uses `is` for comparisons to `None`. This is because semantically we want to compare to the "identity" of `None`, not its value.
See [PEP8 for details](https://www.python.org/dev/peps/pep-0008/#programming-recommendations).
Fix variable name reference
As best I can tell, this was a typo due to how similar it looks to the function above it. PyCharm found this as a unbound local variable.
Add aarch64-unknown-linux-musl target
This adds support for the aarch64-unknown-linux-musl target in the build and CI systems.
This addresses half of issue #42520.
The new file `aarch64_unknown_linux_musl.rs` is a copy of `aarch64_unknown_linux_gnu.rs` with "gnu" replaced by "musl", and the added logic in `build-arm-musl.sh` is similarly a near-copy of the arches around it, so overall the changes were straightforward.
Testing:
```
$ sudo ./src/ci/docker/run.sh cross
...
Dist std stage2 (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-musl)
Building stage2 test artifacts (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-musl)
Compiling getopts v0.2.14
Compiling term v0.0.0 (file:///checkout/src/libterm)
Compiling test v0.0.0 (file:///checkout/src/libtest)
Finished release [optimized] target(s) in 16.91 secs
Copying stage2 test from stage2 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / aarch64-unknown-linux-musl)
...
Build completed successfully in 0:55:22
```
```
$ rustup toolchain link local obj/build/x86_64-unknown-linux-gnu/stage2
$ rustup default local
```
After setting the local toolchain as default, and adding this in ~/.cargo/config:
```
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
```
...then the toolchain was able to build a working ripgrep as a test:
```
$ readelf -a target/aarch64-unknown-linux-musl/debug/rg | grep -i interpreter
$ readelf -a target/aarch64-unknown-linux-musl/debug/rg | grep NEEDED
$ file target/aarch64-unknown-linux-musl/debug/rg
target/aarch64-unknown-linux-musl/debug/rg: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=be11036b0988fac5dccc9f6487eb780b05186582, not stripped
```
Allow writing metadata without llvm
# Todo:
* [x] Rebase
* [x] Fix eventual errors
* [x] <strike>Find some crate to write elf files</strike> (will do it later)
Cc #43842
* Adjust bootstrap to provide useful output on failure
* Add missing package dependencies in the build environment
* Fix permission bits on prebuilt toolchain files
