Add x86_64-unknown-linux-gnux32 target
This adds X32 ABI support for Linux on X86_64. Let's package and dist it so we can star testing libc, libstd, etc.
Fixes https://github.com/rust-lang/rfcs/issues/1339
Enable building clippy in CI
r? @alexcrichton
As discussed at Rustfest. Measured additional time is 4 minutes on my machine if no dependencies are shared with other tools. In reality most dependencies are shared (especially the slow to compile ones like serde).
cc @Manishearth
Does not run clippy's test suite, since
a) it is nontrivial in the rustc build system
b) it breaks more frequently but the breakage is negligible
If clippy breaks, the procedure to follow is documented under https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#external-dependencies
rustdoc is built separately to rustc now so the docs would need to be
generated separately as well. Also rustdoc doesn't build at stage 1
which prevented the compiler docs being built at stage 1.
rustbuild: Prevent spurious rebuilds of the RLS
The RLS currently is rebuilt every time you test it because the `OPENSSL_DIR`
env var is changing, which is in turn caused by an accidental omission of
`prepare_tool_cargo` when testing the RLS.
incr.comp.: Bring back output of -Zincremental-info.
This got kind lost during the transition to red/green.
I also switched back from `eprintln!()` to `println!()` since the former never actually produced any output. I suspect this has to do with `libterm` somehow monopolizing `stderr`.
r? @nikomatsakis
Fix PEP8 style issues in bootstrap code
This fixes PEP8 style issues (other than line-length) in the bootstrap Python code.
The most important fix is in the `set` function where the code was indented with 6 spaces instead of 4.
let rustdoc print the crate version into docs
This PR adds a new unstable flag to rustdoc, `--crate-version`, which when present will add a new entry to the sidebar of the root module, printing the given version number:
Closes#24336
(The WIP status is because i don't want to merge this until i can get the std docs to use it, which i need help from rustbuild people to make sure i get right.)
The RLS currently is rebuilt every time you test it because the `OPENSSL_DIR`
env var is changing, which is in turn caused by an accidental omission of
`prepare_tool_cargo` when testing the RLS.
1. Add -f flag to curl, so when the server returns 403 or 500 it will fail
immediately.
2. Moved the checksum part into the retry loop, assuming checksum failure
is due to broken download that can be fixed by downloading again.
Fix path to x.py in bootstrap/configure.py script
We may see a help message in the end of the output of the ./configure script:
```
$ ./configure
configure: processing command line
configure:
configure: build.configure-args := []
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python ./src/bootstrap/x.py --help`
configure:
```
but the `x.py` script is actually in the rust root directory and
executing of such help string will give us error:
```
$ python ./src/bootstrap/x.py --help
python: can't open file './src/bootstrap/x.py': [Errno 2] No such file
or directory
```
This patch fixes path to the x.py script in the output of the ./configure
We may see a help message in the end of the output of the ./configure script:
$ ./configure
configure: processing command line
configure:
configure: build.configure-args := []
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python ./src/bootstrap/x.py --help`
configure:
but the x.py script is actually in the rust root directory and
executing of such help string will give us error:
$ python ./src/bootstrap/x.py --help
python: can't open file './src/bootstrap/x.py': [Errno 2] No such file
or directory
This patch fixes path to the x.py script in the output of the ./configure
Use identity operator `is` when comparing to None
This is very minor, but idiomatic Python code uses `is` for comparisons to `None`. This is because semantically we want to compare to the "identity" of `None`, not its value.
See [PEP8 for details](https://www.python.org/dev/peps/pep-0008/#programming-recommendations).
Fix variable name reference
As best I can tell, this was a typo due to how similar it looks to the function above it. PyCharm found this as a unbound local variable.
Add aarch64-unknown-linux-musl target
This adds support for the aarch64-unknown-linux-musl target in the build and CI systems.
This addresses half of issue #42520.
The new file `aarch64_unknown_linux_musl.rs` is a copy of `aarch64_unknown_linux_gnu.rs` with "gnu" replaced by "musl", and the added logic in `build-arm-musl.sh` is similarly a near-copy of the arches around it, so overall the changes were straightforward.
Testing:
```
$ sudo ./src/ci/docker/run.sh cross
...
Dist std stage2 (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-musl)
Building stage2 test artifacts (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-musl)
Compiling getopts v0.2.14
Compiling term v0.0.0 (file:///checkout/src/libterm)
Compiling test v0.0.0 (file:///checkout/src/libtest)
Finished release [optimized] target(s) in 16.91 secs
Copying stage2 test from stage2 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / aarch64-unknown-linux-musl)
...
Build completed successfully in 0:55:22
```
```
$ rustup toolchain link local obj/build/x86_64-unknown-linux-gnu/stage2
$ rustup default local
```
After setting the local toolchain as default, and adding this in ~/.cargo/config:
```
[target.aarch64-unknown-linux-musl]
linker = "aarch64-linux-musl-gcc"
```
...then the toolchain was able to build a working ripgrep as a test:
```
$ readelf -a target/aarch64-unknown-linux-musl/debug/rg | grep -i interpreter
$ readelf -a target/aarch64-unknown-linux-musl/debug/rg | grep NEEDED
$ file target/aarch64-unknown-linux-musl/debug/rg
target/aarch64-unknown-linux-musl/debug/rg: ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=be11036b0988fac5dccc9f6487eb780b05186582, not stripped
```
Allow writing metadata without llvm
# Todo:
* [x] Rebase
* [x] Fix eventual errors
* [x] <strike>Find some crate to write elf files</strike> (will do it later)
Cc #43842
* Adjust bootstrap to provide useful output on failure
* Add missing package dependencies in the build environment
* Fix permission bits on prebuilt toolchain files
If config.toml doesn't exist, then an IOError will be raised
on the `with open(...)` line. Prior to e788fa7, this was
caught because the `except` clause didn't specify what
exceptions it caught, so both IOError and OSError were
caught
Run the miri test suite on the aux builder and travis
Reopen of #38350
see https://github.com/rust-lang/rust/pull/43340#issuecomment-316940762 for earlier discussion
Rationale for running miri's test suite in rustc's CI is that miri currently contains many features that we want in const eval in the future, and these features would break if the test suite is not run.
fixes#44077
r? @nikomatsakis
cc @eddyb
rustbuild: Don't pass `-j` if called by `make`
In these situations Cargo just prints out a warning about ignoring the flag
anyway, so let `make` take care of jobs and whatnot instead of getting warnings
printed.
In these situations Cargo just prints out a warning about ignoring the flag
anyway, so let `make` take care of jobs and whatnot instead of getting warnings
printed.
ci: Upload/download from a new S3 bucket
Moving buckets from us-east-1 to us-west-1 because us-west-1 is where
rust-central-station itself runs and in general is where we have all our other
buckets.
rustbuild: Fix test "test rustdoc" invocation
Previously it would use the librustc output directory which would cause rustdoc
to get entirely recompiled, whereas the intention is that it uses the
already-compiled artifacts from building rustdoc itself, using the tool output
directory
bump gcc for bootstrap
On Windows, the gcc crate would send /Wall to msvc, which would cause
builds to get flooded with warnings, exploding compile times from one
hour to more than 72! The gcc crate version 0.3.54 changes this behavior
to send /W4 instead, which greatly cuts down on cl.exe flooding the
command prompt window with warnings.
Moving buckets from us-east-1 to us-west-1 because us-west-1 is where
rust-central-station itself runs and in general is where we have all our other
buckets.
Previously it would use the librustc output directory which would cause rustdoc
to get entirely recompiled, whereas the intention is that it uses the
already-compiled artifacts from building rustdoc itself, using the tool output
directory
Right now we comiple rustdoc in stage 2 and the error index in stage 0, which
ends up compiling rustdoc twice! To avoid compiling rustdoc twice (which takes
awhile) let's just compile it once in stage 2.
Explicitly run perl for OpenSSL Configure
OpenSSL's Configure script is missing a shebang. On some platforms,
execve falls back to execution with the shell. Some other platforms,
like musl, will fail with an exec format error. Avoid this by calling
perl explicitly (since it's a perl script).
On Windows, the gcc crate would send /Wall to msvc, which would cause
builds to get flooded with warnings, exploding compile times from one
hour to more than 72! The gcc crate version 0.3.54 changes this behavior
to send /W4 instead, which greatly cuts down on cl.exe flooding the
command prompt window with warnings.
Apparently `File::create` was called when there was an existing hard link or the
like, causing an existing file to get accidentally truncated!
Closes#44487
rustbuild: Switch back to using hard links
The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.
This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.
Hopefully this can help speed up local builds by doing fewer copies all over the
place!
The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.
This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.
Hopefully this can help speed up local builds by doing fewer copies all over the
place!
Include rustc in the default `./x.py install`
The default install used to include rustc, rust-std, and rust-docs, but
the refactoring in commit 6b3413d825 make rustc only default in
extended builds. This commit makes rustc installed by default again.
use gcc::Build rather than deprecated gcc::Config
I did `cargo update -p gcc` to upgrade only this package. Is there further process that should be follwoed when updating a build dependency from crates.io?
r? @alexcrichton
Fixes#43973
The default install used to include rustc, rust-std, and rust-docs, but
the refactoring in commit 6b3413d825 make rustc only default in
extended builds. This commit makes rustc installed by default again.
Doc tests are temporarily disabled until next release cycle, since
current beta Cargo errors on them. Upgrade should be smooth as the
relevant tests are already fixed in this commit.
Update rls
And expose the `CFG_VERSION` env var to tools so they can determine the version of Rust.
This gets the RLS back on master and so completes the PR dance for the generators PR.
r? @alexcrichton
Add clippy as a submodule
~~This builds clippy as part of `./x.py build` (locally and in CI).~~
This allows building clippy with `./x.py build src/tools/clippy`
~~Needs https://github.com/nrc/dev-tools-team/issues/18#issuecomment-322456461 to be resolved before it can be merged.~~ Contributers can simply open a PR to clippy and point the submodule at the `pull/$pr_number/head` branch.
This does **not** build clippy or test the clippy test suite at all as per https://github.com/nrc/dev-tools-team/issues/18#issuecomment-321411418
r? @nrc
cc @Manishearth @llogiq @mcarton @alexcrichton
rustbuild: update the rust-src filter for compiler-rt
We wanted `src/compiler-rt/test` filtered from the `rust-src` package,
but that path is now `src/libcompiler_builtins/compiler-rt/test`. This
saves over half of the installed rust-src size. (50MB -> 22MB)
We wanted `src/compiler-rt/test` filtered from the `rust-src` package,
but that path is now `src/libcompiler_builtins/compiler-rt/test`. This
saves over half of the installed rust-src size. (50MB -> 22MB)
rustbuild: Avoid some extraneous rustc compiles on cross builds
This tweaks a few locations here and there to avoid compiling rustc too many times on our cross-builders on CI.
Closes https://github.com/rust-lang/rust/issues/44132
Fail ./x.py on invalid command
Make the ./x.py script fail when run with an invalid command, like:
```
./x.py nonsense
```
This helps in case of chaining multiple runs, eg.:
```
./x.py biuld && ./x.py test
```
include Cargo.{toml,lock} in rust-src tarball
The lock file is interesting because e.g. xargo could use it to build libstd against the same dependencies that were used for the main build. More generally speaking, just documenting in this form which exact dependencies should be used IMHO makes lots of sense.
I added the Cargo.toml mostly because having the lock without the toml feels odd. Of course, the toml contains references to paths that don't actually exist in the rust-src tarball. Not sure if that is considered a problem.
OpenSSL's Configure script is missing a shebang. On some platforms,
execve falls back to execution with the shell. Some other platforms,
like musl, will fail with an exec format error. Avoid this by calling
perl explicitly (since it's a perl script).
rustbuild: Rewrite the configure script in Python
This commit rewrites our ancient `./configure` script from shell into Python.
The impetus for this change is to remove `config.mk` which is just a vestige of
the old makefile build system at this point. Instead all configuration is now
solely done through `config.toml`.
The python script allows us to more flexibly program (aka we can use loops
easily) and create a `config.toml` which is based off `config.toml.example`.
This way we can preserve comments and munge various values as we see fit.
It is intended that the configure script here is a drop-in replacement for the
previous configure script, no functional change is intended. Also note that the
rationale for this is also because our build system requires Python, so having a
python script a bit earlier shouldn't cause too many problems.
Closes#40730Closes#43295Closes#42255Closes#38058Closes#32176
This commit rewrites our ancient `./configure` script from shell into Python.
The impetus for this change is to remove `config.mk` which is just a vestige of
the old makefile build system at this point. Instead all configuration is now
solely done through `config.toml`.
The python script allows us to more flexibly program (aka we can use loops
easily) and create a `config.toml` which is based off `config.toml.example`.
This way we can preserve comments and munge various values as we see fit.
It is intended that the configure script here is a drop-in replacement for the
previous configure script, no functional change is intended. Also note that the
rationale for this is also because our build system requires Python, so having a
python script a bit earlier shouldn't cause too many problems.
Closes#40730
Make the ./x.py script fail when run with an invalid command, like:
./x.py nonsense
This helps in case of chaining multiple runs, eg.:
./x.py biuld && ./x.py test
Discovered in #43767 it turns out the default MSBuild generator in CMake for
whatever reason isn't supporting many of the configuration options we give to
LLVM. To improve the contributor experience automatically enable Ninja if we
find it to ensure that "flavorful" configurations of LLVM work by default in
more situations.
Closes#43767
This controls the value of the crt-static feature used when building the
standard library for a target, as well as the compiler itself when that
target is the host.
They are required for linking it, even though it is a library, because
crtn.o in post_link_objects, as hardcoded in src/librustc_back/target/
linux_musl_base.rs, is added to the linker command line for both
executables and libraries.
Implement a temp redirect for cargo docs
As discussed in
https://github.com/rust-lang/cargo/issues/4040#issuecomment-321639074
This is a redirect meant to be replaced once cargo docs have been
converted to mdbook. We just want *a* URL to ride the trains for now so
that we can print doc.rust-lang.org/cargo in the paper book and
guarantee that it will go *somewhere* useful by the time the book is
printed.
Implemented as a meta redirect in HTML because we don't currently have
any google juice at doc.rust-lang.org/cargo to lose.
When I run `./x.py doc`, this creates a `build/x86_64-apple-darwin/doc/cargo/index.html` file that contains a meta redirect to doc.crates.io. As I understand rust-central-station to work, this should be what we need to make `doc.rust-lang.org/cargo` to work.
r? @alexcrichton and/or @steveklabnik
As discussed in
https://github.com/rust-lang/cargo/issues/4040#issuecomment-321639074
This is a redirect meant to be replaced once cargo docs have been
converted to mdbook. We just want *a* URL to ride the trains for now so
that we can print doc.rust-lang.org/cargo in the paper book and
guarantee that it will go *somewhere* useful by the time the book is
printed.
Implemented as a meta redirect in HTML because we don't currently have
any google juice at doc.rust-lang.org/cargo to lose.
