RISC-V GNU/Linux as host platform
This PR add a new builder named `dist-riscv64-linux` that builds the compiler toolchain for RISC-V 64-bit GNU/Linux.
r? @alexcrichton
This change creates a new Docker image, "dist-x86_64-illumos", and sets
things up to build the full set of "dist" packages for illumos hosts, so
that illumos users can use "rustup" to install packages. It also
adjusts the manifest builder to expect complete toolchains for this
platform.
We need to add runners designed for an aarch64 host system, and it'd be
nice to return an error message if someone tries to run an image
designed for an host architecture in another one.
To start the work on this, this commit moves all the existing builders
in the host-x86_64 directory, and changes the run.sh script to look up
the image in the correct directory based on the host architecture.
Remove legacy InnoSetup GUI installer
On Windows the InnoSetup `.exe` installer was superseded by the MSI installer long ago. It's no longer needed.
The `.exe` installer hasn't been linked from the [other installation methods](https://forge.rust-lang.org/infra/other-installation-methods.html#standalone) page in many years. As far as I can tell the intent was always to remove this installer once the MSI proved itself. Though admittedly both installers feel very "legacy" at this point.
Removing this would mean we only maintain one Windows GUI installer and would speed up the distribution phase.
As a result of removing InnoSetup, this closes#24397
Remove vestigial CI job msvc-aux.
This CI job isn't really doing anything, so it seems prudent to remove it.
For some history:
* This was introduced in #48809 when the msvc job was split in two to keep it under 2 hours (oh the good old days). At the time, this check-aux job did a bunch of things:
* tidy
* src/test/pretty
* src/test/run-pass/pretty
* src/test/run-fail/pretty
* src/test/run-pass-valgrind/pretty
* src/test/run-pass-fulldeps/pretty
* src/test/run-fail-fulldeps/pretty
* Tidy was removed in #60777.
* run-pass and run-pass-fulldeps moved to UI in #63029
* src/test/pretty removed in #58140
* src/test/run-fail moved to UI in #71185
* run-fail-fulldeps removed in #51285
Over time through attrition, the job was left with one lonely thing: `src/test/run-pass-valgrind/pretty`. And of course, this wasn't actually running the "pretty" tests. The normal `run-pass-valgrind` tests ran, and then when it tried to run in "pretty" mode, all the tests were ignored because compiletest thought nothing had changed (apparently compiletest isn't fingerprinting the mode? Needs more investigation…). `run-pass-valgrind` is already being run as part of `x86_64-msvc-1`, so there's no need to run it here.
I've taken the liberty of removing `src/test/run-pass-valgrind/pretty` as a distinct test. I'm guessing from the other PR's that the pretty tests should now live in `src/test/pretty`, and that the team has moved away from doing pretty tests on other parts of the `src/test` tree.
This helps us have enough disk space for our builders to be able to complete
successfully. For now, the choices are ad-hoc and 'definitely not needed'. This
should never fail the build, as everything our build needs should be inside
Docker.
LLVM 7 is over a year old, which should be plenty for compatibility. The
last LLVM 6 holdout was llvm-emscripten, which went away in #65501.
I've also included a fix for LLVM 8 lacking `MemorySanitizerOptions`,
which was broken by #66522.
Some environment variables (like DEPLOY or DEPLOY_ALT for dist builders,
or IMAGE on Linux builders) are set on a lot of builders, and whether
they should be present or not can be detected automatically based on the
builder name and the platform.
This commit simplifies the CI configuration by automatically setting
those environment variables.
This commit replaces the mirrors base URL contained in the MINGW_URL
with a CUSTOM_MINGW=1 environment variable. The mirrors base URL will be
fetched instead through the MIRRORS_BASE environment variable, defined
in src/ci/shared.sh.
Currently the `RUST_CONFIGURE_ARGS` variable apparently has a trailing
newline at the end of it due to the way it's configured in yaml. This
causes issues with MSVC's `install-clang.sh` step where the way the bash
syntax works out means that we drop the arg we're trying to add and it
doesn't actually get added!
The hopeful fix here is to tweak how we specify the yaml syntax to not
have a trailing newline, we'll see what CI says about this...
Upload toolstates.json to rust-lang-ci2
This PR does two things:
* Following up with https://github.com/rust-lang/rust/pull/65202, it migrates deploying artifacts to CI in a script. Both uploading release artifacts and CPU stats were merged into the same script, designing it to be easily extended.
* Uploads the toolstate JSON to `rust-lang-ci2` along with the release artifacts, both for Linux and Windows. This is needed because @RalfJung wants to stop shipping MIRI when its tests are failing, and the toolstate repo doesn't have entries for each commit. Having the toolstate data (just for that specific commit) on `rust-lang-ci2` will simplify the code a lot.
r? @alexcrichton
cc @RalfJung
Before this commit toolstates.json was stored in /tmp and it wasn't
mounted outside the build container. That caused uploading the file in
the upload-artifacts task to fail, as the file was missing on the host.
Mounting /tmp/toolstates.json alone is not the best approach: if the
file is missing when the container is started the Docker engine will
create a *directory* named /tmp/toolstates.json.
The Docker issue could be solved by pre-creating an empty file named
/tmp/toolstates.json, but doing that could cause problems if bootstrap
fails to generate the file and the toolstate scripts receive an empty
JSON.
The approach I took in this commit is to instead mount a /tmp/toolstate
directory inside Docker, and create the toolstates.json file in it. That
also required a small bootstrap change to ensure the directory is
created if it's missing.
Uploading the toolstate data for each commit will help our release
tooling understand which components are failing, to possibly skip
shipping broken tools to users.
We have a job in our CI (PR's x86_64-gnu-tools) that's supposed to run
only when a submodule is changed in the PR, and it works by having a
task at the start of the build that skips all the following tasks if the
condition isn't met.
Before this commit that task was gated with template parameters, which
is a unique feature of Azure Pipelines. To make our CI more generic this
commit switches the gate to use a simple environment variable plus a
condition, which should be supported on more CI providers.
Pietro Albini
Mark Rousskov
Josh Triplett
Joshua Nelson
Manish Goregaokar
msizanoen1
Joshua M. Clulow
Ralf Jung
Eric Huss
Mateusz Mikuła
Chris Denton
Josh Stone
Matt Cooper
Matthias Krüger
Dylan DPC
Alex Crichton
Mazdak Farrokhzad