Fix cross compiling dist/build invocations
I am uncertain why the first commit is not affecting CI. I suspect it's because we pass --disable-docs on most of our cross-compilation builders. The second commit doesn't affect CI because CI runs x.py dist, not x.py build.
Both commits are standalone; together they should resolve#76733. The first commit doesn't really fix that issue but rather just fixes cross-compiled x.py dist, resolving a bug introduced in #76549.
The Rust version number is currently embedded in bootstrap's source
code, which makes it hard to update it automatically or access it
outside of ./x.py (as you'd have to parse the source code).
This commit moves the version number to a standalone plaintext file,
which makes accessing or updating it trivial.
This isn't an issue for most folks who use x.py dist, which will directly depend
on this. But for x.py build, if we don't properly set target here rustdoc will
not be built.
Currently, there is not a default-on step for generating a rustc for a given
target either, so we will fail to build a rustc as well.
rustc is a natively cross-compiling compiler, and generally none of our steps
should care whether they are using a compiler built of triple A or B, just the
--target directive being passed to the running compiler. e.g., when building for
some target C, you don't generally want to build two stds: one with a host A
compiler and the other with a host B compiler. Just one std is sufficient.
This is generally a good idea, and will help with being able to build bootstrap
without Python over time as it means we can "just" build with cargo +beta build
rather than needing the user to set environment variables. This is a minor step,
but a necessary one on that road.
Generating the coverage map
@tmandry @wesleywiser
rustc now generates the coverage map and can support (limited)
coverage report generation, at the function level.
Example commands to generate a coverage report:
```shell
$ BUILD=$HOME/rust/build/x86_64-unknown-linux-gnu
$ $BUILD/stage1/bin/rustc -Zinstrument-coverage \
$HOME/rust/src/test/run-make-fulldeps/instrument-coverage/main.rs
$ LLVM_PROFILE_FILE="main.profraw" ./main
called
$ $BUILD/llvm/bin/llvm-profdata merge -sparse main.profraw -o main.profdata
$ $BUILD/llvm/bin/llvm-cov show --instr-profile=main.profdata main
```
r? @wesleywiser
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage instrumentation
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
The current plan is that submodule tracks the `release` branch of
rust-analyzer, which is updated once a week.
rust-analyzer is a workspace (with a virtual manifest), the actual
binary is provide by `crates/rust-analyzer` package.
Note that we intentionally don't add rust-analyzer to `Kind::Test`,
for two reasons.
*First*, at the moment rust-analyzer's test suite does a couple of
things which might not work in the context of rust repository. For
example, it shells out directly to `rustup` and `rustfmt`. So, making
this work requires non-trivial efforts.
*Second*, it seems unlikely that running tests in rust-lang/rust repo
would provide any additional guarantees. rust-analyzer builds with
stable and does not depend on the specifics of the compiler, so
changes to compiler can't break ra, unless they break stability
guarantee. Additionally, rust-analyzer itself is gated on bors, so we
are pretty confident that test suite passes.
Since rustc-ap-* v659 we now need to set CFG_RELEASE for
rustc-ap-rustc_attr for `#[cfg(version(...))]` to work.
Co-authored-by: Eric Huss <ehuss@users.noreply.github.com>
As rustbook now depends transitively on openssl, it needs access to the
rustc-workspace-hack/all-static feature to pick up openssl-sys/vendored.
This fixes the rust build with `all-static = true` on systems where
openssl is not installed (e.g. when cross-compiling).
This commit changes the return type of `Builder::cargo` to return a
builder that allows dynamically adding more `RUSTFLAGS` values
after-the-fact. While not used yet, this will later be used to delete
more of `rustc.rs`
Since its inception rustbuild has always worked in three stages: one for
libstd, one for libtest, and one for rustc. These three stages were
architected around crates.io dependencies, where rustc wants to depend
on crates.io crates but said crates don't explicitly depend on libstd,
requiring a sysroot assembly step in the middle. This same logic was
applied for libtest where libtest wants to depend on crates.io crates
(`getopts`) but `getopts` didn't say that it depended on std, so it
needed `std` built ahead of time.
Lots of time has passed since the inception of rustbuild, however,
and we've since gotten to the point where even `std` itself is depending
on crates.io crates (albeit with some wonky configuration). This
commit applies the same logic to the two dependencies that the `test`
crate pulls in from crates.io, `getopts` and `unicode-width`. Over the
many years since rustbuild's inception `unicode-width` was the only
dependency picked up by the `test` crate, so the extra configuration
necessary to get crates building in this crate graph is unlikely to be
too much of a burden on developers.
After this patch it means that there are now only two build phasese of
rustbuild, one for libstd and one for rustc. The libtest/libproc_macro
build phase is all lumped into one now with `std`.
This was originally motivated by rust-lang/cargo#7216 where Cargo was
having to deal with synthesizing dependency edges but this commit makes
them explicit in this repository.
Now that we've fully moved to Azure Pipelines and bors has been updated
to only gate on Azure this commit removes the remaining Travis/AppVeyor
support contained in this repository. Most of the deletions here are
related to producing better output on Travis by folding certain
sections. This isn't supported by Azure so there's no need to keep it
around, and if Azure ever adds support we can always add it back!
In developing #61557 I noticed that there were two parts of our tools
that were rebuilt twice on CI. One was rustfmt fixed in #61557, but
another was Cargo. The actual fix for Cargo's double compile was
rust-lang/cargo#7010 and took some time to propagate here. In an effort
to continue to assert that Cargo is itself not compiled twice, I updated
the assertion in rustbuild at the time of working on #61557 but couldn't
land it because the fix wouldn't be ready until the next bootstrap.
The next bootstrap is now here, so the fix can now land! This does not
change the behavior of rustbuild but it is intended to catch the
previous iteration of compiling cargo twice. The main update here was to
consider more files than those in `$target/release/deps` but also
consider those in `$target/release`. That's where, for example,
`libcargo.rlib` shows up and it's the file we learn about, and that's
what we want to deduplicate.
Removing the tool argument in the previous commit means it's no longer
restricted to just bootstrap tools despite being written as such.
Inlining it prevents accidental use.
Currently `rustfmt` is excluded from the "don't build dependencies
twice" check but it's currently building dependencies twice! Namely big
dependencies like `rustc-ap-syntax` are built once for rustfmt and once
for the RLS. This commit includes `rustfmt` in these checks and then
fixes the resulting feature mismatches for winapi.
ci: Attempt to skip a full rustc compile on dist*
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
Rework how bootstrap tools are built
This makes bootstrap tools buildable and testable in stage 0 with the downloaded bootstrap compiler, futhermore, it makes it such that they cannot be built in any other stage.
Notably, this will also mean that compiletest may need to wait a cycle before it can use changes to `libtest`, as it no longer depends on the in-tree libtest.
Eric Huss
bors
Mark Rousskov
Pietro Albini
Rich Kadel
Jake Goulding
Aleksey Kladov
Joshua Nelson
Ralf Jung
Igor Matuszewski
Oliver Scherer
Mateusz Mikuła
Josh Stone
Matthias Krüger
Samuel Holland
Alex Crichton
Mateusz Mikuła
ljedrz
Vadim Petrochenkov
Andy Russell
Michael Woerister
bjorn3
John Kåre Alsaker
