Since its inception rustbuild has always worked in three stages: one for
libstd, one for libtest, and one for rustc. These three stages were
architected around crates.io dependencies, where rustc wants to depend
on crates.io crates but said crates don't explicitly depend on libstd,
requiring a sysroot assembly step in the middle. This same logic was
applied for libtest where libtest wants to depend on crates.io crates
(`getopts`) but `getopts` didn't say that it depended on std, so it
needed `std` built ahead of time.
Lots of time has passed since the inception of rustbuild, however,
and we've since gotten to the point where even `std` itself is depending
on crates.io crates (albeit with some wonky configuration). This
commit applies the same logic to the two dependencies that the `test`
crate pulls in from crates.io, `getopts` and `unicode-width`. Over the
many years since rustbuild's inception `unicode-width` was the only
dependency picked up by the `test` crate, so the extra configuration
necessary to get crates building in this crate graph is unlikely to be
too much of a burden on developers.
After this patch it means that there are now only two build phasese of
rustbuild, one for libstd and one for rustc. The libtest/libproc_macro
build phase is all lumped into one now with `std`.
This was originally motivated by rust-lang/cargo#7216 where Cargo was
having to deal with synthesizing dependency edges but this commit makes
them explicit in this repository.
Now that we've fully moved to Azure Pipelines and bors has been updated
to only gate on Azure this commit removes the remaining Travis/AppVeyor
support contained in this repository. Most of the deletions here are
related to producing better output on Travis by folding certain
sections. This isn't supported by Azure so there's no need to keep it
around, and if Azure ever adds support we can always add it back!
In developing #61557 I noticed that there were two parts of our tools
that were rebuilt twice on CI. One was rustfmt fixed in #61557, but
another was Cargo. The actual fix for Cargo's double compile was
rust-lang/cargo#7010 and took some time to propagate here. In an effort
to continue to assert that Cargo is itself not compiled twice, I updated
the assertion in rustbuild at the time of working on #61557 but couldn't
land it because the fix wouldn't be ready until the next bootstrap.
The next bootstrap is now here, so the fix can now land! This does not
change the behavior of rustbuild but it is intended to catch the
previous iteration of compiling cargo twice. The main update here was to
consider more files than those in `$target/release/deps` but also
consider those in `$target/release`. That's where, for example,
`libcargo.rlib` shows up and it's the file we learn about, and that's
what we want to deduplicate.
Removing the tool argument in the previous commit means it's no longer
restricted to just bootstrap tools despite being written as such.
Inlining it prevents accidental use.
Currently `rustfmt` is excluded from the "don't build dependencies
twice" check but it's currently building dependencies twice! Namely big
dependencies like `rustc-ap-syntax` are built once for rustfmt and once
for the RLS. This commit includes `rustfmt` in these checks and then
fixes the resulting feature mismatches for winapi.
ci: Attempt to skip a full rustc compile on dist*
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
Rework how bootstrap tools are built
This makes bootstrap tools buildable and testable in stage 0 with the downloaded bootstrap compiler, futhermore, it makes it such that they cannot be built in any other stage.
Notably, this will also mean that compiletest may need to wait a cycle before it can use changes to `libtest`, as it no longer depends on the in-tree libtest.
Fixes rustdoc in stage 0, stage 1
When a request for rustdoc is passed for stage 0, x.py build --stage 0
src/tools/rustdoc or ensure(tool::Rustdoc { .. }) with top_stage = 0, we
return the rustdoc for that compiler (i.e., the beta rustdoc).
This fixes stage 0 of https://github.com/rust-lang/rust/issues/52186 as well as being part of general workflow improvements (making stage 0 testing for std work) for rustbuild.
The stage 1 fix (second commit) completely resolves the problem, so this fixes https://github.com/rust-lang/rust/issues/52186.
The RUSTDOC_LIBDIR should be rustc_libdir, not sysroot_libdir; rustdoc
is like the compiler and should link against rustc's libdir.
Some people currently (i.e., in general, may not be on master) have doc
tests working, but no attempt to determine why has been attempted.
When a request for rustdoc is passed for stage 0, x.py build --stage 0
src/tools/rustdoc or ensure(tool::Rustdoc { .. }) with top_stage = 0, we
return the rustdoc for that compiler (i.e., the beta rustdoc).
When building a distributed compiler on Linux where we use ThinLTO to
create the LLVM shared object this commit switches the compiler to
dynamically linking that LLVM artifact instead of statically linking to
LLVM. The primary goal here is to reduce CI compile times, avoiding two+
ThinLTO builds of all of LLVM. By linking dynamically to LLVM we'll
reuse the one ThinLTO step done by LLVM's build itself.
Lots of discussion about this change can be found [here] and down. A
perf run will show whether this is worth it or not!
[here]: https://github.com/rust-lang/rust/pull/53245#issuecomment-417015334
When building a distributed compiler on Linux where we use ThinLTO to
create the LLVM shared object this commit switches the compiler to
dynamically linking that LLVM artifact instead of statically linking to
LLVM. The primary goal here is to reduce CI compile times, avoiding two+
ThinLTO builds of all of LLVM. By linking dynamically to LLVM we'll
reuse the one ThinLTO step done by LLVM's build itself.
Lots of discussion about this change can be found [here] and down. A
perf run will show whether this is worth it or not!
[here]: https://github.com/rust-lang/rust/pull/53245#issuecomment-417015334
remove useless format!()s
remove redundant field names in a few struct initializations
pass slice instead of a vector to a function
use is_empty() instead of comparisons to .len()
No functional change intended.
In addition to to updating Cargo's submodule and Cargo's dependencies,
this also updates Cargo's build to build OpenSSL statically into Cargo
as well as libcurl unconditionally. This removes OpenSSL build logic
from the bootstrap code, and otherwise requests that even on OSX we
build curl statically.
Previously when building the error index tool in stage 0 we would
attempt to use stage 0 libraries, but because it depends on rustdoc,
those don't exist: rustdoc is built against stage 1 libraries.
This patch aligns those two and passes the stage 1 libdir to the error
index.
Alex Crichton
Mateusz Mikuła
Mark Rousskov
bors
ljedrz
Vadim Petrochenkov
Andy Russell
Michael Woerister
Josh Stone
bjorn3
John Kåre Alsaker
Taiki Endo
Alexander Regueiro
Oliver Scherer
Eric Huss
kennytm
Eduard-Mihai Burtescu
Matthias Krüger
Collins Abitekaniza
Tatsuyuki Ishi