This commit switches the rustbuild build system to compiling the
compiler twice for a normal bootstrap rather than the historical three
times.
Rust is a bootstrapped language which means that a previous version of
the compiler is used to build the next version of the compiler. Over
time, however, we change many parts of compiler artifacts such as the
metadata format, symbol names, etc. These changes make artifacts from
one compiler incompatible from another compiler. Consequently if a
compiler wants to be able to use some artifacts then it itself must have
compiled the artifacts.
Historically the rustc build system has achieved this by compiling the
compiler three times:
* An older compiler (stage0) is downloaded to kick off the chain.
* This compiler now compiles a new compiler (stage1)
* The stage1 compiler then compiles another compiler (stage2)
* Finally, the stage2 compiler needs libraries to link against, so it
compiles all the libraries again.
This entire process amounts in compiling the compiler three times.
Additionally, this process always guarantees that the Rust source tree
can compile itself because the stage2 compiler (created by a freshly
created compiler) would successfully compile itself again. This
property, ensuring Rust can compile itself, is quite important!
In general, though, this third compilation is not required for general
purpose development on the compiler. The third compiler (stage2) can
reuse the libraries that were created during the second compile. In
other words, the second compilation can produce both a compiler and the
libraries that compiler will use. These artifacts *must* be compatible
due to the way plugins work today anyway, and they were created by the
same source code so they *should* be compatible as well.
So given all that, this commit switches the default build process to
only compile the compiler three times, avoiding this third compilation
by copying artifacts from the previous one. Along the way a new entry in
the Travis matrix was also added to ensure that our full bootstrap can
succeed. This entry does not run tests, though, as it should not be
necessary.
To restore the old behavior of a full bootstrap (three compiles) you can
either pass:
./configure --enable-full-bootstrap
or if you're using config.toml:
[build]
full-bootstrap = true
Overall this will hopefully be an easy 33% win in build times of the
compiler. If we do 33% less work we should be 33% faster! This in turn
should affect cycle times and such on Travis and AppVeyor positively as
well as making it easier to work on the compiler itself.
A new option is introduced under the `[llvm]` section of `config.toml`,
`targets`, for overriding the list of LLVM targets to build support for.
The option is passed through to LLVM configure script. Also notes are
added about the implications of (ab)using the option; since the default
is not changed, and users of the option are expected to know what
they're doing anyway (as every porter should), the impact should be
minimal.
Fixes#38200.
Add prefix to config.toml
This allows `rustbuild` to be used to install to a prefix.
```toml
[build]
prefix = "/path/to/install"
```
For example, the following `config.toml` will cause `x.py dist --install` to install to `/path/to/install`
This commit adds support for sccache, a ccache-like compiler which works on MSVC
and stores results into an S3 bucket. This also switches over all Travis and
AppVeyor automation to using sccache to ensure a shared and unified cache over
time which can be shared across builders.
The support for sccache manifests as a new `--enable-sccache` option which
instructs us to configure LLVM differently to use a 'sccache' binary instead of
a 'ccache' binary. All docker images for Travis builds are updated to download
Mozilla's tooltool builds of sccache onto various containers and systems.
Additionally a new `rust-lang-ci-sccache` bucket is configured to hold all of
our ccache goodies.
A few changes are included here:
* The `winapi` and `url` dependencies were dropped. The source code for these
projects is pretty weighty, and we're about to vendor them, so let's not
commit to that intake just yet. If necessary we can vendor them later but for
now it shouldn't be necessary.
* The `--frozen` flag is now always passed to Cargo, obviating the need for
tidy's `cargo_lock` check.
* Tidy was updated to not check the vendor directory
Closes#34687
This commit enables by default passing the `-C debuginfo=1` argument to the
compiler for the stable, beta, and nightly release channels. A new configure
option was also added, `--enable-debuginfo-lines`, to enable this behavior in
developer builds as well.
Closes#36452
We hope to move to AppVeyor in the near future off of Buildbot + EC2. This adds
an `appveyor.yml` configuration file which is ready to run builds on the auto
branch. This is also accompanied with a few minor fixes to the build system and
such to accomodate AppVeyor.
The intention is that we're not switching over to AppVeyor entirely just yet,
but rather we'll watch the builds for a week or so. If everything checks out
then we'll start gating on AppVeyor instead of Buildbot!
rustbuild: per target musl-root
config.toml now accepts a target.$TARGET.musl-root key that lets you
override the "build" musl-root value, which is set via the --musl-root
flag or via the build.musl-root key.
With this change, it's now possible to compile std for several musl
targets at once. Here's are the sample commands to do such thing:
```
$ configure \
--enable-rustbuild \
--target=x86_64-unknown-linux-musl,arm-unknown-linux-musleabi \
--musl-root=/musl/x86_64-unknown-linux-musl/
$ edit config.toml && tail config.toml
[target.arm-unknown-linux-musleabi]
musl-root = "/x-tools/arm-unknown-linux-musleabi/arm-unknown-linux-musleabi/sysroot/usr"
$ make
```
r? @alexcrichton
With this we should be able to start producing releases of std for arm musl targets
rustbuild: add config.toml option to disable codegen tests
Fixes#36232.
I think it worked? Here's a build log where I tried to bootstrap, it crashed, then I added the setting to config.toml and it continued: https://gist.github.com/durka/cbf97cf04b8e065f1a2cfda4c1b6bf95
r? @alexcrichton
but keep them enabled by default to maintain the status quo.
When disabled shaves ~56KB off every x86_64-unknown-linux-gnu
binary.
To disable backtraces you have to use a config.toml (see
src/bootstrap/config.toml.example for details) when building rustc/std:
$ python bootstrap.py --config=config.toml
Currently rustbuild isn't detecting the `-O` flag for tests via the
`--disable-optimize-tests` or not command line flag to `./configure`, and this
commit patches up the support to pass `-O` by default.
This commit expands the bootstrap build system's `README.md` as well as ensuring
that all API documentation is present and up-to-date. Additionally a new
`config.toml.example` file is checked in with commented out versions of all
possible configuration values.
Alex Crichton
Wang Xuerui
Jeremy Soller
Tim Neumann
bors
Jorge Aparicio
Manish Goregaokar
Alex Burka
Alex Burka