This is a new suite of tests that verifies that the compiler
builds specific revisions of select crates from crates.io.
It does not run by default. It is intended that buildbot
runs these tests against all PRs, and gate on them.
This commit implements the `make dist` command in the new rustbuild build
system, porting over `dist.mk` and `prepare.mk` into Rust. There's a huge amount
of complexity between those two files, not all of which is likely justified, so
the Rust implementation is *much* smaller.
Currently the implementation still shells out to rust-installer as well as some
python scripts, but ideally we'd rewrite it all in the future to not shell out
and be in Rust proper.
rustbuild: Fix cross compiling to FreeBSD
This commit fixes our support for cross compiling a compiler to run on FreeBSD.
Over the weekend I managed to get a cross compiler from Linux to FreeBSD [1]
which I hope to soon use to start producing FreeBSD nightly compilers. With the
`make dist` support added in #32237 we should be able to produce standard
rustc/rust-std packages for FreeBSD through a new slave with this cross compiler.
Currently, however, we don't "Just Work" when cross compiling FreeBSD and a
number of changes were required (part of this PR). They include:
* A few build fixes were needed in LLVM. Our own branch has been rebased on the
actual 3.8 release and I applied one extra commit [2] which contains two fixes:
1. The LLVM CMake build system passes the `-Wl,-z,defs` flag on many
platforms, but *not* when `CMAKE_SYSTEM_NAME` is "FreeBSD". Unfortunately
this doesn't take into account when we're cross compiling, and as predicted
the build will fail if `-Wl,-z,defs` is passed (see [3] for more info). To
fix this we test `TARGET_TRIPLE` instead of the `CMAKE_SYSTEM_NAME` which
is what we're compiling for which fixes the problem.
2. The `PATH_MAX` constant is apparently defined in a different location than
many other Unix systems, so a file which required this just needed some
help to keep compiling.
* Support for compiling compiler-rt with CMake has been added to rustbuild. It
looks like it just emulates Linux in what it compiles as it didn't seem to
naturally produce anything else... At least the architecture is right, so
seems good for now at least!
[1]: https://github.com/alexcrichton/port-of-rust/blob/master/prebuilt/freebsd/Dockerfile
[2]: https://github.com/rust-lang/llvm/commit/be89e4b5
[3]: https://bugs.webkit.org/show_bug.cgi?id=138420
This commit fixes our support for cross compiling a compiler to run on FreeBSD.
Over the weekend I managed to get a cross compiler from Linux to FreeBSD [1]
which I hope to soon use to start producing FreeBSD nightly compilers. With the
`make dist` support added in #32237 we should be able to produce standard
rustc/rust-std packages for FreeBSD through a new slave with this cross compiler.
Currently, however, we don't "Just Work" when cross compiling FreeBSD and a
number of changes were required (part of this PR). They include:
* A few build fixes were needed in LLVM. Our own branch has been rebased on the
actual 3.8 release and I applied one extra commit [2] which contains two fixes:
1. The LLVM CMake build system passes the `-Wl,-z,defs` flag on many
platforms, but *not* when `CMAKE_SYSTEM_NAME` is "FreeBSD". Unfortunately
this doesn't take into account when we're cross compiling, and as predicted
the build will fail if `-Wl,-z,defs` is passed (see [3] for more info). To
fix this we test `TARGET_TRIPLE` instead of the `CMAKE_SYSTEM_NAME` which
is what we're compiling for which fixes the problem.
2. The `PATH_MAX` constant is apparently defined in a different location than
many other Unix systems, so a file which required this just needed some
help to keep compiling.
* Support for compiling compiler-rt with CMake has been added to rustbuild. It
looks like it just emulates Linux in what it compiles as it didn't seem to
naturally produce anything else... At least the architecture is right, so
seems good for now at least!
[1]: https://github.com/alexcrichton/port-of-rust/blob/master/prebuilt/freebsd/Dockerfile
[2]: https://github.com/rust-lang/llvm/commit/be89e4b5
[3]: https://bugs.webkit.org/show_bug.cgi?id=138420
The facet of a stage is rarely relevant when running a tool or building
something, it's all a question of what stage the *compiler* is built in. We've
already got a nice handy `Compiler` structure to carry this information, so
let's use it!
This refactors the signature of the `Build::cargo` function two ways:
1. The `stage` argument is removed, this was just duplicated with the `compiler`
argument's stage field.
2. The `target` argument is now required. This was a bug where if the `--target`
flag isn't passed then the snapshot stage0 compiler is always used, so we
won't pick up any changes.
Much of the other changes in this commit are just propagating these decisions
outwards. For example many of the `Step` variants no longer have a stage
argument as they're baked into the compiler.
Add a script to get run which verifies that `href` links in documents are
correct. We're always getting a steady stream of "fix a broken link" PRs and
issue reports, and we should probably just nip them all in the bud.
Run `cargo doc` to generate all documentation for the standard library, and also
add a target which generates documentation for the compiler as well (but don't
enable it by default).
This commit implements documentation generation of the nomicon, the book, the
style guide, and the standalone docs. New steps were added for each one as well
as appropriate makefile targets for each one as well.
When cross compiling for a new host, we can't actually run the host compiler to
generate its own libs. In theory, however, all stage2 compilers (for any host)
will produce the same libraries, so we just require the build compiler to
produce the necessary host libraries and then we link those into place.
This switches the defaults to ensure that everything is built with the build
compiler rather than the host compiler itself (which we're not guaranteed to be
able to run)
Currently all multi-host builds assume the the build platform can run the
`llvm-config` binary generated for each host platform we're creating a compiler
for. Unfortunately this assumption isn't always true when cross compiling, so we
need to handle this case.
This commit alters the build script of `rustc_llvm` to understand when it's
running an `llvm-config` which is different than the platform we're targeting for.
This commit fixes a longstanding issue with the makefiles where all host
platforms bootstrap themselves. This commit alters the build logic for the
bootstrap to instead only bootstrap the build triple, and all other compilers
are compiled from that one compiler.
The benefit of this change is that we can cross-compile compilers which cannot
run on the build platform. For example our builders could start creating
`arm-unknown-linux-gnueabihf` compilers.
This reduces the amount of bootstrapping we do, reducing the amount of test
coverage, but overall it should largely just end in faster build times for
multi-host compiles as well as enabling a feature which can't be done today.
cc #5258
This commit implements documentation generation of the nomicon, the book, the
style guide, and the standalone docs. New steps were added for each one as well
as appropriate makefile targets for each one as well.
Also add a `clean` target for the makefiles to blow away everything related to
the build. Note that this specifically does not tamper with:
* the LLVM build directory
* the directory of the bootstrap system
* the cached downloads of cargo/rustc
This commit adds a `--enable-rustbuild` option to the configure script which
will copy a different `Makefile.in` into place to intercept all `make`
invocations.
Currently this makefile only has one target, but it's expected to be filled out
quite a bit over time!
During the transition period where we're still using ./configure and makefiles,
read some extra configuration from `config.mk` if it's present. This means that
the bootstrap build should be configured the same as the original ./configure
invocation.
Eventually this will all be removed in favor of only storing information in
`config.toml` (e.g. the configure script will generate config.toml), but for now
this should suffice.
This commit is the start of a series of commits which start to replace the
makefiles with a Cargo-based build system. The aim is not to remove the
makefiles entirely just yet but rather just replace the portions that invoke the
compiler to do the bootstrap. This commit specifically adds enough support to
perform the bootstrap (and all the cross compilation within) along with
generating documentation.
More commits will follow up in this series to actually wire up the makefiles to
call this build system, so stay tuned!
