Fix for bootstrapping on NixOS
NixOS puts Linux's dynamic loader in wierd place. Detect when we're on NixOS and patch the downloaded bootstrap executables appropriately.
travis: Disable source tarballs on most builders
Currently we create a source tarball on almost all of the `DEPLOY=1` builders
but this has the adverse side effect of all source tarballs overriding
themselves in the S3 bucket. Normally this is ok but unfortunately a source
tarball created on Windows is not buildable on Unix.
On Windows the vendored sources contain paths with `\` characters in them which
when interpreted on Unix end up in "file not found" errors.
Instead of this overwriting behavior, whitelist just one linux builder for
producing tarballs and avoid producing tarballs on all other hosts.
Currently we create a source tarball on almost all of the `DEPLOY=1` builders
but this has the adverse side effect of all source tarballs overriding
themselves in the S3 bucket. Normally this is ok but unfortunately a source
tarball created on Windows is not buildable on Unix.
On Windows the vendored sources contain paths with `\` characters in them which
when interpreted on Unix end up in "file not found" errors.
Instead of this overwriting behavior, whitelist just one linux builder for
producing tarballs and avoid producing tarballs on all other hosts.
Port books to mdbook
Part of https://github.com/rust-lang/rust/issues/39588
blocked on https://github.com/rust-lang/rust/pull/39431
As a first step towards the bookshelf, we ~vendor mdbook in-tree and~ port our books to it. Eventually, both of these books will be moved out-of-tree, but the nightly book will rely on doing the same thing. As such, this intermediate step is useful.
r? @alexcrichton @brson
/cc @azerupi
Automate vendoring by invoking cargo-vendor when building src dist tarballs.
This avoids #39633 bringing the `src/vendor` checked into git by #37524, past 200,000 lines of code.
I believe the strategy of having rustbuild run `cargo vendor` during the `dist src` step is sound.
However, the only way to be sure `cargo-vendor` exists is to run `cargo install --force cargo-vendor`, which will recompile it every time (not passing `--force` means you can't tell between "already exists" and "build error"). ~~This is quite suboptimal and I'd like to somehow do it in each `Dockerfile` that would need it.~~
* [ ] Cache `CARGO_HOME` (i.e. `~/.cargo`) between CI runs
* `bin/cargo-vendor` and the actual caches are the relevant bits
* [x] Do not build `cargo-vendor` all the time
* ~~Maybe detect `~/.cargo/bin/cargo-vendor` already exists?~~
* ~~Could also try to build it in a `Dockerfile` but do we have `cargo`/`rustc` there?~~
* Final solution: check `cargo install --list` for a line starting with `cargo-vendor `
cc @rust-lang/tools
mdbook has a lot of optional dependencies that we don't want, so instead
of using it directly, we re-build rustbook to use mdbook as a library.
For convenience' sake, we keep the same CLI interface as mdbook; the
only difference is that it only accepts build and test subcommands,
rather than the full range.
* 'master' of git://github.com/rust-lang/rust: (70 commits)
sanitizer-dylib: only run where std for x86_64-linux is available
travis: Fix build order of dist-x86-linux
fix the sanitizer-dylib test on non x86_64 linux hosts
dist-x86-linux: install newer kernel headers
enable sanitizers on build job that tests x86_64 linux
enable sanitizers on x86_64-linux releases
use helper function in the rebuild logic of the rustc_*san crates
build/test the sanitizers only when --enable-sanitizers is used
sanitizer support
Add missing urls on join_paths
Add test for #27433
Add more examples, get everything passing at last.
Remove some leftover makefiles.
Add more test for rustdoc --test
Rename manifest_version to manifest-version
reference: clarify #[cfg] section
Bump stable release date
rustbuild: Clean build/dist on `make clean`
Add missing urls for current_dir
review nits
...
Delete the makefile build system
This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild.
Rustbuild has been the default build system [since 1.15.0](https://github.com/rust-lang/rust/pull/37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year.
And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!
Add support for test suites emulated in QEMU
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
* Add version info to channel.rs as main.mk is no longer available
* Update `Makefile.in` used with bootstrap to not try to require `mk/util.mk`
* Update the `dist` target to avoid the makefile pieces
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
This commit adds a new tool, `build-manifest`, which is used to generate a
distribution manifest of all produced artifacts. This tool is intended to
replace the `build-rust-manifest.py` script that's currently located on the
buildmaster. The intention is that we'll have a builder which periodically:
* Downloads all artifacts for a commit
* Runs `./x.py dist hash-and-sign`. This will generate `sha256` and `asc` files
as well as TOML manifests.
* Upload all generated hashes and manifests to the directory the artifacts came
from.
* Upload *all* artifacts (tarballs and hashes and manifests) to an archived
location.
* If necessary, upload all artifacts to the main location.
This script is intended to just be the second step here where orchestrating
uploads and such will all happen externally from the build system itself.
This commit adds a new flag to the configure script,
`--enable-extended`, which is intended for specifying a desire to
compile the full suite of Rust tools such as Cargo, the RLS, etc. This
is also an indication that the build system should create combined
installers such as the pkg/exe/msi artifacts.
Currently the `--enable-extended` flag just indicates that combined
installers should be built, and Cargo is itself not compiled just yet
but rather only downloaded from its location. The intention here is to
quickly get to feature parity with the current release process and then
we can start improving it afterwards.
All new files in this PR inside `src/etc/installer` are copied from the
rust-packaging repository.
Fix rustbuild to work with --libdir.
Similar to the makefiles, pass CFG_LIBDIR_RELATIVE to cargo when building
rustc in stages > 0. This tells rustc to check the different directory.
I'm not sure how you want this handled in the toml system (my distribution, Gentoo, uses configure still). I have a feeling the system needs a rework anyways for rustbuild. If there is some discussion that needs to happen, could you merge this in the mean time? I'd be happy to help transition this to a better method.
Make rustbuild force_alloc_system rather than relying on stage0
This 'fixes' jemalloc-less local rebuilds, where we tell cargo that we're actually stage1 (this only fixes the rustbuild path, since I wasn't enthusiastic to dive into the makefiles).
There should be one effect from this PR: `--enable-local-rebuild --disable-jemalloc` will successfully build a stage0 std (rather than erroring). Ideally I think it'd be nice to specify an allocator preference in Cargo.toml/cargo command line (used when an allocator must be picked i.e. dylibs, not rlibs), but since that's not possible we can make do with a force_alloc_system feature. Sadly this locks you into a single allocator in the build libstd, making any eventual implementation of #38575 not quite right in this edge case, but clearly not many people exercise the combination of these two flags.
This PR is also a substitute for #37975 I think. The crucial difference is that the feature name here is distinct from the jemalloc feature (reused in the previous PR) - we don't want someone to be forced into alloc_system just for disabling jemalloc!
Fixes#39054
r? @alexcrichton
