Fix cross compiling dist/build invocations
I am uncertain why the first commit is not affecting CI. I suspect it's because we pass --disable-docs on most of our cross-compilation builders. The second commit doesn't affect CI because CI runs x.py dist, not x.py build.
Both commits are standalone; together they should resolve#76733. The first commit doesn't really fix that issue but rather just fixes cross-compiled x.py dist, resolving a bug introduced in #76549.
The Rust version number is currently embedded in bootstrap's source
code, which makes it hard to update it automatically or access it
outside of ./x.py (as you'd have to parse the source code).
This commit moves the version number to a standalone plaintext file,
which makes accessing or updating it trivial.
The directories for core, alloc, std, proc_macro, and test crates now
correspond directly to the crate name and stripping the "lib" prefix is
no longer necessary.
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
This should run much faster.
There are also some drive-by cleanups here to try to simplify things.
Also, the paths for in-tree crates are now displayed as relative
in `x.py test -h -v`.
On my machine, an error looks like:
Finished release [optimized] target(s) in 0.29s
Opening doc /git/rust/build/x86_64-unknown-linux-gnu/doc/std/index.html
command 'xdg-open (internal)' did not execute successfully; exit code: 4
command stderr:
gio: file:///git/rust/build/x86_64-unknown-linux-gnu/doc/std/index.html: Error when getting information for file “/git/rust/build/x86_64-unknown-linux-gnu/doc/std/index.html”: No such file or directory
Build completed successfully in 0:00:08
build-std compatible sanitizer support
### Motivation
When using `-Z sanitizer=*` feature it is essential that both user code and
standard library is instrumented. Otherwise the utility of sanitizer will be
limited, or its use will be impractical like in the case of memory sanitizer.
The recently introduced cargo feature build-std makes it possible to rebuild
standard library with arbitrary rustc flags. Unfortunately, those changes alone
do not make it easy to rebuild standard library with sanitizers, since runtimes
are dependencies of std that have to be build in specific environment,
generally not available outside rustbuild process. Additionally rebuilding them
requires presence of llvm-config and compiler-rt sources.
The goal of changes proposed here is to make it possible to avoid rebuilding
sanitizer runtimes when rebuilding the std, thus making it possible to
instrument standard library for use with sanitizer with simple, although
verbose command:
```
env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu
```
### Implementation
* Sanitizer runtimes are no long packed into crates. Instead, libraries build
from compiler-rt are used as is, after renaming them into `librusc_rt.*`.
* rustc obtains runtimes from target libdir for default sysroot, so that
they are not required in custom build sysroots created with build-std.
* The runtimes are only linked-in into executables to address issue #64629.
(in previous design it was hard to avoid linking runtimes into static
libraries produced by rustc as demonstrated by sanitizer-staticlib-link
test, which still passes despite changes made in #64780).
cc @kennytm, @japaric, @firstyear, @choller
This commit builds on #65501 continue to simplify the build system and
compiler now that we no longer have multiple LLVM backends to ship by
default. Here this switches the compiler back to what it once was long
long ago, which is linking LLVM directly to the compiler rather than
dynamically loading it at runtime. The `codegen-backends` directory of
the sysroot no longer exists and all relevant support in the build
system is removed. Note that `rustc` still supports a dynamically loaded
codegen backend as it did previously, it just no longer supports
dynamically loaded codegen backends in its own sysroot.
Additionally as part of this the `librustc_codegen_llvm` crate now once
again explicitly depends on all of its crates instead of implicitly
loading them through the sysroot. This involved filling out its
`Cargo.toml` and deleting all the now-unnecessary `extern crate`
annotations in the header of the crate. (this in turn required adding a
number of imports for names of macros too).
The end results of this change are:
* Rustbuild's build process for the compiler as all the "oh don't forget
the codegen backend" checks can be easily removed.
* Building `rustc_codegen_llvm` is much simpler since it's simply
another compiler crate.
* Managing the dependencies of `rustc_codegen_llvm` is much simpler since
it's "just another `Cargo.toml` to edit"
* The build process should be a smidge faster because there's more
parallelism in the main rustc build step rather than splitting
`librustc_codegen_llvm` out to its own step.
* The compiler is expected to be slightly faster by default because the
codegen backend does not need to be dynamically loaded.
* Disabling LLVM as part of rustbuild is still supported, supporting
multiple codegen backends is still supported, and dynamic loading of a
codegen backend is still supported.
This commit changes the return type of `Builder::cargo` to return a
builder that allows dynamically adding more `RUSTFLAGS` values
after-the-fact. While not used yet, this will later be used to delete
more of `rustc.rs`
Previously when building documentation for the standard library we'd
copy all the files 5 times, and these files include libcore/libstd docs
which are huge! This commit instead only copies the files after rustdoc
has been run for each crate, reducing the number of redundant copies
we're making.
Michael Howell
Nixon Enraght-Moony
Eric Huss
Mark Rousskov
Dylan DPC
Joshua Nelson
Mara Bos
12101111
bors
Pietro Albini
mark
Matthias Krüger
Tomasz Miąsko
Jake Goulding
Tamir Duberstein
Manish Goregaokar
Guillaume Gomez
David Tolnay
Nicholas Nethercote
Alex Tokarev
Yuki Okushi
Oliver Middleton
Andy Russell
Alex Crichton
