The new git submodule src/llvm-project is a monorepo replacing src/llvm
and src/tools/{clang,lld,lldb}. This also serves as a rebase for these
projects to the new 8.x branch from trunk.
The src/llvm-emscripten fork is unchanged for now.
This fixes a mistake where miri was accidentally left out of the
build-manifest parsing, meaning that today's nightly generated a
manifest with invalid urls!
When building a distributed compiler on Linux where we use ThinLTO to
create the LLVM shared object this commit switches the compiler to
dynamically linking that LLVM artifact instead of statically linking to
LLVM. The primary goal here is to reduce CI compile times, avoiding two+
ThinLTO builds of all of LLVM. By linking dynamically to LLVM we'll
reuse the one ThinLTO step done by LLVM's build itself.
Lots of discussion about this change can be found [here] and down. A
perf run will show whether this is worth it or not!
[here]: https://github.com/rust-lang/rust/pull/53245#issuecomment-417015334
This commit switches the standard library to using the `backtrace-sys`
crate from crates.io instead of duplicating the logic here in the Rust
repositor with the `backtrace-sys`'s crate's logic.
Eventually this will hopefully be a good step towards using the
`backtrace` crate directly from crates.io itself, but we're not quite
there yet! Hopefully this is a small incremental first step we can take.
When building a distributed compiler on Linux where we use ThinLTO to
create the LLVM shared object this commit switches the compiler to
dynamically linking that LLVM artifact instead of statically linking to
LLVM. The primary goal here is to reduce CI compile times, avoiding two+
ThinLTO builds of all of LLVM. By linking dynamically to LLVM we'll
reuse the one ThinLTO step done by LLVM's build itself.
Lots of discussion about this change can be found [here] and down. A
perf run will show whether this is worth it or not!
[here]: https://github.com/rust-lang/rust/pull/53245#issuecomment-417015334
Ever since we added a Cargo-based build system for the compiler the
standard library has always been a little special, it's never been able
to depend on crates.io crates for runtime dependencies. This has been a
result of various limitations, namely that Cargo doesn't understand that
crates from crates.io depend on libcore, so Cargo tries to build crates
before libcore is finished.
I had an idea this afternoon, however, which lifts the strategy
from #52919 to directly depend on crates.io crates from the standard
library. After all is said and done this removes a whopping three
submodules that we need to manage!
The basic idea here is that for any crate `std` depends on it adds an
*optional* dependency on an empty crate on crates.io, in this case named
`rustc-std-workspace-core`. This crate is overridden via `[patch]` in
this repository to point to a local crate we write, and *that* has a
`path` dependency on libcore.
Note that all `no_std` crates also depend on `compiler_builtins`, but if
we're not using submodules we can publish `compiler_builtins` to
crates.io and all crates can depend on it anyway! The basic strategy
then looks like:
* The standard library (or some transitive dep) decides to depend on a
crate `foo`.
* The standard library adds
```toml
[dependencies]
foo = { version = "0.1", features = ['rustc-dep-of-std'] }
```
* The crate `foo` has an optional dependency on `rustc-std-workspace-core`
* The crate `foo` has an optional dependency on `compiler_builtins`
* The crate `foo` has a feature `rustc-dep-of-std` which activates these
crates and any other necessary infrastructure in the crate.
A sample commit for `dlmalloc` [turns out to be quite simple][commit].
After that all `no_std` crates should largely build "as is" and still be
publishable on crates.io! Notably they should be able to continue to use
stable Rust if necessary, since the `rename-dependency` feature of Cargo
is soon stabilizing.
As a proof of concept, this commit removes the `dlmalloc`,
`libcompiler_builtins`, and `libc` submodules from this repository. Long
thorns in our side these are now gone for good and we can directly
depend on crates.io! It's hoped that in the long term we can bring in
other crates as necessary, but for now this is largely intended to
simply make it easier to manage these crates and remove submodules.
This should be a transparent non-breaking change for all users, but one
possible stickler is that this almost for sure breaks out-of-tree
`std`-building tools like `xargo` and `cargo-xbuild`. I think it should
be relatively easy to get them working, however, as all that's needed is
an entry in the `[patch]` section used to build the standard library.
Hopefully we can work with these tools to solve this problem!
[commit]: 28ee12db81
This commit replaces many usages of `File::open` and reading or writing
with `fs::read_to_string`, `fs::read` and `fs::write`. This reduces code
complexity, and will improve performance for most reads, since the
functions allocate the buffer to be the size of the file.
I believe that this commit will not impact behavior in any way, so some
matches will check the error kind in case the file was not valid UTF-8.
Some of these cases may not actually care about the error.
This commit deletes the `alloc_system` crate from the standard
distribution. This unstable crate is no longer needed in the modern
stable global allocator world, but rather its functionality is folded
directly into the standard library. The standard library was already the
only stable location to access this crate, and as a result this should
not affect any stable code.
Bump cargo-vendor version
Currently we pin `cargo-vendor` to 0.1.4, which doesn't set the `User-Agent` HTTP header. crates.io is going to require that header in the near future, so this PR bumps the pinned version of the crate to the latest one (which correctly sets the header).
r? @Mark-Simulacrum
cc @sgrif
This commit removes all jemalloc related submodules, configuration, etc,
from the bootstrap, from the standard library, and from the compiler.
This will be followed up with a change to use jemalloc specifically as
part of rustc on blessed platforms.
In addition to to updating Cargo's submodule and Cargo's dependencies,
this also updates Cargo's build to build OpenSSL statically into Cargo
as well as libcurl unconditionally. This removes OpenSSL build logic
from the bootstrap code, and otherwise requests that even on OSX we
build curl statically.
We're shipping a rust-enabled lldb, but the "lldb" executable is not
installed into the "bin" directory by rustup. See the discussion in
https://github.com/rust-lang-nursery/rustup.rs/pull/1492 for
background on this decision. There, we agreed to have rust-lldb
prefer the rust-enabled lldb if it is installed.
This patch changes dist.rs to put lldb into rustlib, following what
was done for the other LLVM tools in #53955, and then fixes rust-lldb
to prefer that lldb, if it exists.
See issue #48168
This commit tweaks the layout of a few components that we distribute to
hopefully fix across all platforms the recent issues with LLD being unable to
find the LLVM shared object. In #53245 we switched to building LLVM as a dynamic
library, which means that LLVM tools by default link to LLVM dynamically rather
than statically. This in turn means that the tools, at runtime, need to find the
LLVM shared library.
LLVM's shared library is currently distributed as part of the rustc component.
This library is located, however, at `$sysroot/lib`. The LLVM tools we ship are
in two locations:
* LLD is shipped at `$sysroot/lib/rustlib/$host/bin/rust-lld`
* Other LLVM tools are shipped at `$sysroot/bin`
Each LLVM tool has an embedded rpath directive indicating where it will search
for dynamic libraries. This currently points to `../lib` and is presumably
inserted by LLVM's build system. Unfortunately, though, this directive is only
correct for the LLVM tools at `$sysroot/bin`, not LLD!
This commit is targeted at fixing this situation by making two changes:
* LLVM tools other than LLD are moved in the distribution to
`$sysroot/lib/rustlib/$host/bin`. This moves them next to LLD and should
position them for...
* The LLVM shared object is moved to `$sysroot/lib/rustlib/$host/lib`
Together this means that all tools should natively be able to find the shared
object and the shared object should be installed all the time for the various
tools. Overall this should...
Closes#53813
A recent change (#53245) started to build LLVM with ThinLTO enabled and to
ensure that compile times are kept down it builds LLVM dynamically by default to
ensure that all the various LLVM tools aren't redoing all that optimization
work. This means, however, that all LLVM tools depend on LLVM's dynamic library
by default.
While the LLVM tools and LLDB components were updated to include the shared
library we accidentally forgot about LLD, included with the main rustc
component. LLD also links dynamically to LLVM and ships a non-working binary
right now because of this!
This commit updates our distribution to ship the LLVM dynamic library with the
compiler libraries. While not technically needed for rustc itself to operate
(right now) it may be needed for LLD, and otherwise it serves as a good basis
for the other LLVM tools components to work with as well.
This should...
Closes#53813
This optionally adds lldb (and clang, which it needs) to the build.
Because rust uses LLVM 7, and because clang 7 is not yet released, a
recent git master version of clang is used.
The lldb that is used includes the Rust plugin.
lldb is only built when asked for, or when doing a nightly build on
macOS. Only macOS is done for now due to difficulties with the Python
dependency.
rename rustc's lld to rust-lld
to not shadow the system installed LLD when linking with LLD.
Before:
- `-C linker=lld -Z linker-flavor=ld.lld` uses rustc's LLD
- It's not possible to use a system installed LLD that's named `lld`
With this commit:
- `-C linker=rust-lld -Z linker-flavor=ld.lld` uses rustc's LLD
- `-C linker=lld -Z linker-flavor=ld.lld` uses the system installed LLD
we don't offer guarantees about the availability of LLD in the rustc sysroot so we can rename the tool as long as we don't break the wasm32-unknown-unknown target which depends on it.
r? @alexcrichton we discussed this before
This commit updates the stage0 build of tools to use the libraries of the stage0
compiler instead of the compiled libraries by the stage0 compiler. This should
enable us to avoid any stage0 hacks (like missing SIMD).
Josh Stone
Nicolas Bigaouette
Alex Crichton
Oliver Scherer
Mark Rousskov
kennytm
bors
ljedrz
Alexander Regueiro
Andy Russell
Jethro Beekman
Eduard-Mihai Burtescu
Pietro Albini
Shotaro Yamada
vlad20012
Matthias Krüger
Christian Poveda
Tom Tromey
Michael Woerister
Simon Sapin
Sébastien Marie
Jorge Aparicio