While we're at it update the `backtrace` crate from crates.io. It turns out that
the submodule's configure script has gotten a lot more finnicky as of late so
also switch over to using the `cc` crate manually which allows to avoid some
hacks around the configure script as well
Currently on CI we predominately compile LLVM with the default system compiler
which means gcc on Linux, some version of Clang on OSX, MSVC on Windows, and
gcc on MinGW. This commit switches Linux, OSX, and Windows to all use Clang
6.0.0 to build LLVM (aka the C/C++ compiler as part of the bootstrap). This
looks to generate faster code according to #49879 which translates to a faster
rustc (as LLVM internally is faster)
The major changes here were to the containers that build Linux releases,
basically adding a new step that uses the previous gcc 4.8 compiler to compile
the next Clang 6.0.0 compiler. Otherwise the OSX and Windows scripts have been
updated to download precompiled versions of Clang 6 and configure the build to
use them.
Note that `cc` was updated here to fix using `clang-cl` with `cc-rs` on MSVC, as
well as an update to `sccache` on Windows which was needed to correctly work
with `clang-cl`. Finally the MinGW compiler is entirely left out here
intentionally as it's currently thought that Clang can't generate C++ code for
MinGW and we need to use gcc, but this should be verified eventually.
This commit adds a dedicated mode to compiletest for running rustfix tests,
adding a new `src/test/rustfix` directory which will execute all tests as a
"rustfix" test, namely requiring that a `*.fixed` is next to the main file which
is the result of the rustfix project's application of fixes.
The `rustfix` crate is pulled in to actually perform the fixing, and the rustfix
compiletest mode will assert a few properties about the fixing:
* The expected fixed output must be the same as rustc's output suggestions
applied to the original code.
* The fixed code must compile successfully
* The fixed code must have no further diagnostics emitted about it
In order to run tests, previous commits have cfg'd out various parts of
rustbuild. Generally speaking, these are filesystem-related operations
and process-spawning related parts. Then, rustbuild is run "as normal"
and the various steps that where run are retrieved from the cache and
checked against the expected results.
Note that this means that the current implementation primarily tests
"what" we build, but doesn't actually test that what we build *will*
build. In other words, it doesn't do any form of dependency verification
for any crate. This is possible to implement, but is considered future
work.
This implementation strives to cfg out as little code as possible; it
also does not currently test anywhere near all of rustbuild. The current
tests are also not checked for "correctness," rather, they simply
represent what we do as of this commit, which may be wrong.
Test cases are drawn from the old implementation of rustbuild, though
the expected results may vary.
Also update some `Cargo.lock` dependencies, finishing up some final steps of our
[release process]!
This doesn't update the bootstrap compiler just yet but that will come in a
follow-up PR.
[release process]: https://forge.rust-lang.org/release-process.html
This commit updates the `ToolBuild` step to stream Cargo's JSON messages, parse
them, and record all libraries built. If we build anything twice (aka Cargo)
it'll most likely happen due to dependencies being recompiled which is caught by
this check.
This should fix regressions in Cargo after swithing to clap:
* If an external subcommand name was close to built-in one, clap
errored (fixed by updating clap version)
* External subcomands didn't received their name as a first arg
Required moving all fulldeps tests depending on `rand` to different locations as
now there's multiple `rand` crates that can't be implicitly linked against.
rustc: Migrate to `termcolor` crate from `term`
This crate moves the compiler's error reporting to using the `termcolor` crate
from crates.io. Previously rustc used a super-old version of the `term` crate
in-tree which is basically unmaintained at this point, but Cargo has been using
`termcolor` for some time now and tools like `rg` are using `termcolor` as well,
so it seems like a good strategy to take!
Note that the `term` crate remains in-tree for libtest. Changing libtest will be
a bit tricky due to how the build works, but we can always tackle that later.
cc #45728
This crate moves the compiler's error reporting to using the `termcolor` crate
from crates.io. Previously rustc used a super-old version of the `term` crate
in-tree which is basically unmaintained at this point, but Cargo has been using
`termcolor` for some time now and tools like `rg` are using `termcolor` as well,
so it seems like a good strategy to take!
Note that the `term` crate remains in-tree for libtest. Changing libtest will be
a bit tricky due to how the build works, but we can always tackle that later.
cc #45728
It looks like this cuts down on the number of dependencies in env_logger and
notably cuts out a difference between a shared dependency of rls/cargo. My goal
here is to ensure that when we compile the RLS/Cargo on CI we only compile Cargo
once, and this is one step towards that!
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
These changes were meant to be in
2b18d8fe9dc05415a8e6b7cadf879c7f7ebe020a (rebased from
12a230562ece9b0d29018a436676141054dc53b7), but I messed up the rebase a
bit as the file had been moved.
Is it really time? Have our months, no, *years* of suffering come to an end? Are we finally able to cast off the pall of Hoedown? The weight which has dragged us down for so long?
-----
So, timeline for those who need to catch up:
* Way back in December 2016, [we decided we wanted to switch out the markdown renderer](https://github.com/rust-lang/rust/issues/38400). However, this was put on hold because the build system at the time made it difficult to pull in dependencies from crates.io.
* A few months later, in March 2017, [the first PR was done, to switch out the renderers entirely](https://github.com/rust-lang/rust/pull/40338). The PR itself was fraught with CI and build system issues, but eventually landed.
* However, not all was well in the Rustdoc world. During the PR and shortly after, we noticed [some differences in the way the two parsers handled some things](https://github.com/rust-lang/rust/issues/40912), and some of these differences were major enough to break the docs for some crates.
* A couple weeks afterward, [Hoedown was put back in](https://github.com/rust-lang/rust/pull/41290), at this point just to catch tests that Pulldown was "spuriously" running. This would at least provide some warning about spurious tests, rather than just breaking spontaneously.
* However, the problems had created enough noise by this point that just a few days after that, [Hoedown was switched back to the default](https://github.com/rust-lang/rust/pull/41431) while we came up with a solution for properly warning about the differences.
* That solution came a few weeks later, [as a series of warnings when the HTML emitted by the two parsers was semantically different](https://github.com/rust-lang/rust/pull/41991). But that came at a cost, as now rustdoc needed proc-macro support (the new crate needed some custom derives farther down its dependency tree), and the build system was not equipped to handle it at the time. It was worked on for three months as the issue stumped more and more people.
* In that time, [bootstrap was completely reworked](https://github.com/rust-lang/rust/pull/43059) to change how it ordered compilation, and [the method by which it built rustdoc would change](https://github.com/rust-lang/rust/pull/43482), as well. This allowed it to only be built after stage1, when proc-macros would be available, allowing the "rendering differences" PR to finally land.
* The warnings were not perfect, and revealed a few [spurious](https://github.com/rust-lang/rust/pull/44368) [differences](https://github.com/rust-lang/rust/pull/45421) between how we handled the renderers.
* Once these were handled, [we flipped the switch to turn on the "rendering difference" warnings all the time](https://github.com/rust-lang/rust/pull/45324), in October 2017. This began the "warning cycle" for this change, and landed in stable in 1.23, on 2018-01-04.
* Once those warnings hit stable, and after a couple weeks of seeing whether we would get any more reports than what we got from sitting on nightly/beta, [we switched the renderers](https://github.com/rust-lang/rust/pull/47398), making Pulldown the default but still offering the option to use Hoedown.
And that brings us to the present. We haven't received more new issues from this in the meantime, and the "switch by default" is now on beta. Our reasoning is that, at this point, anyone who would have been affected by this has run into it already.
Use time crate in bootstrap dist instead of date
`bootstrap dist` command is trying to run *NIX specific `date` command to get current month and year. This command keep failing when it's called on a Windows command prompt. This patch is making it use time crate.
Closes: #47908
Building on the work of # 45684 this commit updates the compiler to
unconditionally load the `rustc_trans` crate at runtime instead of linking to it
at compile time. The end goal of this work is to implement # 46819 where rustc
will have multiple backends available to it to load.
This commit starts off by removing the `extern crate rustc_trans` from the
driver. This involved moving some miscellaneous functionality into the
`TransCrate` trait and also required an implementation of how to locate and load
the trans backend. This ended up being a little tricky because the sysroot isn't
always the right location (for example `--sysroot` arguments) so some extra code
was added as well to probe a directory relative to the current dll (the
rustc_driver dll).
Rustbuild has been updated accordingly as well to have a separate compilation
invocation for the `rustc_trans` crate and assembly it accordingly into the
sysroot. Finally, the distribution logic for the `rustc` package was also
updated to slurp up the trans backends folder.
A number of assorted fallout changes were included here as well to ensure tests
pass and such, and they should all be commented inline.
Check for deadlinks from the summary during book generation
Previously, any deadlinks from a book's SUMMARY.md wouldn't
cause any errors or warnings or similar but mdbook would simply
create a page with blank content.
This has kept bug #47394 hidden. It should have been detected
back in the PR when those wrongly named files got added to the
book.
PR #47414 was one component of the solution. This change
is a second line of defense for the unstable book and a first
line of defense for any other book.
We also update mdbook to the most recent version.
Update rust-installer for streaming parallelism
Pull in rust-lang/rust-installer#76 to get streamed tarball generation,
rather than batching it all in memory, while still getting the benefit
of compressing in parallel.
Pull in rust-lang/rust-installer#76 to get streamed tarball generation,
rather than batching it all in memory, while still getting the benefit
of compressing in parallel.
Previously, any deadlinks from a book's SUMMARY.md wouldn't
cause any errors or warnings or similar but mdbook would simply
create a page with blank content.
This has kept bug #47394 hidden. It should have been detected
back in the PR when those wrongly named files got added to the
book.
PR #47414 was one component of the solution. This change
is a second line of defense for the unstable book and a first
line of defense for any other book.
We also update mdbook to the most recent version.
Update `rand` crate to `0.3.19`.
Update `log` crate to `0.3.9` and `0.4.1`.
Update `parking_lot_core` crate to `0.2.9`.
Upgrade all flate2 dependencies to `1.0.1`.
- Update `rust-installer` submodule.
Bastien Orivel
kennytm
Nick Cameron
John Kåre Alsaker
Mark Simulacrum
Vytautas Astrauskas
Alex Crichton
Niko Matsakis
Santiago Pastorino
Douglas Campos
Mateusz Mikuła
Pascal Hertleif
Oliver Schneider
Irina Popa
Eric Huss
Guillaume Gomez
Oliver Schneider
Anthony Ramine
Alex Butler
Oliver Schneider
Matt Ickstadt
Steven Fackler
Simon Sapin
steveklabnik
Boris-Chengbiao Zhou
Manish Goregaokar
Aleksey Kladov
Taylor Cramer
Alan Du
Mark Mansi
Sean Griffin
Martin Algesten
Onur Aslan
topecongiro
bjorn3
Josh Stone
est31
Ed Schouten
Malo Jaffré
Felix Schütt
Igor Matuszewski
David Henningsson
Maik Klein
Andy Russell
Esteban Küber
Vadim Petrochenkov
