Is it really time? Have our months, no, *years* of suffering come to an end? Are we finally able to cast off the pall of Hoedown? The weight which has dragged us down for so long?
-----
So, timeline for those who need to catch up:
* Way back in December 2016, [we decided we wanted to switch out the markdown renderer](https://github.com/rust-lang/rust/issues/38400). However, this was put on hold because the build system at the time made it difficult to pull in dependencies from crates.io.
* A few months later, in March 2017, [the first PR was done, to switch out the renderers entirely](https://github.com/rust-lang/rust/pull/40338). The PR itself was fraught with CI and build system issues, but eventually landed.
* However, not all was well in the Rustdoc world. During the PR and shortly after, we noticed [some differences in the way the two parsers handled some things](https://github.com/rust-lang/rust/issues/40912), and some of these differences were major enough to break the docs for some crates.
* A couple weeks afterward, [Hoedown was put back in](https://github.com/rust-lang/rust/pull/41290), at this point just to catch tests that Pulldown was "spuriously" running. This would at least provide some warning about spurious tests, rather than just breaking spontaneously.
* However, the problems had created enough noise by this point that just a few days after that, [Hoedown was switched back to the default](https://github.com/rust-lang/rust/pull/41431) while we came up with a solution for properly warning about the differences.
* That solution came a few weeks later, [as a series of warnings when the HTML emitted by the two parsers was semantically different](https://github.com/rust-lang/rust/pull/41991). But that came at a cost, as now rustdoc needed proc-macro support (the new crate needed some custom derives farther down its dependency tree), and the build system was not equipped to handle it at the time. It was worked on for three months as the issue stumped more and more people.
* In that time, [bootstrap was completely reworked](https://github.com/rust-lang/rust/pull/43059) to change how it ordered compilation, and [the method by which it built rustdoc would change](https://github.com/rust-lang/rust/pull/43482), as well. This allowed it to only be built after stage1, when proc-macros would be available, allowing the "rendering differences" PR to finally land.
* The warnings were not perfect, and revealed a few [spurious](https://github.com/rust-lang/rust/pull/44368) [differences](https://github.com/rust-lang/rust/pull/45421) between how we handled the renderers.
* Once these were handled, [we flipped the switch to turn on the "rendering difference" warnings all the time](https://github.com/rust-lang/rust/pull/45324), in October 2017. This began the "warning cycle" for this change, and landed in stable in 1.23, on 2018-01-04.
* Once those warnings hit stable, and after a couple weeks of seeing whether we would get any more reports than what we got from sitting on nightly/beta, [we switched the renderers](https://github.com/rust-lang/rust/pull/47398), making Pulldown the default but still offering the option to use Hoedown.
And that brings us to the present. We haven't received more new issues from this in the meantime, and the "switch by default" is now on beta. Our reasoning is that, at this point, anyone who would have been affected by this has run into it already.
Remove librustdoc dependency on log
This change should have been included in PR #46386.
Since librustdoc doesn't explicitly depend on internal crates
(such as librustc_driver) through its Cargo.toml, it ends up using the
sysroot to resolve them. By removing the dependency on 'log',
we ensure that the syroot is used to resolve `log` as well. This ensures
that only one version of log is in use, so that `env_logger::init()`
enables all uses of `log!` macros.
This change should have been included in PR #46386.
Since librustdoc doesn't explicitly depend on internal crates
(such as librustc_driver) through its Cargo.toml, it ends up using the
sysroot to resolve them. By removing the dependency on 'log',
we ensure that the syroot is used to resolve `log` as well. This ensures
that only one version of log is in use, so that `env_logger::init()`
enables all uses of `log!` macros.
We want librustdoc to pickup the env_logger dependency from
the sysroot. This ensures that the same copy of env_logger is used
for both internal crates (e.g. librustc_driver, libsyntax) and
librustdoc
Closes#46383
Doc tests are temporarily disabled until next release cycle, since
current beta Cargo errors on them. Upgrade should be smooth as the
relevant tests are already fixed in this commit.
[on-demand] Turn monomorphic_const_eval into a proper query, not just a cache.
The error definitions and reporting logic, alongside with `eval_length` were moved to `librustc`.
Both local and cross-crate constant evaluation is on-demand now, but the latter is only used for `enum` discriminants, to replace the manual insertion into the cache which was done when decoding variants.
r? @nikomatsakis
This commit deletes the internal liblog in favor of the implementation that
lives on crates.io. Similarly it's also setting a convention for adding crates
to the compiler. The main restriction right now is that we want compiler
implementation details to be unreachable from normal Rust code (e.g. requires a
feature), and by default everything in the sysroot is reachable via `extern
crate`.
The proposal here is to require that crates pulled in have these lines in their
`src/lib.rs`:
#![cfg_attr(rustbuild, feature(staged_api, rustc_private))]
#![cfg_attr(rustbuild, unstable(feature = "rustc_private", issue = "27812"))]
This'll mean that by default they're not using these attributes but when
compiled as part of the compiler they do a few things:
* Mark themselves as entirely unstable via the `staged_api` feature and the
`#![unstable]` attribute.
* Allow usage of other unstable crates via `feature(rustc_private)` which is
required if the crate relies on any other crates to compile (other than std).
This is to pull in changes to support ARM MUSL targets.
This change also commits a couple of other cargo-generated changes
to other dependencies in the various Cargo.toml files.
rustbuild: Fix dist for non-host targets
The `rust-std` package that we produce is expected to have not only the standard
library but also libtest for compiling unit tests. Unfortunately this does not
currently happen due to the way rustbuild is structured.
There are currently two main stages of compilation in rustbuild, one for the
standard library and one for the compiler. This is primarily done to allow us to
fill in the sysroot right after the standard library has finished compiling to
continue compiling the rest of the crates. Consequently the entire compiler does
not have to explicitly depend on the standard library, and this also should
allow us to pull in crates.io dependencies into the build in the future because
they'll just naturally build against the std we just produced.
These phases, however, do not represent a cross-compiled build. Target-only
builds also require libtest, and libtest is currently part of the
all-encompassing "compiler build". There's unfortunately no way to learn about
just libtest and its dependencies (in a great and robust fashion) so to ensure
that we can copy the right artifacts over this commit introduces a new build
step, libtest.
The new libtest build step has documentation, dist, and link steps as std/rustc
already do. The compiler now depends on libtest instead of libstd, and all
compiler crates can now assume that test and its dependencies are implicitly
part of the sysroot (hence explicit dependencies being removed). This makes the
build a tad less parallel as in theory many rustc crates can be compiled in
parallel with libtest, but this likely isn't where we really need parallelism
either (all the time is still spent in the compiler).
All in all this allows the `dist-std` step to depend on both libstd and libtest,
so `rust-std` packages produced by rustbuild should start having both the
standard library and libtest.
Closes#32523
The `rust-std` package that we produce is expected to have not only the standard
library but also libtest for compiling unit tests. Unfortunately this does not
currently happen due to the way rustbuild is structured.
There are currently two main stages of compilation in rustbuild, one for the
standard library and one for the compiler. This is primarily done to allow us to
fill in the sysroot right after the standard library has finished compiling to
continue compiling the rest of the crates. Consequently the entire compiler does
not have to explicitly depend on the standard library, and this also should
allow us to pull in crates.io dependencies into the build in the future because
they'll just naturally build against the std we just produced.
These phases, however, do not represent a cross-compiled build. Target-only
builds also require libtest, and libtest is currently part of the
all-encompassing "compiler build". There's unfortunately no way to learn about
just libtest and its dependencies (in a great and robust fashion) so to ensure
that we can copy the right artifacts over this commit introduces a new build
step, libtest.
The new libtest build step has documentation, dist, and link steps as std/rustc
already do. The compiler now depends on libtest instead of libstd, and all
compiler crates can now assume that test and its dependencies are implicitly
part of the sysroot (hence explicit dependencies being removed). This makes the
build a tad less parallel as in theory many rustc crates can be compiled in
parallel with libtest, but this likely isn't where we really need parallelism
either (all the time is still spent in the compiler).
All in all this allows the `dist-std` step to depend on both libstd and libtest,
so `rust-std` packages produced by rustbuild should start having both the
standard library and libtest.
Closes#32523
Guillaume Gomez
bors
Mark Simulacrum
Manish Goregaokar
Aaron Hill
Irina-Gabriela Popa
Ariel Ben-Yehuda
QuietMisdreavus
Marco A L Barbosa
Alex Crichton
Brian Anderson
Niko Matsakis
Corey Farwell
Eduard-Mihai Burtescu
Michael Woerister
Timon Van Overveldt
Oliver Schneider
Jonathan Turner