Use custom cargo/rustc paths when parsing flags.
Fixes https://github.com/rust-lang/rust/issues/41779, probably also https://github.com/rust-lang/rust/issues/42543 (I think they're duplicates).
I'm not entirely happy with the implementation, since it means we parse the configuration twice, but it's the minimal solution. I think the other choice is to move both calls to Config::parse inside Flags::parse and merge them, but I don't know if that's a good idea.
r? @alexcrichton
Autogenerate stubs and SUMMARY.md in the unstable book
Removes a speed bump in compiler development by autogenerating stubs for features in the unstable book. See #42454 for discussion.
The PR contains three commits, separated in order to make review easy:
* The first commit converts the tidy tool from a binary crate to a crate that contains both a library and a binary. In the second commit, we'll use the tidy library
* The second and main commit introduces autogeneration of SUMMARY.md and feature stub files
* The third commit turns off the tidy lint that checks for features without a stub, and removes the stub files. A separate commit due to the large number of files touched
Members of the doc team who wish to document some features can either do this (where `$rustsrc` is the root of the rust repo git checkout):
1. cd to `$rustsrc/src/tools/unstable-book-gen` and then do `cargo run $rustsrc/src $rustsrc/src/doc/unstable-book` to put the stubs into the unstable book
2. cd to `$rustsrc` and run `git ls-files --others --exclude-standard` to list the newly added stubs
3. choose a file to edit, then `git add` it and `git commit`
4. afterwards, remove all changes by the tool by doing `git --reset hard` and `git clean -f`
Or they can do this:
1. remove the comment marker in `src/tools/tidy/src/unstable_book.rs` line 122
2. run `./x.py test src/tools/tidy` to list the unstable features which only have stubs
3. revert the change in 1
3. document one of the chosen unstable features
The changes done by this PR also allow for further development:
* tidy obtains information about tracking issues. We can now forbid differing tracking issues between differing `#![unstable]` annotations. I haven't done this but plan to in a future PR
* we now have a general framework for generating stuff for the unstable book at build time. Further changes can autogenerate a list of the API a given library feature exposes.
The old way to simply click through the documentation after it has been uploaded to rust-lang.org works as well.
r? @nagisa
Fixes#42454
Add a travis builder for wasm32-unknown-emscripten
This commits add an entry to travis matrix that will execute wasm32-unknown-emscripten tests suites.
- Emscripten for asmjs was updated to sdk-1.37.13-64bit
- The tests are run with node 8.0.0 (it can execute wasm)
- A wrapper script is used to run each test from the directory where it is (workaround for https://github.com/kripken/emscripten/issues/4542)
- Some tests are ignore, see #42629 and #42630
Build instruction profiler runtime as part of compiler-rt
r? @alexcrichton
This is #38608 with some fixes.
Still missing:
- [x] testing with profiler enabled on some builders (on which ones? Should I add the option to some of the already existing configurations, or create a new configuration?);
- [x] enabling distribution (on which builders?);
- [x] documentation.
bootstrap: improve 'build --help' by explaining what exactly the last example does
I recently found myself confused about what exactly gets built how often when I run which command; I think this would have helped me.
One thing I did not touch, but I feel could also be improved, is the wording just above: "For a quick build with a usable compile, you can pass". I am not a native speaker, but this sounds odd to me. Do you mean "For a quick build of a usable compiler" (but then it should say where the usable compiler is produced)? Or do you mean "For a quick build testing if the compiler is usable"? I can reword this, but I'd like to make sure I understand the intent of the message.
What about
```
For a quick build of a usable compiler, you can pass:
./x.py build --stage 1 src/libtest
This will first build everything once (like --stage 0 without further
arguments would), and then use the compiler built in stage 0 to build
src/libtest and its dependencies.
Once this is done, build/$ARCH/stage1 contains a usable compiler.
```
However, I am not sure this is actually true. In particular, why even bother building the libstd in stage 1? AFAIK that ends up in `build/*/stage1-std`, not in `build/*/stage1` (which is filled from `build/*/stage0-*`).
rustbuild: Remove RUSTFLAGS logic in rustc shim
This was added in #38072 but I can't recall why and AFAIK Cargo already handles
this. This was discovered through #42146 where passing duplicate flags was
causing problems.
rustbuild: Add `./x.py test --no-fail-fast`
This option forwards to each `cargo test` invocation, and applies the
same logic across all test steps to keep going after failures. At the
end, a brief summary line reports how many commands failed, if any.
Note that if a test program fails to even start at all, or if an
auxiliary build command related to testing fails, these are still left
to stop everything right away.
Fixes#40219.
Reduce verbosity of build logs
This does two separate things.
- Sets sccache logging to warn instead of info.
- Makes tests when running for a given PR (not on auto branch) quiet. (cc @eddyb)
r? @alexcrichton
This was added in #38072 but I can't recall why and AFAIK Cargo already handles
this. This was discovered through #42146 where passing duplicate flags was
causing problems.
This option forwards to each `cargo test` invocation, and applies the
same logic across all test steps to keep going after failures. At the
end, a brief summary line reports how many commands failed, if any.
Note that if a test program fails to even start at all, or if an
auxiliary build command related to testing fails, these are still left
to stop everything right away.
Fixes#40219.
Support VS 2017
Fixes#38584
This replaces all the MSVC linker logic with that from the 'gcc' crate. The code looks the same, but there could be regressions.
I've only tested this with x86_64.
r? @alexcrichton
cc @vadimcn @retep998
When `--quiet` is passed to rustbuild, suppress rustdoc test output unless
failure.
Added a `--quiet` flag to `tidy`, which suppresses the features table.
The actual `--quiet` flag is enabled in #42354.
Since details of failed tests will still be printed, and the name of slow
tests taking >60 to runtime will also be printed, the debugging difficulty
caused by information loss should be minimal; but it is very worthwhile to
keep the log under 10000 lines on Travis CI so that common errors can be
spotted without reading the raw log.
* Bring back colors on Travis, which was disabled since #39036.
Append --color=always to cargo when running in CI environment.
* Removed `set -x` in the shell scripts. The `retry` function already
prints which command it is running, add `-x` just add noise to the
output.
* Support travis_fold/travis_time. Matching pairs of these allow Travis CI
to collapse the output in between. This greatly cut down the unnecessary
"successful" output one need to scroll through before finding the failed
statement.
rustbuild: Fix copying duplicate crates into the sysroot
After compiling a project (e.g. libstd, libtest, or librustc) rustbuild needs to
copy over all artifacts into the sysroot of the compiler it's assembling.
Unfortunately rustbuild doesn't know precisely what files to copy! Today it has
a heuristic where it just looks at the most recent version of all files that
look like rlibs/dylibs and copies those over. This unfortunately leads to bugs
with different versions of the same crate as seen in #42261.
This commit updates rustbuild's strategy of copying artifacts to work off the
list of artifacts produced by `cargo build --message-format=json`. The build
system will now parse json messages coming out of Cargo to watch for files being
generated, and then it'll only copy over those precise files.
Note that there's still a bit of weird logic where Cargo prints that it's
creating `libstd.rlib` where we actually want `libstd-xxxxx.rlib`, so we still
do a bit of "most recent file" probing for those. This commit should take care
of the crates.io dependency issues, however, as they're all copied over
precisely.
Closes#42261
Add the RLS to .exe, .msi, and .pkg installers
This directly addresses issue #42157, adding the RLS as a non-default component in the mentioned installers. The windows installers appear to have the right functionality added, but I don't have a machine that runs OSX, so it would be great if someone could test whether my .pkg commit adds the RLS correctly. The final commit also fixes some formatting issues I'd noticed while working on the installers, but I don't know if that's within the scope of this PR, so input would be appreciated.
After compiling a project (e.g. libstd, libtest, or librustc) rustbuild needs to
copy over all artifacts into the sysroot of the compiler it's assembling.
Unfortunately rustbuild doesn't know precisely what files to copy! Today it has
a heuristic where it just looks at the most recent version of all files that
look like rlibs/dylibs and copies those over. This unfortunately leads to bugs
with different versions of the same craet as seen in #42261.
This commit updates rustbuild's strategy of copying artifacts to work off the
list of artifacts produced by `cargo build --message-format=json`. The build
system will now parse json messages coming out of Cargo to watch for files being
generated, and then it'll only copy over those precise files.
Note that there's still a bit of weird logic where Cargo prints that it's
creating `libstd.rlib` where we actually want `libstd-xxxxx.rlib`, so we still
do a bit of "most recent file" probing for those. This commit should take care
of the crates.io dependency issues, however, as they're all copied over
precisely.
Closes#42261
rustbuild: don't create a source tarball when installing
This splits Install out of Dist as it is not a full dist anymore, and creates the source tarball only for the Dist command.
This will allow splitting install in a few rules if we want as it's done for other phases.
Use the improved submodule handling
r? @alexcrichton
That was a crap...
```
Updating submodules
Traceback (most recent call last):
File "./x.py", line 20, in <module>
bootstrap.main()
File "/home/ishitatsuyuki/Documents/rust/src/bootstrap/bootstrap.py", line 684, in main
bootstrap()
File "/home/ishitatsuyuki/Documents/rust/src/bootstrap/bootstrap.py", line 662, in bootstrap
rb.update_submodules()
File "/home/ishitatsuyuki/Documents/rust/src/bootstrap/bootstrap.py", line 566, in update_submodules
path = line[1:].split(' ')[1]
TypeError: a bytes-like object is required, not 'str'
```
Maybe we need to confirm the compatibility of git options, such as `git config` or `git -C` (I believe they existed long before, though). This is tested locally.
