2020-02-24 19:09:44 +01:00
|
|
|
//! Checks the licenses of third-party dependencies.
|
2016-12-11 00:27:42 +01:00
|
|
|
|
2020-02-24 22:47:49 +01:00
|
|
|
use cargo_metadata::{Metadata, Package, PackageId, Resolve};
|
2020-02-24 19:09:44 +01:00
|
|
|
use std::collections::{BTreeSet, HashSet};
|
2016-12-11 00:27:42 +01:00
|
|
|
use std::path::Path;
|
2018-02-23 02:52:56 +01:00
|
|
|
|
2020-02-24 22:52:08 +01:00
|
|
|
/// These are licenses that are allowed for all crates, including the runtime,
|
|
|
|
/// rustc, tools, etc.
|
2018-09-02 01:46:57 +02:00
|
|
|
const LICENSES: &[&str] = &[
|
2017-02-09 20:44:39 +01:00
|
|
|
"MIT/Apache-2.0",
|
2017-02-16 00:57:06 +01:00
|
|
|
"MIT / Apache-2.0",
|
2017-02-09 20:44:39 +01:00
|
|
|
"Apache-2.0/MIT",
|
2017-07-18 23:26:55 +02:00
|
|
|
"Apache-2.0 / MIT",
|
2017-02-09 20:44:39 +01:00
|
|
|
"MIT OR Apache-2.0",
|
2019-05-04 21:20:24 +02:00
|
|
|
"Apache-2.0 OR MIT",
|
2019-08-29 19:22:24 +02:00
|
|
|
"Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT", // wasi license
|
2017-02-09 20:44:39 +01:00
|
|
|
"MIT",
|
|
|
|
"Unlicense/MIT",
|
2018-07-26 02:25:12 +02:00
|
|
|
"Unlicense OR MIT",
|
std: Switch from libbacktrace to gimli
This commit is a proof-of-concept for switching the standard library's
backtrace symbolication mechanism on most platforms from libbacktrace to
gimli. The standard library's support for `RUST_BACKTRACE=1` requires
in-process parsing of object files and DWARF debug information to
interpret it and print the filename/line number of stack frames as part
of a backtrace.
Historically this support in the standard library has come from a
library called "libbacktrace". The libbacktrace library seems to have
been extracted from gcc at some point and is written in C. We've had a
lot of issues with libbacktrace over time, unfortunately, though. The
library does not appear to be actively maintained since we've had
patches sit for months-to-years without comments. We have discovered a
good number of soundness issues with the library itself, both when
parsing valid DWARF as well as invalid DWARF. This is enough of an issue
that the libs team has previously decided that we cannot feed untrusted
inputs to libbacktrace. This also doesn't take into account the
portability of libbacktrace which has been difficult to manage and
maintain over time. While possible there are lots of exceptions and it's
the main C dependency of the standard library right now.
For years it's been the desire to switch over to a Rust-based solution
for symbolicating backtraces. It's been assumed that we'll be using the
Gimli family of crates for this purpose, which are targeted at safely
and efficiently parsing DWARF debug information. I've been working
recently to shore up the Gimli support in the `backtrace` crate. As of a
few weeks ago the `backtrace` crate, by default, uses Gimli when loaded
from crates.io. This transition has gone well enough that I figured it
was time to start talking seriously about this change to the standard
library.
This commit is a preview of what's probably the best way to integrate
the `backtrace` crate into the standard library with the Gimli feature
turned on. While today it's used as a crates.io dependency, this commit
switches the `backtrace` crate to a submodule of this repository which
will need to be updated manually. This is not done lightly, but is
thought to be the best solution. The primary reason for this is that the
`backtrace` crate needs to do some pretty nontrivial filesystem
interactions to locate debug information. Working without `std::fs` is
not an option, and while it might be possible to do some sort of
trait-based solution when prototyped it was found to be too unergonomic.
Using a submodule allows the `backtrace` crate to build as a submodule
of the `std` crate itself, enabling it to use `std::fs` and such.
Otherwise this adds new dependencies to the standard library. This step
requires extra attention because this means that these crates are now
going to be included with all Rust programs by default. It's important
to note, however, that we're already shipping libbacktrace with all Rust
programs by default and it has a bunch of C code implementing all of
this internally anyway, so we're basically already switching
already-shipping functionality to Rust from C.
* `object` - this crate is used to parse object file headers and
contents. Very low-level support is used from this crate and almost
all of it is disabled. Largely we're just using struct definitions as
well as convenience methods internally to read bytes and such.
* `addr2line` - this is the main meat of the implementation for
symbolication. This crate depends on `gimli` for DWARF parsing and
then provides interfaces needed by the `backtrace` crate to turn an
address into a filename / line number. This crate is actually pretty
small (fits in a single file almost!) and mirrors most of what
`dwarf.c` does for libbacktrace.
* `miniz_oxide` - the libbacktrace crate transparently handles
compressed debug information which is compressed with zlib. This crate
is used to decompress compressed debug sections.
* `gimli` - not actually used directly, but a dependency of `addr2line`.
* `adler32`- not used directly either, but a dependency of
`miniz_oxide`.
The goal of this change is to improve the safety of backtrace
symbolication in the standard library, especially in the face of
possibly malformed DWARF debug information. Even to this day we're still
seeing segfaults in libbacktrace which could possibly become security
vulnerabilities. This change should almost entirely eliminate this
possibility whilc also paving the way forward to adding more features
like split debug information.
Some references for those interested are:
* Original addition of libbacktrace - #12602
* OOM with libbacktrace - #24231
* Backtrace failure due to use of uninitialized value - #28447
* Possibility to feed untrusted data to libbacktrace - #21889
* Soundness fix for libbacktrace - #33729
* Crash in libbacktrace - #39468
* Support for macOS, never merged - ianlancetaylor/libbacktrace#2
* Performance issues with libbacktrace - #29293, #37477
* Update procedure is quite complicated due to how many patches we
need to carry - #50955
* Libbacktrace doesn't work on MinGW with dynamic libs - #71060
* Segfault in libbacktrace on macOS - #71397
Switching to Rust will not make us immune to all of these issues. The
crashes are expected to go away, but correctness and performance may
still have bugs arise. The gimli and `backtrace` crates, however, are
actively maintained unlike libbacktrace, so this should enable us to at
least efficiently apply fixes as situations come up.
2020-05-13 23:22:37 +02:00
|
|
|
"0BSD OR MIT OR Apache-2.0", // adler license
|
2020-08-18 01:07:09 +02:00
|
|
|
"Zlib OR Apache-2.0 OR MIT", // tinyvec
|
2017-02-09 20:44:39 +01:00
|
|
|
];
|
|
|
|
|
2018-02-27 06:45:04 +01:00
|
|
|
/// These are exceptions to Rust's permissive licensing policy, and
|
|
|
|
/// should be considered bugs. Exceptions are only allowed in Rust
|
|
|
|
/// tooling. It is _crucial_ that no exception crates be dependencies
|
2019-01-11 17:32:31 +01:00
|
|
|
/// of the Rust runtime (std/test).
|
2020-02-24 21:54:32 +01:00
|
|
|
const EXCEPTIONS: &[(&str, &str)] = &[
|
2020-08-17 21:13:04 +02:00
|
|
|
("mdbook", "MPL-2.0"), // mdbook
|
|
|
|
("openssl", "Apache-2.0"), // cargo, mdbook
|
|
|
|
("fuchsia-zircon-sys", "BSD-3-Clause"), // rustdoc, rustc, cargo
|
|
|
|
("fuchsia-zircon", "BSD-3-Clause"), // rustdoc, rustc, cargo (jobserver & tempdir)
|
|
|
|
("colored", "MPL-2.0"), // rustfmt
|
|
|
|
("ordslice", "Apache-2.0"), // rls
|
|
|
|
("cloudabi", "BSD-2-Clause"), // (rls -> crossbeam-channel 0.2 -> rand 0.5)
|
|
|
|
("ryu", "Apache-2.0 OR BSL-1.0"), // rls/cargo/... (because of serde)
|
|
|
|
("bytesize", "Apache-2.0"), // cargo
|
|
|
|
("im-rc", "MPL-2.0+"), // cargo
|
|
|
|
("constant_time_eq", "CC0-1.0"), // rustfmt
|
|
|
|
("sized-chunks", "MPL-2.0+"), // cargo via im-rc
|
|
|
|
("bitmaps", "MPL-2.0+"), // cargo via im-rc
|
|
|
|
("crossbeam-queue", "MIT/Apache-2.0 AND BSD-2-Clause"), // rls via rayon
|
|
|
|
("arrayref", "BSD-2-Clause"), // cargo-miri/directories/.../rust-argon2 (redox)
|
|
|
|
("instant", "BSD-3-Clause"), // rustc_driver/tracing-subscriber/parking_lot
|
2020-08-21 01:16:30 +02:00
|
|
|
("snap", "BSD-3-Clause"), // rustc
|
2019-06-21 17:35:48 +02:00
|
|
|
// FIXME: this dependency violates the documentation comment above:
|
2020-02-24 21:54:32 +01:00
|
|
|
("fortanix-sgx-abi", "MPL-2.0"), // libstd but only for `sgx` target
|
2016-12-11 00:27:42 +01:00
|
|
|
];
|
|
|
|
|
2020-02-24 22:47:49 +01:00
|
|
|
/// These are the root crates that are part of the runtime. The licenses for
|
|
|
|
/// these and all their dependencies *must not* be in the exception list.
|
2020-02-25 16:28:46 +01:00
|
|
|
const RUNTIME_CRATES: &[&str] = &["std", "core", "alloc", "test", "panic_abort", "panic_unwind"];
|
2020-02-24 22:47:49 +01:00
|
|
|
|
2020-07-07 17:12:44 +02:00
|
|
|
/// Crates whose dependencies must be explicitly permitted.
|
|
|
|
const RESTRICTED_DEPENDENCY_CRATES: &[&str] = &["rustc_middle", "rustc_codegen_llvm"];
|
2018-02-27 06:45:04 +01:00
|
|
|
|
2020-07-07 17:12:44 +02:00
|
|
|
/// Crates rustc is allowed to depend on. Avoid adding to the list if possible.
|
2020-02-24 22:52:08 +01:00
|
|
|
///
|
|
|
|
/// This list is here to provide a speed-bump to adding a new dependency to
|
|
|
|
/// rustc. Please check with the compiler team before adding an entry.
|
2020-07-07 17:12:44 +02:00
|
|
|
const PERMITTED_DEPENDENCIES: &[&str] = &[
|
std: Switch from libbacktrace to gimli
This commit is a proof-of-concept for switching the standard library's
backtrace symbolication mechanism on most platforms from libbacktrace to
gimli. The standard library's support for `RUST_BACKTRACE=1` requires
in-process parsing of object files and DWARF debug information to
interpret it and print the filename/line number of stack frames as part
of a backtrace.
Historically this support in the standard library has come from a
library called "libbacktrace". The libbacktrace library seems to have
been extracted from gcc at some point and is written in C. We've had a
lot of issues with libbacktrace over time, unfortunately, though. The
library does not appear to be actively maintained since we've had
patches sit for months-to-years without comments. We have discovered a
good number of soundness issues with the library itself, both when
parsing valid DWARF as well as invalid DWARF. This is enough of an issue
that the libs team has previously decided that we cannot feed untrusted
inputs to libbacktrace. This also doesn't take into account the
portability of libbacktrace which has been difficult to manage and
maintain over time. While possible there are lots of exceptions and it's
the main C dependency of the standard library right now.
For years it's been the desire to switch over to a Rust-based solution
for symbolicating backtraces. It's been assumed that we'll be using the
Gimli family of crates for this purpose, which are targeted at safely
and efficiently parsing DWARF debug information. I've been working
recently to shore up the Gimli support in the `backtrace` crate. As of a
few weeks ago the `backtrace` crate, by default, uses Gimli when loaded
from crates.io. This transition has gone well enough that I figured it
was time to start talking seriously about this change to the standard
library.
This commit is a preview of what's probably the best way to integrate
the `backtrace` crate into the standard library with the Gimli feature
turned on. While today it's used as a crates.io dependency, this commit
switches the `backtrace` crate to a submodule of this repository which
will need to be updated manually. This is not done lightly, but is
thought to be the best solution. The primary reason for this is that the
`backtrace` crate needs to do some pretty nontrivial filesystem
interactions to locate debug information. Working without `std::fs` is
not an option, and while it might be possible to do some sort of
trait-based solution when prototyped it was found to be too unergonomic.
Using a submodule allows the `backtrace` crate to build as a submodule
of the `std` crate itself, enabling it to use `std::fs` and such.
Otherwise this adds new dependencies to the standard library. This step
requires extra attention because this means that these crates are now
going to be included with all Rust programs by default. It's important
to note, however, that we're already shipping libbacktrace with all Rust
programs by default and it has a bunch of C code implementing all of
this internally anyway, so we're basically already switching
already-shipping functionality to Rust from C.
* `object` - this crate is used to parse object file headers and
contents. Very low-level support is used from this crate and almost
all of it is disabled. Largely we're just using struct definitions as
well as convenience methods internally to read bytes and such.
* `addr2line` - this is the main meat of the implementation for
symbolication. This crate depends on `gimli` for DWARF parsing and
then provides interfaces needed by the `backtrace` crate to turn an
address into a filename / line number. This crate is actually pretty
small (fits in a single file almost!) and mirrors most of what
`dwarf.c` does for libbacktrace.
* `miniz_oxide` - the libbacktrace crate transparently handles
compressed debug information which is compressed with zlib. This crate
is used to decompress compressed debug sections.
* `gimli` - not actually used directly, but a dependency of `addr2line`.
* `adler32`- not used directly either, but a dependency of
`miniz_oxide`.
The goal of this change is to improve the safety of backtrace
symbolication in the standard library, especially in the face of
possibly malformed DWARF debug information. Even to this day we're still
seeing segfaults in libbacktrace which could possibly become security
vulnerabilities. This change should almost entirely eliminate this
possibility whilc also paving the way forward to adding more features
like split debug information.
Some references for those interested are:
* Original addition of libbacktrace - #12602
* OOM with libbacktrace - #24231
* Backtrace failure due to use of uninitialized value - #28447
* Possibility to feed untrusted data to libbacktrace - #21889
* Soundness fix for libbacktrace - #33729
* Crash in libbacktrace - #39468
* Support for macOS, never merged - ianlancetaylor/libbacktrace#2
* Performance issues with libbacktrace - #29293, #37477
* Update procedure is quite complicated due to how many patches we
need to carry - #50955
* Libbacktrace doesn't work on MinGW with dynamic libs - #71060
* Segfault in libbacktrace on macOS - #71397
Switching to Rust will not make us immune to all of these issues. The
crashes are expected to go away, but correctness and performance may
still have bugs arise. The gimli and `backtrace` crates, however, are
actively maintained unlike libbacktrace, so this should enable us to at
least efficiently apply fixes as situations come up.
2020-05-13 23:22:37 +02:00
|
|
|
"addr2line",
|
|
|
|
"adler",
|
2020-02-24 19:09:44 +01:00
|
|
|
"aho-corasick",
|
|
|
|
"annotate-snippets",
|
|
|
|
"ansi_term",
|
|
|
|
"arrayvec",
|
|
|
|
"atty",
|
|
|
|
"autocfg",
|
|
|
|
"bitflags",
|
2020-03-31 07:17:15 +02:00
|
|
|
"block-buffer",
|
|
|
|
"block-padding",
|
2020-02-24 19:09:44 +01:00
|
|
|
"byteorder",
|
2020-08-17 21:13:04 +02:00
|
|
|
"byte-tools",
|
2020-02-24 19:09:44 +01:00
|
|
|
"cc",
|
|
|
|
"cfg-if",
|
2020-03-03 17:25:03 +01:00
|
|
|
"chalk-derive",
|
|
|
|
"chalk-ir",
|
2020-02-24 19:09:44 +01:00
|
|
|
"cloudabi",
|
|
|
|
"cmake",
|
|
|
|
"compiler_builtins",
|
2020-10-13 17:41:06 +02:00
|
|
|
"cpuid-bool",
|
2020-02-24 19:09:44 +01:00
|
|
|
"crc32fast",
|
|
|
|
"crossbeam-deque",
|
|
|
|
"crossbeam-epoch",
|
|
|
|
"crossbeam-queue",
|
|
|
|
"crossbeam-utils",
|
2021-02-13 12:17:15 +01:00
|
|
|
"cstr",
|
2020-02-24 19:09:44 +01:00
|
|
|
"datafrog",
|
Introduce expect snapshot testing library into rustc
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
![expect](https://user-images.githubusercontent.com/1711539/90888810-3bcc8180-e3b7-11ea-9626-d06e89e1a0bb.gif)
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See https://github.com/rust-analyzer/rust-analyzer/pull/5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
2020-08-21 14:03:50 +02:00
|
|
|
"difference",
|
2020-03-31 07:17:15 +02:00
|
|
|
"digest",
|
2020-02-24 19:09:44 +01:00
|
|
|
"dlmalloc",
|
|
|
|
"either",
|
|
|
|
"ena",
|
|
|
|
"env_logger",
|
Introduce expect snapshot testing library into rustc
Snapshot testing is a technique for writing maintainable unit tests.
Unlike usual `assert_eq!` tests, snapshot tests allow
to *automatically* upgrade expected values on test failure.
In a sense, snapshot tests are inline-version of our beloved
UI-tests.
Example:
![expect](https://user-images.githubusercontent.com/1711539/90888810-3bcc8180-e3b7-11ea-9626-d06e89e1a0bb.gif)
A particular library we use, `expect_test` provides an `expect!`
macro, which creates a sort of self-updating string literal (by using
`file!` macro). Self-update is triggered by setting `UPDATE_EXPECT`
environmental variable (this info is printed during the test failure).
This library was extracted from rust-analyzer, where we use it for
most of our tests.
There are some other, more popular snapshot testing libraries:
* https://github.com/mitsuhiko/insta
* https://github.com/aaronabramov/k9
The main differences of `expect` are:
* first-class snapshot objects (so, tests can be written as functions,
rather than as macros)
* focus on inline-snapshots (but file snapshots are also supported)
* restricted feature set (only `assert_eq` and `assert_debug_eq`)
* no extra runtime (ie, no `cargo insta`)
See https://github.com/rust-analyzer/rust-analyzer/pull/5101 for a
an extended comparison.
It is unclear if this testing style will stick with rustc in the long
run. At the moment, rustc is mainly tested via integrated UI tests.
But in the library-ified world, unit-tests will become somewhat more
important (that's why use use `rustc_lexer` library-ified library as
an example in this PR). Given that the cost of removal shouldn't be
too high, it probably makes sense to just see if this flies!
2020-08-21 14:03:50 +02:00
|
|
|
"expect-test",
|
2020-03-31 07:17:15 +02:00
|
|
|
"fake-simd",
|
2020-02-24 19:09:44 +01:00
|
|
|
"filetime",
|
|
|
|
"flate2",
|
|
|
|
"fortanix-sgx-abi",
|
|
|
|
"fuchsia-zircon",
|
|
|
|
"fuchsia-zircon-sys",
|
2020-03-31 07:17:15 +02:00
|
|
|
"generic-array",
|
2020-02-24 19:09:44 +01:00
|
|
|
"getopts",
|
|
|
|
"getrandom",
|
std: Switch from libbacktrace to gimli
This commit is a proof-of-concept for switching the standard library's
backtrace symbolication mechanism on most platforms from libbacktrace to
gimli. The standard library's support for `RUST_BACKTRACE=1` requires
in-process parsing of object files and DWARF debug information to
interpret it and print the filename/line number of stack frames as part
of a backtrace.
Historically this support in the standard library has come from a
library called "libbacktrace". The libbacktrace library seems to have
been extracted from gcc at some point and is written in C. We've had a
lot of issues with libbacktrace over time, unfortunately, though. The
library does not appear to be actively maintained since we've had
patches sit for months-to-years without comments. We have discovered a
good number of soundness issues with the library itself, both when
parsing valid DWARF as well as invalid DWARF. This is enough of an issue
that the libs team has previously decided that we cannot feed untrusted
inputs to libbacktrace. This also doesn't take into account the
portability of libbacktrace which has been difficult to manage and
maintain over time. While possible there are lots of exceptions and it's
the main C dependency of the standard library right now.
For years it's been the desire to switch over to a Rust-based solution
for symbolicating backtraces. It's been assumed that we'll be using the
Gimli family of crates for this purpose, which are targeted at safely
and efficiently parsing DWARF debug information. I've been working
recently to shore up the Gimli support in the `backtrace` crate. As of a
few weeks ago the `backtrace` crate, by default, uses Gimli when loaded
from crates.io. This transition has gone well enough that I figured it
was time to start talking seriously about this change to the standard
library.
This commit is a preview of what's probably the best way to integrate
the `backtrace` crate into the standard library with the Gimli feature
turned on. While today it's used as a crates.io dependency, this commit
switches the `backtrace` crate to a submodule of this repository which
will need to be updated manually. This is not done lightly, but is
thought to be the best solution. The primary reason for this is that the
`backtrace` crate needs to do some pretty nontrivial filesystem
interactions to locate debug information. Working without `std::fs` is
not an option, and while it might be possible to do some sort of
trait-based solution when prototyped it was found to be too unergonomic.
Using a submodule allows the `backtrace` crate to build as a submodule
of the `std` crate itself, enabling it to use `std::fs` and such.
Otherwise this adds new dependencies to the standard library. This step
requires extra attention because this means that these crates are now
going to be included with all Rust programs by default. It's important
to note, however, that we're already shipping libbacktrace with all Rust
programs by default and it has a bunch of C code implementing all of
this internally anyway, so we're basically already switching
already-shipping functionality to Rust from C.
* `object` - this crate is used to parse object file headers and
contents. Very low-level support is used from this crate and almost
all of it is disabled. Largely we're just using struct definitions as
well as convenience methods internally to read bytes and such.
* `addr2line` - this is the main meat of the implementation for
symbolication. This crate depends on `gimli` for DWARF parsing and
then provides interfaces needed by the `backtrace` crate to turn an
address into a filename / line number. This crate is actually pretty
small (fits in a single file almost!) and mirrors most of what
`dwarf.c` does for libbacktrace.
* `miniz_oxide` - the libbacktrace crate transparently handles
compressed debug information which is compressed with zlib. This crate
is used to decompress compressed debug sections.
* `gimli` - not actually used directly, but a dependency of `addr2line`.
* `adler32`- not used directly either, but a dependency of
`miniz_oxide`.
The goal of this change is to improve the safety of backtrace
symbolication in the standard library, especially in the face of
possibly malformed DWARF debug information. Even to this day we're still
seeing segfaults in libbacktrace which could possibly become security
vulnerabilities. This change should almost entirely eliminate this
possibility whilc also paving the way forward to adding more features
like split debug information.
Some references for those interested are:
* Original addition of libbacktrace - #12602
* OOM with libbacktrace - #24231
* Backtrace failure due to use of uninitialized value - #28447
* Possibility to feed untrusted data to libbacktrace - #21889
* Soundness fix for libbacktrace - #33729
* Crash in libbacktrace - #39468
* Support for macOS, never merged - ianlancetaylor/libbacktrace#2
* Performance issues with libbacktrace - #29293, #37477
* Update procedure is quite complicated due to how many patches we
need to carry - #50955
* Libbacktrace doesn't work on MinGW with dynamic libs - #71060
* Segfault in libbacktrace on macOS - #71397
Switching to Rust will not make us immune to all of these issues. The
crashes are expected to go away, but correctness and performance may
still have bugs arise. The gimli and `backtrace` crates, however, are
actively maintained unlike libbacktrace, so this should enable us to at
least efficiently apply fixes as situations come up.
2020-05-13 23:22:37 +02:00
|
|
|
"gimli",
|
2020-10-26 14:34:29 +01:00
|
|
|
"gsgdt",
|
2020-02-24 19:09:44 +01:00
|
|
|
"hashbrown",
|
2020-02-25 16:30:33 +01:00
|
|
|
"hermit-abi",
|
2020-02-24 19:09:44 +01:00
|
|
|
"humantime",
|
|
|
|
"indexmap",
|
2020-08-17 21:13:04 +02:00
|
|
|
"instant",
|
2020-02-24 19:09:44 +01:00
|
|
|
"itertools",
|
|
|
|
"jobserver",
|
|
|
|
"kernel32-sys",
|
|
|
|
"lazy_static",
|
|
|
|
"libc",
|
|
|
|
"libz-sys",
|
|
|
|
"lock_api",
|
|
|
|
"log",
|
2020-08-17 21:13:04 +02:00
|
|
|
"maybe-uninit",
|
2020-03-31 07:17:15 +02:00
|
|
|
"md-5",
|
2020-02-24 19:09:44 +01:00
|
|
|
"measureme",
|
|
|
|
"memchr",
|
|
|
|
"memmap",
|
2021-03-17 17:29:45 +01:00
|
|
|
"memmap2",
|
2020-02-24 19:09:44 +01:00
|
|
|
"memoffset",
|
|
|
|
"miniz_oxide",
|
|
|
|
"num_cpus",
|
std: Switch from libbacktrace to gimli
This commit is a proof-of-concept for switching the standard library's
backtrace symbolication mechanism on most platforms from libbacktrace to
gimli. The standard library's support for `RUST_BACKTRACE=1` requires
in-process parsing of object files and DWARF debug information to
interpret it and print the filename/line number of stack frames as part
of a backtrace.
Historically this support in the standard library has come from a
library called "libbacktrace". The libbacktrace library seems to have
been extracted from gcc at some point and is written in C. We've had a
lot of issues with libbacktrace over time, unfortunately, though. The
library does not appear to be actively maintained since we've had
patches sit for months-to-years without comments. We have discovered a
good number of soundness issues with the library itself, both when
parsing valid DWARF as well as invalid DWARF. This is enough of an issue
that the libs team has previously decided that we cannot feed untrusted
inputs to libbacktrace. This also doesn't take into account the
portability of libbacktrace which has been difficult to manage and
maintain over time. While possible there are lots of exceptions and it's
the main C dependency of the standard library right now.
For years it's been the desire to switch over to a Rust-based solution
for symbolicating backtraces. It's been assumed that we'll be using the
Gimli family of crates for this purpose, which are targeted at safely
and efficiently parsing DWARF debug information. I've been working
recently to shore up the Gimli support in the `backtrace` crate. As of a
few weeks ago the `backtrace` crate, by default, uses Gimli when loaded
from crates.io. This transition has gone well enough that I figured it
was time to start talking seriously about this change to the standard
library.
This commit is a preview of what's probably the best way to integrate
the `backtrace` crate into the standard library with the Gimli feature
turned on. While today it's used as a crates.io dependency, this commit
switches the `backtrace` crate to a submodule of this repository which
will need to be updated manually. This is not done lightly, but is
thought to be the best solution. The primary reason for this is that the
`backtrace` crate needs to do some pretty nontrivial filesystem
interactions to locate debug information. Working without `std::fs` is
not an option, and while it might be possible to do some sort of
trait-based solution when prototyped it was found to be too unergonomic.
Using a submodule allows the `backtrace` crate to build as a submodule
of the `std` crate itself, enabling it to use `std::fs` and such.
Otherwise this adds new dependencies to the standard library. This step
requires extra attention because this means that these crates are now
going to be included with all Rust programs by default. It's important
to note, however, that we're already shipping libbacktrace with all Rust
programs by default and it has a bunch of C code implementing all of
this internally anyway, so we're basically already switching
already-shipping functionality to Rust from C.
* `object` - this crate is used to parse object file headers and
contents. Very low-level support is used from this crate and almost
all of it is disabled. Largely we're just using struct definitions as
well as convenience methods internally to read bytes and such.
* `addr2line` - this is the main meat of the implementation for
symbolication. This crate depends on `gimli` for DWARF parsing and
then provides interfaces needed by the `backtrace` crate to turn an
address into a filename / line number. This crate is actually pretty
small (fits in a single file almost!) and mirrors most of what
`dwarf.c` does for libbacktrace.
* `miniz_oxide` - the libbacktrace crate transparently handles
compressed debug information which is compressed with zlib. This crate
is used to decompress compressed debug sections.
* `gimli` - not actually used directly, but a dependency of `addr2line`.
* `adler32`- not used directly either, but a dependency of
`miniz_oxide`.
The goal of this change is to improve the safety of backtrace
symbolication in the standard library, especially in the face of
possibly malformed DWARF debug information. Even to this day we're still
seeing segfaults in libbacktrace which could possibly become security
vulnerabilities. This change should almost entirely eliminate this
possibility whilc also paving the way forward to adding more features
like split debug information.
Some references for those interested are:
* Original addition of libbacktrace - #12602
* OOM with libbacktrace - #24231
* Backtrace failure due to use of uninitialized value - #28447
* Possibility to feed untrusted data to libbacktrace - #21889
* Soundness fix for libbacktrace - #33729
* Crash in libbacktrace - #39468
* Support for macOS, never merged - ianlancetaylor/libbacktrace#2
* Performance issues with libbacktrace - #29293, #37477
* Update procedure is quite complicated due to how many patches we
need to carry - #50955
* Libbacktrace doesn't work on MinGW with dynamic libs - #71060
* Segfault in libbacktrace on macOS - #71397
Switching to Rust will not make us immune to all of these issues. The
crashes are expected to go away, but correctness and performance may
still have bugs arise. The gimli and `backtrace` crates, however, are
actively maintained unlike libbacktrace, so this should enable us to at
least efficiently apply fixes as situations come up.
2020-05-13 23:22:37 +02:00
|
|
|
"object",
|
2020-05-16 06:43:06 +02:00
|
|
|
"once_cell",
|
2020-03-31 07:17:15 +02:00
|
|
|
"opaque-debug",
|
2020-02-24 19:09:44 +01:00
|
|
|
"parking_lot",
|
|
|
|
"parking_lot_core",
|
2020-07-18 01:00:17 +02:00
|
|
|
"pathdiff",
|
2021-02-26 00:25:38 +01:00
|
|
|
"perf-event-open-sys",
|
2021-02-28 01:00:00 +01:00
|
|
|
"pin-project-lite",
|
2020-02-24 19:09:44 +01:00
|
|
|
"pkg-config",
|
|
|
|
"polonius-engine",
|
|
|
|
"ppv-lite86",
|
|
|
|
"proc-macro2",
|
2019-11-26 14:12:59 +01:00
|
|
|
"psm",
|
2020-02-24 19:09:44 +01:00
|
|
|
"punycode",
|
|
|
|
"quick-error",
|
|
|
|
"quote",
|
|
|
|
"rand",
|
|
|
|
"rand_chacha",
|
|
|
|
"rand_core",
|
|
|
|
"rand_hc",
|
|
|
|
"rand_pcg",
|
|
|
|
"rand_xorshift",
|
|
|
|
"redox_syscall",
|
|
|
|
"regex",
|
|
|
|
"regex-syntax",
|
|
|
|
"remove_dir_all",
|
|
|
|
"rustc-demangle",
|
|
|
|
"rustc-hash",
|
|
|
|
"rustc-rayon",
|
|
|
|
"rustc-rayon-core",
|
|
|
|
"rustc_version",
|
|
|
|
"scoped-tls",
|
|
|
|
"scopeguard",
|
|
|
|
"semver",
|
|
|
|
"semver-parser",
|
|
|
|
"serde",
|
|
|
|
"serde_derive",
|
2020-03-31 07:17:15 +02:00
|
|
|
"sha-1",
|
2020-10-13 17:41:06 +02:00
|
|
|
"sha2",
|
2020-02-24 19:09:44 +01:00
|
|
|
"smallvec",
|
2020-08-21 01:16:30 +02:00
|
|
|
"snap",
|
2020-02-24 19:09:44 +01:00
|
|
|
"stable_deref_trait",
|
2019-11-26 14:12:59 +01:00
|
|
|
"stacker",
|
2020-02-24 19:09:44 +01:00
|
|
|
"syn",
|
|
|
|
"synstructure",
|
|
|
|
"tempfile",
|
|
|
|
"termcolor",
|
|
|
|
"termize",
|
|
|
|
"thread_local",
|
2020-07-24 21:35:14 +02:00
|
|
|
"tracing",
|
|
|
|
"tracing-attributes",
|
|
|
|
"tracing-core",
|
2020-03-31 07:17:15 +02:00
|
|
|
"typenum",
|
2020-02-24 19:09:44 +01:00
|
|
|
"unicode-normalization",
|
|
|
|
"unicode-script",
|
|
|
|
"unicode-security",
|
|
|
|
"unicode-width",
|
|
|
|
"unicode-xid",
|
|
|
|
"vcpkg",
|
|
|
|
"version_check",
|
|
|
|
"wasi",
|
|
|
|
"winapi",
|
|
|
|
"winapi-build",
|
|
|
|
"winapi-i686-pc-windows-gnu",
|
|
|
|
"winapi-util",
|
|
|
|
"winapi-x86_64-pc-windows-gnu",
|
2018-02-24 01:25:21 +01:00
|
|
|
];
|
2018-02-23 02:52:56 +01:00
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
/// Dependency checks.
|
|
|
|
///
|
2020-06-12 04:31:49 +02:00
|
|
|
/// `root` is path to the directory with the root `Cargo.toml` (for the workspace). `cargo` is path
|
|
|
|
/// to the cargo executable.
|
|
|
|
pub fn check(root: &Path, cargo: &Path, bad: &mut bool) {
|
2020-02-24 19:09:44 +01:00
|
|
|
let mut cmd = cargo_metadata::MetadataCommand::new();
|
|
|
|
cmd.cargo_path(cargo)
|
2020-06-12 04:31:49 +02:00
|
|
|
.manifest_path(root.join("Cargo.toml"))
|
2020-02-24 19:09:44 +01:00
|
|
|
.features(cargo_metadata::CargoOpt::AllFeatures);
|
|
|
|
let metadata = t!(cmd.exec());
|
|
|
|
check_exceptions(&metadata, bad);
|
2020-07-07 17:12:44 +02:00
|
|
|
check_dependencies(&metadata, bad);
|
2020-02-24 19:09:44 +01:00
|
|
|
check_crate_duplicate(&metadata, bad);
|
2018-02-27 06:45:04 +01:00
|
|
|
}
|
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
/// Check that all licenses are in the valid list in `LICENSES`.
|
2018-02-23 02:52:56 +01:00
|
|
|
///
|
2020-02-24 19:09:44 +01:00
|
|
|
/// Packages listed in `EXCEPTIONS` are allowed for tools.
|
|
|
|
fn check_exceptions(metadata: &Metadata, bad: &mut bool) {
|
2020-02-24 21:54:32 +01:00
|
|
|
// Validate the EXCEPTIONS list hasn't changed.
|
|
|
|
for (name, license) in EXCEPTIONS {
|
|
|
|
// Check that the package actually exists.
|
|
|
|
if !metadata.packages.iter().any(|p| p.name == *name) {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(
|
|
|
|
bad,
|
2020-02-24 19:38:29 +01:00
|
|
|
"could not find exception package `{}`\n\
|
|
|
|
Remove from EXCEPTIONS list if it is no longer used.",
|
2020-02-24 21:54:32 +01:00
|
|
|
name
|
2020-02-24 19:38:29 +01:00
|
|
|
);
|
|
|
|
}
|
2020-02-24 21:54:32 +01:00
|
|
|
// Check that the license hasn't changed.
|
|
|
|
for pkg in metadata.packages.iter().filter(|p| p.name == *name) {
|
|
|
|
if pkg.name == "fuchsia-cprng" {
|
|
|
|
// This package doesn't declare a license expression. Manual
|
|
|
|
// inspection of the license file is necessary, which appears
|
|
|
|
// to be BSD-3-Clause.
|
|
|
|
assert!(pkg.license.is_none());
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
match &pkg.license {
|
|
|
|
None => {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(
|
|
|
|
bad,
|
2020-02-24 21:54:32 +01:00
|
|
|
"dependency exception `{}` does not declare a license expression",
|
|
|
|
pkg.id
|
|
|
|
);
|
|
|
|
}
|
|
|
|
Some(pkg_license) => {
|
|
|
|
if pkg_license.as_str() != *license {
|
2020-08-17 21:13:04 +02:00
|
|
|
if *name == "crossbeam-queue"
|
|
|
|
&& *license == "MIT/Apache-2.0 AND BSD-2-Clause"
|
|
|
|
{
|
|
|
|
// We have two versions of crossbeam-queue and both
|
|
|
|
// are fine.
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2020-02-24 21:54:32 +01:00
|
|
|
println!("dependency exception `{}` license has changed", name);
|
|
|
|
println!(" previously `{}` now `{}`", license, pkg_license);
|
|
|
|
println!(" update EXCEPTIONS for the new license");
|
|
|
|
*bad = true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2020-02-24 19:38:29 +01:00
|
|
|
}
|
2020-02-24 22:47:49 +01:00
|
|
|
|
2020-02-24 21:54:32 +01:00
|
|
|
let exception_names: Vec<_> = EXCEPTIONS.iter().map(|(name, _license)| *name).collect();
|
2020-02-24 22:47:49 +01:00
|
|
|
let runtime_ids = compute_runtime_crates(metadata);
|
|
|
|
|
2020-02-24 19:38:29 +01:00
|
|
|
// Check if any package does not have a valid license.
|
2020-02-24 19:09:44 +01:00
|
|
|
for pkg in &metadata.packages {
|
|
|
|
if pkg.source.is_none() {
|
|
|
|
// No need to check local packages.
|
2018-02-23 02:52:56 +01:00
|
|
|
continue;
|
2017-02-09 20:44:39 +01:00
|
|
|
}
|
2020-02-24 22:47:49 +01:00
|
|
|
if !runtime_ids.contains(&pkg.id) && exception_names.contains(&pkg.name.as_str()) {
|
2020-02-24 19:09:44 +01:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
let license = match &pkg.license {
|
|
|
|
Some(license) => license,
|
|
|
|
None => {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(bad, "dependency `{}` does not define a license expression", pkg.id);
|
2020-02-24 19:09:44 +01:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
};
|
|
|
|
if !LICENSES.contains(&license.as_str()) {
|
2020-02-24 22:47:49 +01:00
|
|
|
if pkg.name == "fortanix-sgx-abi" {
|
|
|
|
// This is a specific exception because SGX is considered
|
|
|
|
// "third party". See
|
|
|
|
// https://github.com/rust-lang/rust/issues/62620 for more. In
|
|
|
|
// general, these should never be added.
|
|
|
|
continue;
|
|
|
|
}
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(bad, "invalid license `{}` in `{}`", license, pkg.id);
|
2020-02-24 19:09:44 +01:00
|
|
|
}
|
2016-12-11 00:27:42 +01:00
|
|
|
}
|
2018-02-24 01:01:51 +01:00
|
|
|
}
|
2018-02-23 02:52:56 +01:00
|
|
|
|
2020-07-07 17:12:44 +02:00
|
|
|
/// Checks the dependency of `RESTRICTED_DEPENDENCY_CRATES` at the given path. Changes `bad` to
|
|
|
|
/// `true` if a check failed.
|
2018-02-24 01:01:51 +01:00
|
|
|
///
|
2020-07-07 17:12:44 +02:00
|
|
|
/// Specifically, this checks that the dependencies are on the `PERMITTED_DEPENDENCIES`.
|
|
|
|
fn check_dependencies(metadata: &Metadata, bad: &mut bool) {
|
|
|
|
// Check that the PERMITTED_DEPENDENCIES does not have unused entries.
|
|
|
|
for name in PERMITTED_DEPENDENCIES {
|
2020-02-24 19:38:29 +01:00
|
|
|
if !metadata.packages.iter().any(|p| p.name == *name) {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(
|
|
|
|
bad,
|
2020-07-07 17:12:44 +02:00
|
|
|
"could not find allowed package `{}`\n\
|
|
|
|
Remove from PERMITTED_DEPENDENCIES list if it is no longer used.",
|
2020-02-24 19:38:29 +01:00
|
|
|
name
|
|
|
|
);
|
|
|
|
}
|
|
|
|
}
|
2020-07-07 17:12:44 +02:00
|
|
|
// Get the list in a convenient form.
|
|
|
|
let permitted_dependencies: HashSet<_> = PERMITTED_DEPENDENCIES.iter().cloned().collect();
|
2018-02-27 06:45:04 +01:00
|
|
|
|
2019-01-11 17:32:31 +01:00
|
|
|
// Check dependencies.
|
2018-02-27 19:02:54 +01:00
|
|
|
let mut visited = BTreeSet::new();
|
|
|
|
let mut unapproved = BTreeSet::new();
|
2020-07-07 17:12:44 +02:00
|
|
|
for &krate in RESTRICTED_DEPENDENCY_CRATES.iter() {
|
2020-02-24 19:09:44 +01:00
|
|
|
let pkg = pkg_from_name(metadata, krate);
|
2020-07-07 17:12:44 +02:00
|
|
|
let mut bad =
|
|
|
|
check_crate_dependencies(&permitted_dependencies, metadata, &mut visited, pkg);
|
2018-02-27 06:45:04 +01:00
|
|
|
unapproved.append(&mut bad);
|
|
|
|
}
|
2018-02-24 01:01:51 +01:00
|
|
|
|
2018-09-01 21:17:57 +02:00
|
|
|
if !unapproved.is_empty() {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(bad, "Dependencies not explicitly permitted:");
|
2018-02-24 01:01:51 +01:00
|
|
|
for dep in unapproved {
|
2020-02-24 19:09:44 +01:00
|
|
|
println!("* {}", dep);
|
2018-02-24 01:01:51 +01:00
|
|
|
}
|
|
|
|
}
|
2018-02-27 06:45:04 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/// Checks the dependencies of the given crate from the given cargo metadata to see if they are on
|
2020-07-07 17:12:44 +02:00
|
|
|
/// the list of permitted dependencies. Returns a list of disallowed dependencies.
|
|
|
|
fn check_crate_dependencies<'a>(
|
|
|
|
permitted_dependencies: &'a HashSet<&'static str>,
|
2020-02-24 19:09:44 +01:00
|
|
|
metadata: &'a Metadata,
|
|
|
|
visited: &mut BTreeSet<&'a PackageId>,
|
|
|
|
krate: &'a Package,
|
|
|
|
) -> BTreeSet<&'a PackageId> {
|
2019-01-11 17:32:31 +01:00
|
|
|
// This will contain bad deps.
|
2018-02-27 19:02:54 +01:00
|
|
|
let mut unapproved = BTreeSet::new();
|
|
|
|
|
2019-01-11 17:32:31 +01:00
|
|
|
// Check if we have already visited this crate.
|
2020-02-24 19:09:44 +01:00
|
|
|
if visited.contains(&krate.id) {
|
2018-02-27 19:02:54 +01:00
|
|
|
return unapproved;
|
|
|
|
}
|
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
visited.insert(&krate.id);
|
2018-02-27 06:45:04 +01:00
|
|
|
|
2020-07-07 17:12:44 +02:00
|
|
|
// If this path is in-tree, we don't require it to be explicitly permitted.
|
2020-02-24 19:09:44 +01:00
|
|
|
if krate.source.is_some() {
|
2020-07-07 17:12:44 +02:00
|
|
|
// If this dependency is not on `PERMITTED_DEPENDENCIES`, add to bad set.
|
|
|
|
if !permitted_dependencies.contains(krate.name.as_str()) {
|
2020-02-24 19:09:44 +01:00
|
|
|
unapproved.insert(&krate.id);
|
2018-02-28 23:28:30 +01:00
|
|
|
}
|
2018-02-27 06:45:04 +01:00
|
|
|
}
|
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
// Do a DFS in the crate graph.
|
|
|
|
let to_check = deps_of(metadata, &krate.id);
|
2018-02-27 06:45:04 +01:00
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
for dep in to_check {
|
2020-07-07 17:12:44 +02:00
|
|
|
let mut bad = check_crate_dependencies(permitted_dependencies, metadata, visited, dep);
|
2018-02-28 23:28:30 +01:00
|
|
|
unapproved.append(&mut bad);
|
2018-02-27 06:45:04 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
unapproved
|
2018-02-23 02:52:56 +01:00
|
|
|
}
|
2018-08-08 08:09:36 +02:00
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
/// Prevents multiple versions of some expensive crates.
|
|
|
|
fn check_crate_duplicate(metadata: &Metadata, bad: &mut bool) {
|
2018-08-08 08:09:36 +02:00
|
|
|
const FORBIDDEN_TO_HAVE_DUPLICATES: &[&str] = &[
|
2019-01-11 17:32:31 +01:00
|
|
|
// These two crates take quite a long time to build, so don't allow two versions of them
|
|
|
|
// to accidentally sneak into our dependency graph, in order to ensure we keep our CI times
|
|
|
|
// under control.
|
2019-03-01 15:06:14 +01:00
|
|
|
"cargo",
|
2020-03-31 14:47:40 +02:00
|
|
|
"rustc-ap-rustc_ast",
|
2018-08-08 08:09:36 +02:00
|
|
|
];
|
|
|
|
|
2020-02-24 19:09:44 +01:00
|
|
|
for &name in FORBIDDEN_TO_HAVE_DUPLICATES {
|
|
|
|
let matches: Vec<_> = metadata.packages.iter().filter(|pkg| pkg.name == name).collect();
|
|
|
|
match matches.len() {
|
|
|
|
0 => {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(
|
|
|
|
bad,
|
2020-02-24 19:09:44 +01:00
|
|
|
"crate `{}` is missing, update `check_crate_duplicate` \
|
|
|
|
if it is no longer used",
|
|
|
|
name
|
|
|
|
);
|
|
|
|
}
|
|
|
|
1 => {}
|
|
|
|
_ => {
|
2020-12-14 15:10:15 +01:00
|
|
|
tidy_error!(
|
|
|
|
bad,
|
2020-02-24 19:09:44 +01:00
|
|
|
"crate `{}` is duplicated in `Cargo.lock`, \
|
|
|
|
it is too expensive to build multiple times, \
|
|
|
|
so make sure only one version appears across all dependencies",
|
|
|
|
name
|
|
|
|
);
|
|
|
|
for pkg in matches {
|
|
|
|
println!(" * {}", pkg.id);
|
|
|
|
}
|
|
|
|
}
|
2018-08-08 08:09:36 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2020-02-24 19:09:44 +01:00
|
|
|
|
|
|
|
/// Returns a list of dependencies for the given package.
|
|
|
|
fn deps_of<'a>(metadata: &'a Metadata, pkg_id: &'a PackageId) -> Vec<&'a Package> {
|
2020-02-24 22:47:49 +01:00
|
|
|
let resolve = metadata.resolve.as_ref().unwrap();
|
|
|
|
let node = resolve
|
2020-02-24 19:09:44 +01:00
|
|
|
.nodes
|
|
|
|
.iter()
|
|
|
|
.find(|n| &n.id == pkg_id)
|
|
|
|
.unwrap_or_else(|| panic!("could not find `{}` in resolve", pkg_id));
|
|
|
|
node.deps
|
|
|
|
.iter()
|
|
|
|
.map(|dep| {
|
|
|
|
metadata.packages.iter().find(|pkg| pkg.id == dep.pkg).unwrap_or_else(|| {
|
|
|
|
panic!("could not find dep `{}` for pkg `{}` in resolve", dep.pkg, pkg_id)
|
|
|
|
})
|
|
|
|
})
|
|
|
|
.collect()
|
|
|
|
}
|
|
|
|
|
|
|
|
/// Finds a package with the given name.
|
|
|
|
fn pkg_from_name<'a>(metadata: &'a Metadata, name: &'static str) -> &'a Package {
|
|
|
|
let mut i = metadata.packages.iter().filter(|p| p.name == name);
|
|
|
|
let result =
|
|
|
|
i.next().unwrap_or_else(|| panic!("could not find package `{}` in package list", name));
|
|
|
|
assert!(i.next().is_none(), "more than one package found for `{}`", name);
|
|
|
|
result
|
|
|
|
}
|
2020-02-24 22:47:49 +01:00
|
|
|
|
|
|
|
/// Finds all the packages that are in the rust runtime.
|
|
|
|
fn compute_runtime_crates<'a>(metadata: &'a Metadata) -> HashSet<&'a PackageId> {
|
|
|
|
let resolve = metadata.resolve.as_ref().unwrap();
|
|
|
|
let mut result = HashSet::new();
|
|
|
|
for name in RUNTIME_CRATES {
|
|
|
|
let id = &pkg_from_name(metadata, name).id;
|
|
|
|
normal_deps_of_r(resolve, id, &mut result);
|
|
|
|
}
|
|
|
|
result
|
|
|
|
}
|
|
|
|
|
|
|
|
/// Recursively find all normal dependencies.
|
|
|
|
fn normal_deps_of_r<'a>(
|
|
|
|
resolve: &'a Resolve,
|
|
|
|
pkg_id: &'a PackageId,
|
|
|
|
result: &mut HashSet<&'a PackageId>,
|
|
|
|
) {
|
|
|
|
if !result.insert(pkg_id) {
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
let node = resolve
|
|
|
|
.nodes
|
|
|
|
.iter()
|
|
|
|
.find(|n| &n.id == pkg_id)
|
|
|
|
.unwrap_or_else(|| panic!("could not find `{}` in resolve", pkg_id));
|
|
|
|
// Don't care about dev-dependencies.
|
|
|
|
// Build dependencies *shouldn't* matter unless they do some kind of
|
|
|
|
// codegen. For now we'll assume they don't.
|
|
|
|
let deps = node.deps.iter().filter(|node_dep| {
|
|
|
|
node_dep
|
|
|
|
.dep_kinds
|
|
|
|
.iter()
|
|
|
|
.any(|kind_info| kind_info.kind == cargo_metadata::DependencyKind::Normal)
|
|
|
|
});
|
|
|
|
for dep in deps {
|
|
|
|
normal_deps_of_r(resolve, &dep.pkg, result);
|
|
|
|
}
|
|
|
|
}
|