rust/src/build_helper/lib.rs

297 lines
8.9 KiB
Rust
Raw Normal View History

use std::fs::File;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
2018-04-01 04:32:40 +02:00
use std::time::{SystemTime, UNIX_EPOCH};
2018-03-25 16:20:56 +02:00
use std::{env, fs};
use std::thread;
/// A helper macro to `unwrap` a result except also print out details like:
///
/// * The file/line of the panic
/// * The expression that failed
/// * The error itself
///
/// This is currently used judiciously throughout the build system rather than
/// using a `Result` with `try!`, but this may change one day...
#[macro_export]
macro_rules! t {
2018-03-25 16:20:56 +02:00
($e:expr) => {
match $e {
Ok(e) => e,
Err(e) => panic!("{} failed with {}", stringify!($e), e),
}
};
}
pub fn run(cmd: &mut Command) {
println!("running: {:?}", cmd);
run_silent(cmd);
}
pub fn run_silent(cmd: &mut Command) {
if !try_run_silent(cmd) {
std::process::exit(1);
}
}
pub fn try_run_silent(cmd: &mut Command) -> bool {
let status = match cmd.status() {
Ok(status) => status,
2018-03-25 16:20:56 +02:00
Err(e) => fail(&format!(
"failed to execute command: {:?}\nerror: {}",
cmd, e
)),
};
if !status.success() {
2018-03-25 16:20:56 +02:00
println!(
"\n\ncommand did not execute successfully: {:?}\n\
expected success, got: {}\n\n",
cmd, status
);
}
status.success()
}
pub fn run_suppressed(cmd: &mut Command) {
if !try_run_suppressed(cmd) {
std::process::exit(1);
}
}
pub fn try_run_suppressed(cmd: &mut Command) -> bool {
rustbuild: Add support for compiling Cargo This commit adds support to rustbuild for compiling Cargo as part of the release process. Previously rustbuild would simply download a Cargo snapshot and repackage it. With this change we should be able to turn off artifacts from the rust-lang/cargo repository and purely rely on the artifacts Cargo produces here. The infrastructure added here is intended to be extensible to other components, such as the RLS. It won't exactly be a one-line addition, but the addition of Cargo didn't require too much hooplah anyway. The process for release Cargo will now look like: * The rust-lang/rust repository has a Cargo submodule which is used to build a Cargo to pair with the rust-lang/rust release * Periodically we'll update the cargo submodule as necessary on rust-lang/rust's master branch * When branching beta we'll create a new branch of Cargo (as we do today), and the first commit to the beta branch will be to update the Cargo submodule to this exact revision. * When branching stable, we'll ensure that the Cargo submodule is updated and then make a stable release. Backports to Cargo will look like: * Send a PR to cargo's master branch * Send a PR to cargo's release branch (e.g. rust-1.16.0) * Send a PR to rust-lang/rust's beta branch updating the submodule * Eventually send a PR to rust-lang/rust's master branch updating the submodule For reference, the process to add a new component to the rust-lang/rust release would look like: * Add `$foo` as a submodule in `src/tools` * Add a `tool-$foo` step which compiles `$foo` with the specified compiler, likely mirroring what Cargo does. * Add a `dist-$foo` step which uses `src/tools/$foo` and the `tool-$foo` output to create a rust-installer package for `$foo` likely mirroring what Cargo does. * Update the `dist-extended` step with a new dependency on `dist-$foo` * Update `src/tools/build-manifest` for the new component.
2017-02-16 00:57:06 +01:00
let output = match cmd.output() {
Ok(status) => status,
2018-03-25 16:20:56 +02:00
Err(e) => fail(&format!(
"failed to execute command: {:?}\nerror: {}",
cmd, e
)),
rustbuild: Add support for compiling Cargo This commit adds support to rustbuild for compiling Cargo as part of the release process. Previously rustbuild would simply download a Cargo snapshot and repackage it. With this change we should be able to turn off artifacts from the rust-lang/cargo repository and purely rely on the artifacts Cargo produces here. The infrastructure added here is intended to be extensible to other components, such as the RLS. It won't exactly be a one-line addition, but the addition of Cargo didn't require too much hooplah anyway. The process for release Cargo will now look like: * The rust-lang/rust repository has a Cargo submodule which is used to build a Cargo to pair with the rust-lang/rust release * Periodically we'll update the cargo submodule as necessary on rust-lang/rust's master branch * When branching beta we'll create a new branch of Cargo (as we do today), and the first commit to the beta branch will be to update the Cargo submodule to this exact revision. * When branching stable, we'll ensure that the Cargo submodule is updated and then make a stable release. Backports to Cargo will look like: * Send a PR to cargo's master branch * Send a PR to cargo's release branch (e.g. rust-1.16.0) * Send a PR to rust-lang/rust's beta branch updating the submodule * Eventually send a PR to rust-lang/rust's master branch updating the submodule For reference, the process to add a new component to the rust-lang/rust release would look like: * Add `$foo` as a submodule in `src/tools` * Add a `tool-$foo` step which compiles `$foo` with the specified compiler, likely mirroring what Cargo does. * Add a `dist-$foo` step which uses `src/tools/$foo` and the `tool-$foo` output to create a rust-installer package for `$foo` likely mirroring what Cargo does. * Update the `dist-extended` step with a new dependency on `dist-$foo` * Update `src/tools/build-manifest` for the new component.
2017-02-16 00:57:06 +01:00
};
if !output.status.success() {
2018-03-25 16:20:56 +02:00
println!(
"\n\ncommand did not execute successfully: {:?}\n\
expected success, got: {}\n\n\
stdout ----\n{}\n\
stderr ----\n{}\n\n",
cmd,
output.status,
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
}
output.status.success()
rustbuild: Add support for compiling Cargo This commit adds support to rustbuild for compiling Cargo as part of the release process. Previously rustbuild would simply download a Cargo snapshot and repackage it. With this change we should be able to turn off artifacts from the rust-lang/cargo repository and purely rely on the artifacts Cargo produces here. The infrastructure added here is intended to be extensible to other components, such as the RLS. It won't exactly be a one-line addition, but the addition of Cargo didn't require too much hooplah anyway. The process for release Cargo will now look like: * The rust-lang/rust repository has a Cargo submodule which is used to build a Cargo to pair with the rust-lang/rust release * Periodically we'll update the cargo submodule as necessary on rust-lang/rust's master branch * When branching beta we'll create a new branch of Cargo (as we do today), and the first commit to the beta branch will be to update the Cargo submodule to this exact revision. * When branching stable, we'll ensure that the Cargo submodule is updated and then make a stable release. Backports to Cargo will look like: * Send a PR to cargo's master branch * Send a PR to cargo's release branch (e.g. rust-1.16.0) * Send a PR to rust-lang/rust's beta branch updating the submodule * Eventually send a PR to rust-lang/rust's master branch updating the submodule For reference, the process to add a new component to the rust-lang/rust release would look like: * Add `$foo` as a submodule in `src/tools` * Add a `tool-$foo` step which compiles `$foo` with the specified compiler, likely mirroring what Cargo does. * Add a `dist-$foo` step which uses `src/tools/$foo` and the `tool-$foo` output to create a rust-installer package for `$foo` likely mirroring what Cargo does. * Update the `dist-extended` step with a new dependency on `dist-$foo` * Update `src/tools/build-manifest` for the new component.
2017-02-16 00:57:06 +01:00
}
pub fn gnu_target(target: &str) -> &str {
match target {
"i686-pc-windows-msvc" => "i686-pc-win32",
"x86_64-pc-windows-msvc" => "x86_64-pc-win32",
"i686-pc-windows-gnu" => "i686-w64-mingw32",
"x86_64-pc-windows-gnu" => "x86_64-w64-mingw32",
s => s,
}
}
pub fn make(host: &str) -> PathBuf {
2018-03-25 16:20:56 +02:00
if host.contains("bitrig") || host.contains("dragonfly") || host.contains("freebsd")
|| host.contains("netbsd") || host.contains("openbsd")
{
PathBuf::from("gmake")
} else {
PathBuf::from("make")
}
}
pub fn output(cmd: &mut Command) -> String {
let output = match cmd.stderr(Stdio::inherit()).output() {
Ok(status) => status,
2018-03-25 16:20:56 +02:00
Err(e) => fail(&format!(
"failed to execute command: {:?}\nerror: {}",
cmd, e
)),
};
if !output.status.success() {
2018-03-25 16:20:56 +02:00
panic!(
"command did not execute successfully: {:?}\n\
expected success, got: {}",
cmd, output.status
);
}
String::from_utf8(output.stdout).unwrap()
}
pub fn rerun_if_changed_anything_in_dir(dir: &Path) {
2018-03-25 16:20:56 +02:00
let mut stack = dir.read_dir()
.unwrap()
.map(|e| e.unwrap())
.filter(|e| &*e.file_name() != ".git")
.collect::<Vec<_>>();
while let Some(entry) = stack.pop() {
let path = entry.path();
if entry.file_type().unwrap().is_dir() {
stack.extend(path.read_dir().unwrap().map(|e| e.unwrap()));
} else {
println!("cargo:rerun-if-changed={}", path.display());
}
}
}
/// Returns the last-modified time for `path`, or zero if it doesn't exist.
2018-04-01 04:32:40 +02:00
pub fn mtime(path: &Path) -> SystemTime {
2018-03-25 16:20:56 +02:00
fs::metadata(path)
.and_then(|f| f.modified())
.unwrap_or(UNIX_EPOCH)
}
/// Returns whether `dst` is up to date given that the file or files in `src`
/// are used to generate it.
///
/// Uses last-modified time checks to verify this.
pub fn up_to_date(src: &Path, dst: &Path) -> bool {
if !dst.exists() {
return false;
}
let threshold = mtime(dst);
let meta = match fs::metadata(src) {
Ok(meta) => meta,
Err(e) => panic!("source {:?} failed to get metadata: {}", src, e),
};
if meta.is_dir() {
2018-04-01 04:32:40 +02:00
dir_up_to_date(src, threshold)
} else {
2018-04-01 04:32:40 +02:00
meta.modified().unwrap_or(UNIX_EPOCH) <= threshold
}
}
#[must_use]
pub struct NativeLibBoilerplate {
pub src_dir: PathBuf,
pub out_dir: PathBuf,
}
impl NativeLibBoilerplate {
/// On OSX we don't want to ship the exact filename that compiler-rt builds.
/// This conflicts with the system and ours is likely a wildly different
/// version, so they can't be substituted.
///
/// As a result, we rename it here but we need to also use
/// `install_name_tool` on OSX to rename the commands listed inside of it to
/// ensure it's linked against correctly.
pub fn fixup_sanitizer_lib_name(&self, sanitizer_name: &str) {
if env::var("TARGET").unwrap() != "x86_64-apple-darwin" {
return
}
let dir = self.out_dir.join("build/lib/darwin");
let name = format!("clang_rt.{}_osx_dynamic", sanitizer_name);
let src = dir.join(&format!("lib{}.dylib", name));
let new_name = format!("lib__rustc__{}.dylib", name);
let dst = dir.join(&new_name);
println!("{} => {}", src.display(), dst.display());
fs::rename(&src, &dst).unwrap();
let status = Command::new("install_name_tool")
.arg("-id")
.arg(format!("@rpath/{}", new_name))
.arg(&dst)
.status()
.expect("failed to execute `install_name_tool`");
assert!(status.success());
}
}
impl Drop for NativeLibBoilerplate {
fn drop(&mut self) {
if !thread::panicking() {
t!(File::create(self.out_dir.join("rustbuild.timestamp")));
}
}
}
// Perform standard preparations for native libraries that are build only once for all stages.
// Emit rerun-if-changed and linking attributes for Cargo, check if any source files are
// updated, calculate paths used later in actual build with CMake/make or C/C++ compiler.
// If Err is returned, then everything is up-to-date and further build actions can be skipped.
// Timestamps are created automatically when the result of `native_lib_boilerplate` goes out
// of scope, so all the build actions should be completed until then.
2018-03-25 16:20:56 +02:00
pub fn native_lib_boilerplate(
std: Depend directly on crates.io crates Ever since we added a Cargo-based build system for the compiler the standard library has always been a little special, it's never been able to depend on crates.io crates for runtime dependencies. This has been a result of various limitations, namely that Cargo doesn't understand that crates from crates.io depend on libcore, so Cargo tries to build crates before libcore is finished. I had an idea this afternoon, however, which lifts the strategy from #52919 to directly depend on crates.io crates from the standard library. After all is said and done this removes a whopping three submodules that we need to manage! The basic idea here is that for any crate `std` depends on it adds an *optional* dependency on an empty crate on crates.io, in this case named `rustc-std-workspace-core`. This crate is overridden via `[patch]` in this repository to point to a local crate we write, and *that* has a `path` dependency on libcore. Note that all `no_std` crates also depend on `compiler_builtins`, but if we're not using submodules we can publish `compiler_builtins` to crates.io and all crates can depend on it anyway! The basic strategy then looks like: * The standard library (or some transitive dep) decides to depend on a crate `foo`. * The standard library adds ```toml [dependencies] foo = { version = "0.1", features = ['rustc-dep-of-std'] } ``` * The crate `foo` has an optional dependency on `rustc-std-workspace-core` * The crate `foo` has an optional dependency on `compiler_builtins` * The crate `foo` has a feature `rustc-dep-of-std` which activates these crates and any other necessary infrastructure in the crate. A sample commit for `dlmalloc` [turns out to be quite simple][commit]. After that all `no_std` crates should largely build "as is" and still be publishable on crates.io! Notably they should be able to continue to use stable Rust if necessary, since the `rename-dependency` feature of Cargo is soon stabilizing. As a proof of concept, this commit removes the `dlmalloc`, `libcompiler_builtins`, and `libc` submodules from this repository. Long thorns in our side these are now gone for good and we can directly depend on crates.io! It's hoped that in the long term we can bring in other crates as necessary, but for now this is largely intended to simply make it easier to manage these crates and remove submodules. This should be a transparent non-breaking change for all users, but one possible stickler is that this almost for sure breaks out-of-tree `std`-building tools like `xargo` and `cargo-xbuild`. I think it should be relatively easy to get them working, however, as all that's needed is an entry in the `[patch]` section used to build the standard library. Hopefully we can work with these tools to solve this problem! [commit]: https://github.com/alexcrichton/dlmalloc-rs/commit/28ee12db813a3b650a7c25d1c36d2c17dcb88ae3
2018-11-20 06:52:50 +01:00
src_dir: &Path,
2018-03-25 16:20:56 +02:00
out_name: &str,
link_name: &str,
search_subdir: &str,
) -> Result<NativeLibBoilerplate, ()> {
std: Depend directly on crates.io crates Ever since we added a Cargo-based build system for the compiler the standard library has always been a little special, it's never been able to depend on crates.io crates for runtime dependencies. This has been a result of various limitations, namely that Cargo doesn't understand that crates from crates.io depend on libcore, so Cargo tries to build crates before libcore is finished. I had an idea this afternoon, however, which lifts the strategy from #52919 to directly depend on crates.io crates from the standard library. After all is said and done this removes a whopping three submodules that we need to manage! The basic idea here is that for any crate `std` depends on it adds an *optional* dependency on an empty crate on crates.io, in this case named `rustc-std-workspace-core`. This crate is overridden via `[patch]` in this repository to point to a local crate we write, and *that* has a `path` dependency on libcore. Note that all `no_std` crates also depend on `compiler_builtins`, but if we're not using submodules we can publish `compiler_builtins` to crates.io and all crates can depend on it anyway! The basic strategy then looks like: * The standard library (or some transitive dep) decides to depend on a crate `foo`. * The standard library adds ```toml [dependencies] foo = { version = "0.1", features = ['rustc-dep-of-std'] } ``` * The crate `foo` has an optional dependency on `rustc-std-workspace-core` * The crate `foo` has an optional dependency on `compiler_builtins` * The crate `foo` has a feature `rustc-dep-of-std` which activates these crates and any other necessary infrastructure in the crate. A sample commit for `dlmalloc` [turns out to be quite simple][commit]. After that all `no_std` crates should largely build "as is" and still be publishable on crates.io! Notably they should be able to continue to use stable Rust if necessary, since the `rename-dependency` feature of Cargo is soon stabilizing. As a proof of concept, this commit removes the `dlmalloc`, `libcompiler_builtins`, and `libc` submodules from this repository. Long thorns in our side these are now gone for good and we can directly depend on crates.io! It's hoped that in the long term we can bring in other crates as necessary, but for now this is largely intended to simply make it easier to manage these crates and remove submodules. This should be a transparent non-breaking change for all users, but one possible stickler is that this almost for sure breaks out-of-tree `std`-building tools like `xargo` and `cargo-xbuild`. I think it should be relatively easy to get them working, however, as all that's needed is an entry in the `[patch]` section used to build the standard library. Hopefully we can work with these tools to solve this problem! [commit]: https://github.com/alexcrichton/dlmalloc-rs/commit/28ee12db813a3b650a7c25d1c36d2c17dcb88ae3
2018-11-20 06:52:50 +01:00
rerun_if_changed_anything_in_dir(src_dir);
let out_dir = env::var_os("RUSTBUILD_NATIVE_DIR").unwrap_or_else(||
env::var_os("OUT_DIR").unwrap());
let out_dir = PathBuf::from(out_dir).join(out_name);
t!(fs::create_dir_all(&out_dir));
if link_name.contains('=') {
println!("cargo:rustc-link-lib={}", link_name);
} else {
println!("cargo:rustc-link-lib=static={}", link_name);
}
2018-03-25 16:20:56 +02:00
println!(
"cargo:rustc-link-search=native={}",
out_dir.join(search_subdir).display()
);
let timestamp = out_dir.join("rustbuild.timestamp");
std: Depend directly on crates.io crates Ever since we added a Cargo-based build system for the compiler the standard library has always been a little special, it's never been able to depend on crates.io crates for runtime dependencies. This has been a result of various limitations, namely that Cargo doesn't understand that crates from crates.io depend on libcore, so Cargo tries to build crates before libcore is finished. I had an idea this afternoon, however, which lifts the strategy from #52919 to directly depend on crates.io crates from the standard library. After all is said and done this removes a whopping three submodules that we need to manage! The basic idea here is that for any crate `std` depends on it adds an *optional* dependency on an empty crate on crates.io, in this case named `rustc-std-workspace-core`. This crate is overridden via `[patch]` in this repository to point to a local crate we write, and *that* has a `path` dependency on libcore. Note that all `no_std` crates also depend on `compiler_builtins`, but if we're not using submodules we can publish `compiler_builtins` to crates.io and all crates can depend on it anyway! The basic strategy then looks like: * The standard library (or some transitive dep) decides to depend on a crate `foo`. * The standard library adds ```toml [dependencies] foo = { version = "0.1", features = ['rustc-dep-of-std'] } ``` * The crate `foo` has an optional dependency on `rustc-std-workspace-core` * The crate `foo` has an optional dependency on `compiler_builtins` * The crate `foo` has a feature `rustc-dep-of-std` which activates these crates and any other necessary infrastructure in the crate. A sample commit for `dlmalloc` [turns out to be quite simple][commit]. After that all `no_std` crates should largely build "as is" and still be publishable on crates.io! Notably they should be able to continue to use stable Rust if necessary, since the `rename-dependency` feature of Cargo is soon stabilizing. As a proof of concept, this commit removes the `dlmalloc`, `libcompiler_builtins`, and `libc` submodules from this repository. Long thorns in our side these are now gone for good and we can directly depend on crates.io! It's hoped that in the long term we can bring in other crates as necessary, but for now this is largely intended to simply make it easier to manage these crates and remove submodules. This should be a transparent non-breaking change for all users, but one possible stickler is that this almost for sure breaks out-of-tree `std`-building tools like `xargo` and `cargo-xbuild`. I think it should be relatively easy to get them working, however, as all that's needed is an entry in the `[patch]` section used to build the standard library. Hopefully we can work with these tools to solve this problem! [commit]: https://github.com/alexcrichton/dlmalloc-rs/commit/28ee12db813a3b650a7c25d1c36d2c17dcb88ae3
2018-11-20 06:52:50 +01:00
if !up_to_date(Path::new("build.rs"), &timestamp) || !up_to_date(src_dir, &timestamp) {
2018-03-25 16:20:56 +02:00
Ok(NativeLibBoilerplate {
std: Depend directly on crates.io crates Ever since we added a Cargo-based build system for the compiler the standard library has always been a little special, it's never been able to depend on crates.io crates for runtime dependencies. This has been a result of various limitations, namely that Cargo doesn't understand that crates from crates.io depend on libcore, so Cargo tries to build crates before libcore is finished. I had an idea this afternoon, however, which lifts the strategy from #52919 to directly depend on crates.io crates from the standard library. After all is said and done this removes a whopping three submodules that we need to manage! The basic idea here is that for any crate `std` depends on it adds an *optional* dependency on an empty crate on crates.io, in this case named `rustc-std-workspace-core`. This crate is overridden via `[patch]` in this repository to point to a local crate we write, and *that* has a `path` dependency on libcore. Note that all `no_std` crates also depend on `compiler_builtins`, but if we're not using submodules we can publish `compiler_builtins` to crates.io and all crates can depend on it anyway! The basic strategy then looks like: * The standard library (or some transitive dep) decides to depend on a crate `foo`. * The standard library adds ```toml [dependencies] foo = { version = "0.1", features = ['rustc-dep-of-std'] } ``` * The crate `foo` has an optional dependency on `rustc-std-workspace-core` * The crate `foo` has an optional dependency on `compiler_builtins` * The crate `foo` has a feature `rustc-dep-of-std` which activates these crates and any other necessary infrastructure in the crate. A sample commit for `dlmalloc` [turns out to be quite simple][commit]. After that all `no_std` crates should largely build "as is" and still be publishable on crates.io! Notably they should be able to continue to use stable Rust if necessary, since the `rename-dependency` feature of Cargo is soon stabilizing. As a proof of concept, this commit removes the `dlmalloc`, `libcompiler_builtins`, and `libc` submodules from this repository. Long thorns in our side these are now gone for good and we can directly depend on crates.io! It's hoped that in the long term we can bring in other crates as necessary, but for now this is largely intended to simply make it easier to manage these crates and remove submodules. This should be a transparent non-breaking change for all users, but one possible stickler is that this almost for sure breaks out-of-tree `std`-building tools like `xargo` and `cargo-xbuild`. I think it should be relatively easy to get them working, however, as all that's needed is an entry in the `[patch]` section used to build the standard library. Hopefully we can work with these tools to solve this problem! [commit]: https://github.com/alexcrichton/dlmalloc-rs/commit/28ee12db813a3b650a7c25d1c36d2c17dcb88ae3
2018-11-20 06:52:50 +01:00
src_dir: src_dir.to_path_buf(),
2018-03-25 16:20:56 +02:00
out_dir: out_dir,
})
} else {
Err(())
}
}
pub fn sanitizer_lib_boilerplate(sanitizer_name: &str)
-> Result<(NativeLibBoilerplate, String), ()>
{
let (link_name, search_path, apple) = match &*env::var("TARGET").unwrap() {
"x86_64-unknown-linux-gnu" => (
format!("clang_rt.{}-x86_64", sanitizer_name),
"build/lib/linux",
false,
),
"x86_64-apple-darwin" => (
format!("clang_rt.{}_osx_dynamic", sanitizer_name),
"build/lib/darwin",
true,
),
_ => return Err(()),
};
let to_link = if apple {
format!("dylib=__rustc__{}", link_name)
} else {
format!("static={}", link_name)
};
std: Depend directly on crates.io crates Ever since we added a Cargo-based build system for the compiler the standard library has always been a little special, it's never been able to depend on crates.io crates for runtime dependencies. This has been a result of various limitations, namely that Cargo doesn't understand that crates from crates.io depend on libcore, so Cargo tries to build crates before libcore is finished. I had an idea this afternoon, however, which lifts the strategy from #52919 to directly depend on crates.io crates from the standard library. After all is said and done this removes a whopping three submodules that we need to manage! The basic idea here is that for any crate `std` depends on it adds an *optional* dependency on an empty crate on crates.io, in this case named `rustc-std-workspace-core`. This crate is overridden via `[patch]` in this repository to point to a local crate we write, and *that* has a `path` dependency on libcore. Note that all `no_std` crates also depend on `compiler_builtins`, but if we're not using submodules we can publish `compiler_builtins` to crates.io and all crates can depend on it anyway! The basic strategy then looks like: * The standard library (or some transitive dep) decides to depend on a crate `foo`. * The standard library adds ```toml [dependencies] foo = { version = "0.1", features = ['rustc-dep-of-std'] } ``` * The crate `foo` has an optional dependency on `rustc-std-workspace-core` * The crate `foo` has an optional dependency on `compiler_builtins` * The crate `foo` has a feature `rustc-dep-of-std` which activates these crates and any other necessary infrastructure in the crate. A sample commit for `dlmalloc` [turns out to be quite simple][commit]. After that all `no_std` crates should largely build "as is" and still be publishable on crates.io! Notably they should be able to continue to use stable Rust if necessary, since the `rename-dependency` feature of Cargo is soon stabilizing. As a proof of concept, this commit removes the `dlmalloc`, `libcompiler_builtins`, and `libc` submodules from this repository. Long thorns in our side these are now gone for good and we can directly depend on crates.io! It's hoped that in the long term we can bring in other crates as necessary, but for now this is largely intended to simply make it easier to manage these crates and remove submodules. This should be a transparent non-breaking change for all users, but one possible stickler is that this almost for sure breaks out-of-tree `std`-building tools like `xargo` and `cargo-xbuild`. I think it should be relatively easy to get them working, however, as all that's needed is an entry in the `[patch]` section used to build the standard library. Hopefully we can work with these tools to solve this problem! [commit]: https://github.com/alexcrichton/dlmalloc-rs/commit/28ee12db813a3b650a7c25d1c36d2c17dcb88ae3
2018-11-20 06:52:50 +01:00
// The source for `compiler-rt` comes from the `compiler-builtins` crate, so
// load our env var set by cargo to find the source code.
let dir = env::var_os("DEP_COMPILER_RT_COMPILER_RT").unwrap();
let lib = native_lib_boilerplate(
std: Depend directly on crates.io crates Ever since we added a Cargo-based build system for the compiler the standard library has always been a little special, it's never been able to depend on crates.io crates for runtime dependencies. This has been a result of various limitations, namely that Cargo doesn't understand that crates from crates.io depend on libcore, so Cargo tries to build crates before libcore is finished. I had an idea this afternoon, however, which lifts the strategy from #52919 to directly depend on crates.io crates from the standard library. After all is said and done this removes a whopping three submodules that we need to manage! The basic idea here is that for any crate `std` depends on it adds an *optional* dependency on an empty crate on crates.io, in this case named `rustc-std-workspace-core`. This crate is overridden via `[patch]` in this repository to point to a local crate we write, and *that* has a `path` dependency on libcore. Note that all `no_std` crates also depend on `compiler_builtins`, but if we're not using submodules we can publish `compiler_builtins` to crates.io and all crates can depend on it anyway! The basic strategy then looks like: * The standard library (or some transitive dep) decides to depend on a crate `foo`. * The standard library adds ```toml [dependencies] foo = { version = "0.1", features = ['rustc-dep-of-std'] } ``` * The crate `foo` has an optional dependency on `rustc-std-workspace-core` * The crate `foo` has an optional dependency on `compiler_builtins` * The crate `foo` has a feature `rustc-dep-of-std` which activates these crates and any other necessary infrastructure in the crate. A sample commit for `dlmalloc` [turns out to be quite simple][commit]. After that all `no_std` crates should largely build "as is" and still be publishable on crates.io! Notably they should be able to continue to use stable Rust if necessary, since the `rename-dependency` feature of Cargo is soon stabilizing. As a proof of concept, this commit removes the `dlmalloc`, `libcompiler_builtins`, and `libc` submodules from this repository. Long thorns in our side these are now gone for good and we can directly depend on crates.io! It's hoped that in the long term we can bring in other crates as necessary, but for now this is largely intended to simply make it easier to manage these crates and remove submodules. This should be a transparent non-breaking change for all users, but one possible stickler is that this almost for sure breaks out-of-tree `std`-building tools like `xargo` and `cargo-xbuild`. I think it should be relatively easy to get them working, however, as all that's needed is an entry in the `[patch]` section used to build the standard library. Hopefully we can work with these tools to solve this problem! [commit]: https://github.com/alexcrichton/dlmalloc-rs/commit/28ee12db813a3b650a7c25d1c36d2c17dcb88ae3
2018-11-20 06:52:50 +01:00
dir.as_ref(),
2018-03-25 16:20:56 +02:00
sanitizer_name,
&to_link,
2018-03-25 16:20:56 +02:00
search_path,
)?;
Ok((lib, link_name))
}
2018-04-01 04:32:40 +02:00
fn dir_up_to_date(src: &Path, threshold: SystemTime) -> bool {
t!(fs::read_dir(src)).map(|e| t!(e)).all(|e| {
let meta = t!(e.metadata());
if meta.is_dir() {
dir_up_to_date(&e.path(), threshold)
} else {
2018-04-01 04:32:40 +02:00
meta.modified().unwrap_or(UNIX_EPOCH) < threshold
}
})
}
fn fail(s: &str) -> ! {
println!("\n\n{}\n\n", s);
std::process::exit(1);
}