rust/src/librustc/metadata/loader.rs

804 lines
31 KiB
Rust
Raw Normal View History

// Copyright 2012-2014 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//! Finds crate binaries and loads their metadata
//!
//! Might I be the first to welcome you to a world of platform differences,
2014-08-02 01:40:21 +02:00
//! version requirements, dependency graphs, conflicting desires, and fun! This
//! is the major guts (along with metadata::creader) of the compiler for loading
//! crates and resolving dependencies. Let's take a tour!
//!
//! # The problem
//!
//! Each invocation of the compiler is immediately concerned with one primary
//! problem, to connect a set of crates to resolved crates on the filesystem.
//! Concretely speaking, the compiler follows roughly these steps to get here:
//!
//! 1. Discover a set of `extern crate` statements.
//! 2. Transform these directives into crate names. If the directive does not
//! have an explicit name, then the identifier is the name.
//! 3. For each of these crate names, find a corresponding crate on the
//! filesystem.
//!
//! Sounds easy, right? Let's walk into some of the nuances.
//!
//! ## Transitive Dependencies
//!
//! Let's say we've got three crates: A, B, and C. A depends on B, and B depends
//! on C. When we're compiling A, we primarily need to find and locate B, but we
//! also end up needing to find and locate C as well.
//!
//! The reason for this is that any of B's types could be composed of C's types,
//! any function in B could return a type from C, etc. To be able to guarantee
//! that we can always typecheck/translate any function, we have to have
//! complete knowledge of the whole ecosystem, not just our immediate
//! dependencies.
//!
//! So now as part of the "find a corresponding crate on the filesystem" step
//! above, this involves also finding all crates for *all upstream
//! dependencies*. This includes all dependencies transitively.
//!
//! ## Rlibs and Dylibs
//!
//! The compiler has two forms of intermediate dependencies. These are dubbed
//! rlibs and dylibs for the static and dynamic variants, respectively. An rlib
//! is a rustc-defined file format (currently just an ar archive) while a dylib
//! is a platform-defined dynamic library. Each library has a metadata somewhere
//! inside of it.
//!
//! When translating a crate name to a crate on the filesystem, we all of a
//! sudden need to take into account both rlibs and dylibs! Linkage later on may
//! use either one of these files, as each has their pros/cons. The job of crate
//! loading is to discover what's possible by finding all candidates.
//!
//! Most parts of this loading systems keep the dylib/rlib as just separate
//! variables.
//!
//! ## Where to look?
//!
//! We can't exactly scan your whole hard drive when looking for dependencies,
//! so we need to places to look. Currently the compiler will implicitly add the
//! target lib search path ($prefix/lib/rustlib/$target/lib) to any compilation,
//! and otherwise all -L flags are added to the search paths.
//!
//! ## What criterion to select on?
//!
//! This a pretty tricky area of loading crates. Given a file, how do we know
//! whether it's the right crate? Currently, the rules look along these lines:
//!
//! 1. Does the filename match an rlib/dylib pattern? That is to say, does the
//! filename have the right prefix/suffix?
//! 2. Does the filename have the right prefix for the crate name being queried?
//! This is filtering for files like `libfoo*.rlib` and such.
//! 3. Is the file an actual rust library? This is done by loading the metadata
//! from the library and making sure it's actually there.
//! 4. Does the name in the metadata agree with the name of the library?
//! 5. Does the target in the metadata agree with the current target?
//! 6. Does the SVH match? (more on this later)
//!
2014-08-02 01:40:21 +02:00
//! If the file answers `yes` to all these questions, then the file is
//! considered as being *candidate* for being accepted. It is illegal to have
//! more than two candidates as the compiler has no method by which to resolve
//! this conflict. Additionally, rlib/dylib candidates are considered
//! separately.
//!
//! After all this has happened, we have 1 or two files as candidates. These
//! represent the rlib/dylib file found for a library, and they're returned as
//! being found.
//!
//! ### What about versions?
//!
//! A lot of effort has been put forth to remove versioning from the compiler.
//! There have been forays in the past to have versioning baked in, but it was
//! largely always deemed insufficient to the point that it was recognized that
//! it's probably something the compiler shouldn't do anyway due to its
//! complicated nature and the state of the half-baked solutions.
//!
//! With a departure from versioning, the primary criterion for loading crates
//! is just the name of a crate. If we stopped here, it would imply that you
//! could never link two crates of the same name from different sources
//! together, which is clearly a bad state to be in.
//!
//! To resolve this problem, we come to the next section!
//!
//! # Expert Mode
//!
//! A number of flags have been added to the compiler to solve the "version
//! problem" in the previous section, as well as generally enabling more
//! powerful usage of the crate loading system of the compiler. The goal of
//! these flags and options are to enable third-party tools to drive the
//! compiler with prior knowledge about how the world should look.
//!
//! ## The `--extern` flag
//!
//! The compiler accepts a flag of this form a number of times:
//!
//! ```text
//! --extern crate-name=path/to/the/crate.rlib
//! ```
//!
//! This flag is basically the following letter to the compiler:
//!
//! > Dear rustc,
//! >
//! > When you are attempting to load the immediate dependency `crate-name`, I
//! > would like you too assume that the library is located at
//! > `path/to/the/crate.rlib`, and look nowhere else. Also, please do not
//! > assume that the path I specified has the name `crate-name`.
//!
//! This flag basically overrides most matching logic except for validating that
//! the file is indeed a rust library. The same `crate-name` can be specified
//! twice to specify the rlib/dylib pair.
//!
//! ## Enabling "multiple versions"
//!
//! This basically boils down to the ability to specify arbitrary packages to
//! the compiler. For example, if crate A wanted to use Bv1 and Bv2, then it
//! would look something like:
//!
//! ```ignore
//! extern crate b1;
//! extern crate b2;
//!
//! fn main() {}
//! ```
//!
//! and the compiler would be invoked as:
//!
//! ```text
//! rustc a.rs --extern b1=path/to/libb1.rlib --extern b2=path/to/libb2.rlib
//! ```
//!
//! In this scenario there are two crates named `b` and the compiler must be
//! manually driven to be informed where each crate is.
//!
//! ## Frobbing symbols
//!
//! One of the immediate problems with linking the same library together twice
//! in the same problem is dealing with duplicate symbols. The primary way to
//! deal with this in rustc is to add hashes to the end of each symbol.
//!
//! In order to force hashes to change between versions of a library, if
//! desired, the compiler exposes an option `-C metadata=foo`, which is used to
//! initially seed each symbol hash. The string `foo` is prepended to each
//! string-to-hash to ensure that symbols change over time.
//!
//! ## Loading transitive dependencies
//!
//! Dealing with same-named-but-distinct crates is not just a local problem, but
2014-08-02 01:40:21 +02:00
//! one that also needs to be dealt with for transitive dependencies. Note that
//! in the letter above `--extern` flags only apply to the *local* set of
//! dependencies, not the upstream transitive dependencies. Consider this
//! dependency graph:
//!
//! ```text
//! A.1 A.2
//! | |
//! | |
//! B C
//! \ /
//! \ /
//! D
//! ```
//!
//! In this scenario, when we compile `D`, we need to be able to distinctly
//! resolve `A.1` and `A.2`, but an `--extern` flag cannot apply to these
//! transitive dependencies.
//!
//! Note that the key idea here is that `B` and `C` are both *already compiled*.
//! That is, they have already resolved their dependencies. Due to unrelated
//! technical reasons, when a library is compiled, it is only compatible with
//! the *exact same* version of the upstream libraries it was compiled against.
//! We use the "Strict Version Hash" to identify the exact copy of an upstream
//! library.
//!
//! With this knowledge, we know that `B` and `C` will depend on `A` with
//! different SVH values, so we crawl the normal `-L` paths looking for
//! `liba*.rlib` and filter based on the contained SVH.
//!
//! In the end, this ends up not needing `--extern` to specify upstream
//! transitive dependencies.
//!
//! # Wrapping up
//!
//! That's the general overview of loading crates in the compiler, but it's by
//! no means all of the necessary details. Take a look at the rest of
//! metadata::loader or metadata::creader for all the juicy details!
use back::archive::{METADATA_FILENAME};
use back::svh::Svh;
use session::Session;
use llvm;
use llvm::{False, ObjectFile, mk_section_iter};
use llvm::archive_ro::ArchiveRO;
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
use metadata::cstore::{MetadataBlob, MetadataVec, MetadataArchive};
use metadata::decoder;
use metadata::encoder;
use metadata::filesearch::{FileSearch, FileMatches, FileDoesntMatch};
use syntax::codemap::Span;
use syntax::diagnostic::SpanHandler;
use util::fs;
use rustc_back::target::Target;
use std::ffi::CString;
2014-02-06 08:34:33 +01:00
use std::cmp;
use std::collections::{HashMap, HashSet};
use std::io::fs::PathExtensions;
2013-11-11 07:46:32 +01:00
use std::io;
use std::ptr;
use std::slice;
use std::time::Duration;
2014-02-20 04:08:12 +01:00
use flate;
pub struct CrateMismatch {
path: Path,
got: String,
}
pub struct Context<'a> {
pub sess: &'a Session,
pub span: Span,
pub ident: &'a str,
pub crate_name: &'a str,
pub hash: Option<&'a Svh>,
// points to either self.sess.target.target or self.sess.host, must match triple
pub target: &'a Target,
pub triple: &'a str,
pub filesearch: FileSearch<'a>,
pub root: &'a Option<CratePaths>,
pub rejected_via_hash: Vec<CrateMismatch>,
pub rejected_via_triple: Vec<CrateMismatch>,
pub should_match_name: bool,
}
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
pub struct Library {
pub dylib: Option<Path>,
pub rlib: Option<Path>,
pub metadata: MetadataBlob,
}
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
pub struct ArchiveMetadata {
_archive: ArchiveRO,
// points into self._archive
data: *const [u8],
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
}
pub struct CratePaths {
pub ident: String,
pub dylib: Option<Path>,
pub rlib: Option<Path>
}
impl CratePaths {
fn paths(&self) -> Vec<Path> {
match (&self.dylib, &self.rlib) {
(&None, &None) => vec!(),
(&Some(ref p), &None) |
(&None, &Some(ref p)) => vec!(p.clone()),
(&Some(ref p1), &Some(ref p2)) => vec!(p1.clone(), p2.clone()),
}
}
}
impl<'a> Context<'a> {
pub fn maybe_load_library_crate(&mut self) -> Option<Library> {
self.find_library_crate()
}
pub fn load_library_crate(&mut self) -> Library {
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
match self.find_library_crate() {
Some(t) => t,
None => {
self.report_load_errs();
unreachable!()
}
}
}
pub fn report_load_errs(&mut self) {
let message = if self.rejected_via_hash.len() > 0 {
format!("found possibly newer version of crate `{}`",
self.ident)
} else if self.rejected_via_triple.len() > 0 {
format!("couldn't find crate `{}` with expected target triple {}",
self.ident, self.triple)
} else {
format!("can't find crate for `{}`", self.ident)
};
let message = match self.root {
&None => message,
&Some(ref r) => format!("{} which `{}` depends on",
message, r.ident)
};
2015-01-07 17:58:31 +01:00
self.sess.span_err(self.span, &message[]);
if self.rejected_via_triple.len() > 0 {
let mismatches = self.rejected_via_triple.iter();
for (i, &CrateMismatch{ ref path, ref got }) in mismatches.enumerate() {
self.sess.fileline_note(self.span,
2015-01-07 17:58:31 +01:00
&format!("crate `{}`, path #{}, triple {}: {}",
self.ident, i+1, got, path.display())[]);
}
}
if self.rejected_via_hash.len() > 0 {
self.sess.span_note(self.span, "perhaps this crate needs \
to be recompiled?");
let mismatches = self.rejected_via_hash.iter();
for (i, &CrateMismatch{ ref path, .. }) in mismatches.enumerate() {
self.sess.fileline_note(self.span,
2015-01-07 17:58:31 +01:00
&format!("crate `{}` path {}{}: {}",
self.ident, "#", i+1, path.display())[]);
}
match self.root {
&None => {}
&Some(ref r) => {
for (i, path) in r.paths().iter().enumerate() {
self.sess.fileline_note(self.span,
2015-01-07 17:58:31 +01:00
&format!("crate `{}` path #{}: {}",
r.ident, i+1, path.display())[]);
}
}
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
}
self.sess.abort_if_errors();
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
fn find_library_crate(&mut self) -> Option<Library> {
// If an SVH is specified, then this is a transitive dependency that
// must be loaded via -L plus some filtering.
if self.hash.is_none() {
self.should_match_name = false;
match self.find_commandline_library() {
Some(l) => return Some(l),
None => {}
}
self.should_match_name = true;
}
let dypair = self.dylibname();
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
// want: crate_name.dir_part() + prefix + crate_name.file_part + "-"
2014-12-09 18:21:18 +01:00
let dylib_prefix = format!("{}{}", dypair.0, self.crate_name);
let rlib_prefix = format!("lib{}", self.crate_name);
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
let mut candidates = HashMap::new();
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
// First, find all possible candidate rlibs and dylibs purely based on
// the name of the files themselves. We're trying to match against an
// exact crate name and a possibly an exact hash.
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
//
// During this step, we can filter all found libraries based on the
// name and id found in the crate id (we ignore the path portion for
// filename matching), as well as the exact hash (if specified). If we
// end up having many candidates, we must look at the metadata to
// perform exact matches against hashes/crate ids. Note that opening up
// the metadata is where we do an exact match against the full contents
// of the crate id (path/name/id).
//
// The goal of this step is to look at as little metadata as possible.
self.filesearch.search(|path| {
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
let file = match path.filename_str() {
None => return FileDoesntMatch,
Some(file) => file,
};
2015-01-07 17:58:31 +01:00
let (hash, rlib) = if file.starts_with(&rlib_prefix[]) &&
file.ends_with(".rlib") {
(file.slice(rlib_prefix.len(), file.len() - ".rlib".len()),
true)
} else if file.starts_with(dylib_prefix.as_slice()) &&
2014-12-09 18:21:18 +01:00
file.ends_with(dypair.1.as_slice()) {
(file.slice(dylib_prefix.len(), file.len() - dypair.1.len()),
false)
} else {
return FileDoesntMatch
};
info!("lib candidate: {}", path.display());
2014-09-18 23:05:52 +02:00
let hash_str = hash.to_string();
let slot = candidates.entry(hash_str).get().unwrap_or_else(
|vacant_entry| vacant_entry.insert((HashSet::new(), HashSet::new())));
let (ref mut rlibs, ref mut dylibs) = *slot;
if rlib {
rlibs.insert(fs::realpath(path).unwrap());
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
} else {
dylibs.insert(fs::realpath(path).unwrap());
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
2014-09-18 23:05:52 +02:00
FileMatches
2013-11-29 03:03:38 +01:00
});
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
// We have now collected all known libraries into a set of candidates
// keyed of the filename hash listed. For each filename, we also have a
// list of rlibs/dylibs that apply. Here, we map each of these lists
// (per hash), to a Library candidate for returning.
//
// A Library candidate is created if the metadata for the set of
// libraries corresponds to the crate id and hash criteria that this
2014-04-21 06:49:39 +02:00
// search is being performed for.
let mut libraries = Vec::new();
2014-09-15 05:27:36 +02:00
for (_hash, (rlibs, dylibs)) in candidates.into_iter() {
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
let mut metadata = None;
let rlib = self.extract_one(rlibs, "rlib", &mut metadata);
let dylib = self.extract_one(dylibs, "dylib", &mut metadata);
match metadata {
Some(metadata) => {
libraries.push(Library {
dylib: dylib,
rlib: rlib,
metadata: metadata,
})
}
None => {}
}
}
// Having now translated all relevant found hashes into libraries, see
// what we've got and figure out if we found multiple candidates for
// libraries or not.
match libraries.len() {
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
0 => None,
2014-09-15 05:27:36 +02:00
1 => Some(libraries.into_iter().next().unwrap()),
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
_ => {
self.sess.span_err(self.span,
2015-01-07 17:58:31 +01:00
&format!("multiple matching crates for `{}`",
self.crate_name)[]);
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
self.sess.note("candidates:");
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
for lib in libraries.iter() {
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
match lib.dylib {
Some(ref p) => {
2015-01-07 17:58:31 +01:00
self.sess.note(&format!("path: {}",
p.display())[]);
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
None => {}
}
match lib.rlib {
Some(ref p) => {
2015-01-07 17:58:31 +01:00
self.sess.note(&format!("path: {}",
p.display())[]);
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
None => {}
}
let data = lib.metadata.as_slice();
let name = decoder::get_crate_name(data);
2015-01-07 17:58:31 +01:00
note_crate_name(self.sess.diagnostic(), &name[]);
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
None
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
}
}
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
// Attempts to extract *one* library from the set `m`. If the set has no
// elements, `None` is returned. If the set has more than one element, then
// the errors and notes are emitted about the set of libraries.
//
// With only one library in the set, this function will extract it, and then
// read the metadata from it if `*slot` is `None`. If the metadata couldn't
// be read, it is assumed that the file isn't a valid rust library (no
// errors are emitted).
fn extract_one(&mut self, m: HashSet<Path>, flavor: &str,
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
slot: &mut Option<MetadataBlob>) -> Option<Path> {
let mut ret = None::<Path>;
let mut error = 0u;
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
if slot.is_some() {
// FIXME(#10786): for an optimization, we only read one of the
// library's metadata sections. In theory we should
// read both, but reading dylib metadata is quite
// slow.
if m.len() == 0 {
return None
} else if m.len() == 1 {
2014-09-15 05:27:36 +02:00
return Some(m.into_iter().next().unwrap())
}
}
2014-09-15 05:27:36 +02:00
for lib in m.into_iter() {
info!("{} reading metadata from: {}", flavor, lib.display());
let metadata = match get_metadata_section(self.target.options.is_like_osx,
&lib) {
Ok(blob) => {
if self.crate_matches(blob.as_slice(), &lib) {
blob
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
} else {
info!("metadata mismatch");
continue
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
}
}
Err(_) => {
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
info!("no metadata found");
continue
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
};
if ret.is_some() {
self.sess.span_err(self.span,
2015-01-07 17:58:31 +01:00
&format!("multiple {} candidates for `{}` \
found",
flavor,
2015-01-07 17:58:31 +01:00
self.crate_name)[]);
self.sess.span_note(self.span,
2015-01-07 17:58:31 +01:00
&format!(r"candidate #1: {}",
ret.as_ref().unwrap()
2015-01-07 17:58:31 +01:00
.display())[]);
error = 1;
ret = None;
}
if error > 0 {
error += 1;
self.sess.span_note(self.span,
2015-01-07 17:58:31 +01:00
&format!(r"candidate #{}: {}", error,
lib.display())[]);
continue
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
*slot = Some(metadata);
ret = Some(lib);
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
return if error > 0 {None} else {ret}
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
}
fn crate_matches(&mut self, crate_data: &[u8], libpath: &Path) -> bool {
if self.should_match_name {
match decoder::maybe_get_crate_name(crate_data) {
Some(ref name) if self.crate_name == *name => {}
_ => { info!("Rejecting via crate name"); return false }
}
}
let hash = match decoder::maybe_get_crate_hash(crate_data) {
Some(hash) => hash, None => {
info!("Rejecting via lack of crate hash");
return false;
}
};
let triple = match decoder::get_crate_triple(crate_data) {
None => { debug!("triple not present"); return false }
Some(t) => t,
};
if triple != self.triple {
info!("Rejecting via crate triple: expected {} got {}", self.triple, triple);
self.rejected_via_triple.push(CrateMismatch {
path: libpath.clone(),
got: triple.to_string()
});
return false;
}
match self.hash {
None => true,
Some(myhash) => {
if *myhash != hash {
info!("Rejecting via hash: expected {} got {}", *myhash, hash);
self.rejected_via_hash.push(CrateMismatch {
path: libpath.clone(),
got: myhash.as_str().to_string()
});
false
} else {
true
}
}
}
}
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
// Returns the corresponding (prefix, suffix) that files need to have for
// dynamic libraries
fn dylibname(&self) -> (String, String) {
let t = &self.target;
(t.options.dll_prefix.clone(), t.options.dll_suffix.clone())
}
fn find_commandline_library(&mut self) -> Option<Library> {
let locs = match self.sess.opts.externs.get(self.crate_name) {
Some(s) => s,
None => return None,
};
// First, filter out all libraries that look suspicious. We only accept
// files which actually exist that have the correct naming scheme for
// rlibs/dylibs.
let sess = self.sess;
let dylibname = self.dylibname();
let mut rlibs = HashSet::new();
let mut dylibs = HashSet::new();
{
2015-01-07 17:58:31 +01:00
let mut locs = locs.iter().map(|l| Path::new(&l[])).filter(|loc| {
if !loc.exists() {
2015-01-07 17:58:31 +01:00
sess.err(&format!("extern location for {} does not exist: {}",
self.crate_name, loc.display())[]);
return false;
}
let file = match loc.filename_str() {
Some(file) => file,
None => {
2015-01-07 17:58:31 +01:00
sess.err(&format!("extern location for {} is not a file: {}",
self.crate_name, loc.display())[]);
return false;
}
};
if file.starts_with("lib") && file.ends_with(".rlib") {
return true
} else {
let (ref prefix, ref suffix) = dylibname;
2015-01-07 17:58:31 +01:00
if file.starts_with(&prefix[]) &&
file.ends_with(&suffix[]) {
return true
}
}
2015-01-07 17:58:31 +01:00
sess.err(&format!("extern location for {} is of an unknown type: {}",
self.crate_name, loc.display())[]);
false
});
// Now that we have an iterator of good candidates, make sure there's at
// most one rlib and at most one dylib.
for loc in locs {
if loc.filename_str().unwrap().ends_with(".rlib") {
rlibs.insert(fs::realpath(&loc).unwrap());
} else {
dylibs.insert(fs::realpath(&loc).unwrap());
}
}
};
// Extract the rlib/dylib pair.
let mut metadata = None;
let rlib = self.extract_one(rlibs, "rlib", &mut metadata);
let dylib = self.extract_one(dylibs, "dylib", &mut metadata);
if rlib.is_none() && dylib.is_none() { return None }
match metadata {
Some(metadata) => Some(Library {
dylib: dylib,
rlib: rlib,
metadata: metadata,
}),
None => None,
}
}
}
pub fn note_crate_name(diag: &SpanHandler, name: &str) {
2015-01-07 17:58:31 +01:00
diag.handler().note(&format!("crate name: {}", name)[]);
}
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
impl ArchiveMetadata {
fn new(ar: ArchiveRO) -> Option<ArchiveMetadata> {
let data = match ar.read(METADATA_FILENAME) {
Some(data) => data as *const [u8],
None => {
debug!("didn't find '{}' in the archive", METADATA_FILENAME);
return None;
}
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
};
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
Some(ArchiveMetadata {
_archive: ar,
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
data: data,
})
}
pub fn as_slice<'a>(&'a self) -> &'a [u8] { unsafe { &*self.data } }
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
}
// Just a small wrapper to time how long reading metadata takes.
fn get_metadata_section(is_osx: bool, filename: &Path) -> Result<MetadataBlob, String> {
let mut ret = None;
let dur = Duration::span(|| {
ret = Some(get_metadata_section_imp(is_osx, filename));
});
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
info!("reading {} => {}ms", filename.filename_display(),
dur.num_milliseconds());
return ret.unwrap();;
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
}
fn get_metadata_section_imp(is_osx: bool, filename: &Path) -> Result<MetadataBlob, String> {
if !filename.exists() {
return Err(format!("no such file: '{}'", filename.display()));
}
Store metadata separately in rlib files Right now whenever an rlib file is linked against, all of the metadata from the rlib is pulled in to the final staticlib or binary. The reason for this is that the metadata is currently stored in a section of the object file. Note that this is intentional for dynamic libraries in order to distribute metadata bundled with static libraries. This commit alters the situation for rlib libraries to instead store the metadata in a separate file in the archive. In doing so, when the archive is passed to the linker, none of the metadata will get pulled into the result executable. Furthermore, the metadata file is skipped when assembling rlibs into an archive. The snag in this implementation comes with multiple output formats. When generating a dylib, the metadata needs to be in the object file, but when generating an rlib this needs to be separate. In order to accomplish this, the metadata variable is inserted into an entirely separate LLVM Module which is then codegen'd into a different location (foo.metadata.o). This is then linked into dynamic libraries and silently ignored for rlib files. While changing how metadata is inserted into archives, I have also stopped compressing metadata when inserted into rlib files. We have wanted to stop compressing metadata, but the sections it creates in object file sections are apparently too large. Thankfully if it's just an arbitrary file it doesn't matter how large it is. I have seen massive reductions in executable sizes, as well as staticlib output sizes (to confirm that this is all working).
2013-12-04 02:41:01 +01:00
if filename.filename_str().unwrap().ends_with(".rlib") {
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
// Use ArchiveRO for speed here, it's backed by LLVM and uses mmap
// internally to read the file. We also avoid even using a memcpy by
// just keeping the archive along while the metadata is in use.
let archive = match ArchiveRO::open(filename) {
Some(ar) => ar,
None => {
debug!("llvm didn't like `{}`", filename.display());
return Err(format!("failed to read rlib metadata: '{}'",
filename.display()));
rustc: Optimize reading metadata by 4x We were previously reading metadata via `ar p`, but as learned from rustdoc awhile back, spawning a process to do something is pretty slow. Turns out LLVM has an Archive class to read archives, but it cannot write archives. This commits adds bindings to the read-only version of the LLVM archive class (with a new type that only has a read() method), and then it uses this class when reading the metadata out of rlibs. When you put this in tandem of not compressing the metadata, reading the metadata is 4x faster than it used to be The timings I got for reading metadata from the respective libraries was: libstd-04ff901e-0.9-pre.dylib => 100ms libstd-04ff901e-0.9-pre.rlib => 23ms librustuv-7945354c-0.9-pre.dylib => 4ms librustuv-7945354c-0.9-pre.rlib => 1ms librustc-5b94a16f-0.9-pre.dylib => 87ms librustc-5b94a16f-0.9-pre.rlib => 35ms libextra-a6ebb16f-0.9-pre.dylib => 63ms libextra-a6ebb16f-0.9-pre.rlib => 15ms libsyntax-2e4c0458-0.9-pre.dylib => 86ms libsyntax-2e4c0458-0.9-pre.rlib => 22ms In order to always take advantage of these faster metadata read-times, I sort the files in filesearch based on whether they have an rlib extension or not (prefer all rlib files first). Overall, this halved the compile time for a `fn main() {}` crate from 0.185s to 0.095s on my system (when preferring dynamic linking). Reading metadata is still the slowest pass of the compiler at 0.035s, but it's getting pretty close to linking at 0.021s! The next best optimization is to just not copy the metadata from LLVM because that's the most expensive part of reading metadata right now.
2013-12-17 05:58:21 +01:00
}
};
return match ArchiveMetadata::new(archive).map(|ar| MetadataArchive(ar)) {
None => {
return Err((format!("failed to read rlib metadata: '{}'",
filename.display())))
}
Some(blob) => return Ok(blob)
}
Store metadata separately in rlib files Right now whenever an rlib file is linked against, all of the metadata from the rlib is pulled in to the final staticlib or binary. The reason for this is that the metadata is currently stored in a section of the object file. Note that this is intentional for dynamic libraries in order to distribute metadata bundled with static libraries. This commit alters the situation for rlib libraries to instead store the metadata in a separate file in the archive. In doing so, when the archive is passed to the linker, none of the metadata will get pulled into the result executable. Furthermore, the metadata file is skipped when assembling rlibs into an archive. The snag in this implementation comes with multiple output formats. When generating a dylib, the metadata needs to be in the object file, but when generating an rlib this needs to be separate. In order to accomplish this, the metadata variable is inserted into an entirely separate LLVM Module which is then codegen'd into a different location (foo.metadata.o). This is then linked into dynamic libraries and silently ignored for rlib files. While changing how metadata is inserted into archives, I have also stopped compressing metadata when inserted into rlib files. We have wanted to stop compressing metadata, but the sections it creates in object file sections are apparently too large. Thankfully if it's just an arbitrary file it doesn't matter how large it is. I have seen massive reductions in executable sizes, as well as staticlib output sizes (to confirm that this is all working).
2013-12-04 02:41:01 +01:00
}
unsafe {
let buf = CString::from_slice(filename.as_vec());
let mb = llvm::LLVMRustCreateMemoryBufferWithContentsOfFile(buf.as_ptr());
if mb as int == 0 {
return Err(format!("error reading library: '{}'",
filename.display()))
}
Add generation of static libraries to rustc This commit implements the support necessary for generating both intermediate and result static rust libraries. This is an implementation of my thoughts in https://mail.mozilla.org/pipermail/rust-dev/2013-November/006686.html. When compiling a library, we still retain the "lib" option, although now there are "rlib", "staticlib", and "dylib" as options for crate_type (and these are stackable). The idea of "lib" is to generate the "compiler default" instead of having too choose (although all are interchangeable). For now I have left the "complier default" to be a dynamic library for size reasons. Of the rust libraries, lib{std,extra,rustuv} will bootstrap with an rlib/dylib pair, but lib{rustc,syntax,rustdoc,rustpkg} will only be built as a dynamic object. I chose this for size reasons, but also because you're probably not going to be embedding the rustc compiler anywhere any time soon. Other than the options outlined above, there are a few defaults/preferences that are now opinionated in the compiler: * If both a .dylib and .rlib are found for a rust library, the compiler will prefer the .rlib variant. This is overridable via the -Z prefer-dynamic option * If generating a "lib", the compiler will generate a dynamic library. This is overridable by explicitly saying what flavor you'd like (rlib, staticlib, dylib). * If no options are passed to the command line, and no crate_type is found in the destination crate, then an executable is generated With this change, you can successfully build a rust program with 0 dynamic dependencies on rust libraries. There is still a dynamic dependency on librustrt, but I plan on removing that in a subsequent commit. This change includes no tests just yet. Our current testing infrastructure/harnesses aren't very amenable to doing flavorful things with linking, so I'm planning on adding a new mode of testing which I believe belongs as a separate commit. Closes #552
2013-11-15 23:03:29 +01:00
let of = match ObjectFile::new(mb) {
Some(of) => of,
_ => {
return Err((format!("provided path not an object file: '{}'",
filename.display())))
}
};
let si = mk_section_iter(of.llof);
while llvm::LLVMIsSectionIteratorAtEnd(of.llof, si.llsi) == False {
let mut name_buf = ptr::null();
let name_len = llvm::LLVMRustGetSectionName(si.llsi, &mut name_buf);
let name = slice::from_raw_buf(&(name_buf as *const u8),
name_len as uint).to_vec();
let name = String::from_utf8(name).unwrap();
debug!("get_metadata_section: name {}", name);
if read_meta_section_name(is_osx) == name {
let cbuf = llvm::LLVMGetSectionContents(si.llsi);
let csz = llvm::LLVMGetSectionSize(si.llsi) as uint;
let cvbuf: *const u8 = cbuf as *const u8;
2013-06-20 07:52:02 +02:00
let vlen = encoder::metadata_encoding_version.len();
debug!("checking {} bytes of metadata-version stamp",
2013-06-20 07:52:02 +02:00
vlen);
2014-02-06 08:34:33 +01:00
let minsz = cmp::min(vlen, csz);
let buf0 = slice::from_raw_buf(&cvbuf, minsz);
let version_ok = buf0 == encoder::metadata_encoding_version;
if !version_ok {
return Err((format!("incompatible metadata version found: '{}'",
filename.display())));
}
2013-06-20 07:52:02 +02:00
let cvbuf1 = cvbuf.offset(vlen as int);
debug!("inflating {} bytes of compressed metadata",
csz - vlen);
let bytes = slice::from_raw_buf(&cvbuf1, csz-vlen);
match flate::inflate_bytes(bytes) {
Some(inflated) => return Ok(MetadataVec(inflated)),
None => {}
}
}
llvm::LLVMMoveToNextSection(si.llsi);
}
return Err(format!("metadata not found: '{}'", filename.display()));
}
}
pub fn meta_section_name(is_osx: bool) -> &'static str {
if is_osx {
"__DATA,__note.rustc"
} else {
".note.rustc"
}
}
pub fn read_meta_section_name(is_osx: bool) -> &'static str {
if is_osx {
"__note.rustc"
} else {
".note.rustc"
2013-03-13 09:22:01 +01:00
}
}
// A diagnostic function for dumping crate metadata to an output stream
pub fn list_file_metadata(is_osx: bool, path: &Path,
2014-01-30 03:42:19 +01:00
out: &mut io::Writer) -> io::IoResult<()> {
match get_metadata_section(is_osx, path) {
Ok(bytes) => decoder::list_crate_metadata(bytes.as_slice(), out),
Err(msg) => {
write!(out, "{}\n", msg)
Re-work loading crates with nicer errors This commit rewrites crate loading internally in attempt to look at less metadata and provide nicer errors. The loading is now split up into a few stages: 1. Collect a mapping of (hash => ~[Path]) for a set of candidate libraries for a given search. The hash is the hash in the filename and the Path is the location of the library in question. All candidates are filtered based on their prefix/suffix (dylib/rlib appropriate) and then the hash/version are split up and are compared (if necessary). This means that if you're looking for an exact hash of library you don't have to open up the metadata of all libraries named the same, but also in your path. 2. Once this mapping is constructed, each (hash, ~[Path]) pair is filtered down to just a Path. This is necessary because the same rlib could show up twice in the path in multiple locations. Right now the filenames are based on just the crate id, so this could be indicative of multiple version of a crate during one crate_id lifetime in the path. If multiple duplicate crates are found, an error is generated. 3. Now that we have a mapping of (hash => Path), we error on multiple versions saying that multiple versions were found. Only if there's one (hash => Path) pair do we actually return that Path and its metadata. With this restructuring, it restructures code so errors which were assertions previously are now first-class errors. Additionally, this should read much less metadata with lots of crates of the same name or same version in a path. Closes #11908
2014-02-10 21:50:53 +01:00
}
}
}