As with the previous commit with `librand`, this commit shuffles around some
`collections` code. The new state of the world is similar to that of librand:
* The libcollections crate now only depends on libcore and liballoc.
* The standard library has a new module, `std::collections`. All functionality
of libcollections is reexported through this module.
I would like to stress that this change is purely cosmetic. There are very few
alterations to these primitives.
There are a number of notable points about the new organization:
* std::{str, slice, string, vec} all moved to libcollections. There is no reason
that these primitives shouldn't be necessarily usable in a freestanding
context that has allocation. These are all reexported in their usual places in
the standard library.
* The `hashmap`, and transitively the `lru_cache`, modules no longer reside in
`libcollections`, but rather in libstd. The reason for this is because the
`HashMap::new` contructor requires access to the OSRng for initially seeding
the hash map. Beyond this requirement, there is no reason that the hashmap
could not move to libcollections.
I do, however, have a plan to move the hash map to the collections module. The
`HashMap::new` function could be altered to require that the `H` hasher
parameter ascribe to the `Default` trait, allowing the entire `hashmap` module
to live in libcollections. The key idea would be that the default hasher would
be different in libstd. Something along the lines of:
// src/libstd/collections/mod.rs
pub type HashMap<K, V, H = RandomizedSipHasher> =
core_collections::HashMap<K, V, H>;
This is not possible today because you cannot invoke static methods through
type aliases. If we modified the compiler, however, to allow invocation of
static methods through type aliases, then this type definition would
essentially be switching the default hasher from `SipHasher` in libcollections
to a libstd-defined `RandomizedSipHasher` type. This type's `Default`
implementation would randomly seed the `SipHasher` instance, and otherwise
perform the same as `SipHasher`.
This future state doesn't seem incredibly far off, but until that time comes,
the hashmap module will live in libstd to not compromise on functionality.
* In preparation for the hashmap moving to libcollections, the `hash` module has
moved from libstd to libcollections. A previously snapshotted commit enables a
distinct `Writer` trait to live in the `hash` module which `Hash`
implementations are now parameterized over.
Due to using a custom trait, the `SipHasher` implementation has lost its
specialized methods for writing integers. These can be re-added
backwards-compatibly in the future via default methods if necessary, but the
FNV hashing should satisfy much of the need for speedier hashing.
A list of breaking changes:
* HashMap::{get, get_mut} no longer fails with the key formatted into the error
message with `{:?}`, instead, a generic message is printed. With backtraces,
it should still be not-too-hard to track down errors.
* The HashMap, HashSet, and LruCache types are now available through
std::collections instead of the collections crate.
* Manual implementations of hash should be parameterized over `hash::Writer`
instead of just `Writer`.
[breaking-change]
This commit shuffles around some of the `rand` code, along with some
reorganization. The new state of the world is as follows:
* The librand crate now only depends on libcore. This interface is experimental.
* The standard library has a new module, `std::rand`. This interface will
eventually become stable.
Unfortunately, this entailed more of a breaking change than just shuffling some
names around. The following breaking changes were made to the rand library:
* Rng::gen_vec() was removed. This has been replaced with Rng::gen_iter() which
will return an infinite stream of random values. Previous behavior can be
regained with `rng.gen_iter().take(n).collect()`
* Rng::gen_ascii_str() was removed. This has been replaced with
Rng::gen_ascii_chars() which will return an infinite stream of random ascii
characters. Similarly to gen_iter(), previous behavior can be emulated with
`rng.gen_ascii_chars().take(n).collect()`
* {IsaacRng, Isaac64Rng, XorShiftRng}::new() have all been removed. These all
relied on being able to use an OSRng for seeding, but this is no longer
available in librand (where these types are defined). To retain the same
functionality, these types now implement the `Rand` trait so they can be
generated with a random seed from another random number generator. This allows
the stdlib to use an OSRng to create seeded instances of these RNGs.
* Rand implementations for `Box<T>` and `@T` were removed. These seemed to be
pretty rare in the codebase, and it allows for librand to not depend on
liballoc. Additionally, other pointer types like Rc<T> and Arc<T> were not
supported. If this is undesirable, librand can depend on liballoc and regain
these implementations.
* The WeightedChoice structure is no longer built with a `Vec<Weighted<T>>`,
but rather a `&mut [Weighted<T>]`. This means that the WeightedChoice
structure now has a lifetime associated with it.
* The `sample` method on `Rng` has been moved to a top-level function in the
`rand` module due to its dependence on `Vec`.
cc #13851
[breaking-change]
This commit moves reflection (as well as the {:?} format modifier) to a new
libdebug crate, all of which is marked experimental.
This is a breaking change because it now requires the debug crate to be
explicitly linked if the :? format qualifier is used. This means that any code
using this feature will have to add `extern crate debug;` to the top of the
crate. Any code relying on reflection will also need to do this.
Closes#12019
[breaking-change]
I mostly tried to remain backwards compatible with old invocations of
the `configure` script; if you do not want to use `CC` et al., you
should not have to; you can keep using `--enable-clang` and/or
`--enable-ccache`.
The overall intention is to capture the following precedences for
guessing the C compiler:
1. Value of `CC` at make invocation time.
2. Value of `CC` at configure invocation time.
3. Compiler inferred at configure invocation time (`gcc` or `clang`).
The strategy is to check (at `configure` time) if each of the
environment variables is set, and if so, save its value in a
corresponding `CFG_` variable (e.g. `CFG_CC`).
Then, in the makefiles, if `CC` is not set but `CFG_CC` is, then we
use the `CFG_CC` setting as `CC`.
Also, I fold the potential user-provided `CFLAGS` and `CXXFLAGS`
values into all of the per-platform `CFLAGS` and `CXXFLAGS` settings.
(This was opposed to adding `$(CFLAGS)` in an ad-hoc manner to various
parts of the mk files.)
Fix#13805.
----
Note that if you try to set the compiler to clang via the `CC` and
`CXX` environment variables, you will probably need to also set
`CXXFLAGS` to `--enable-libcpp` so that LLVM will be configured
properly.
----
Introduce CFG_USING_CLANG, which is distinguished from
CFG_ENABLE_CLANG because the former represents "we think we're using
clang, choose appropriate warning-control options" while the latter
represents "we asked configure (or the host required) that we attempt
to use clang, so check that we have an appropriate version of clang."
The main reason I added this is that I wanted to allow the user to
choose clang via setting the `CC` environment variable, but I did not
want that method of selection to get confused with the user passing
the `--enable-clang` option.
----
A digression: The `configure` script does not infer the compiler
setting if `CC` is set; but if `--enable-clang` was passed, then it
*does* still attempt to validate that the clang version is compatible.
Supporting this required revising `CLANG_VERSION` check to be robust
in face of user-provided `CC` value.
In particular, on Travis, the `CC` is set to `gcc` and so the natural
thing to do is to attempt to use `gcc` as the compiler, but Travis is
also passing `--enable-clang` to configure. So, what is the right
answer in the face of these contradictory requests?
One approach would be to have `--enable-clang` supersede the setting
for `CC` (and instead just call whatever we inferred for `CFG_CLANG`).
That sounds maximally inflexible to me (pnkfelix): a developer
requesting a `CC` value probably wants it respected, and should be
able to set it to something else; it is harder for that developer to
hack our configure script to change its inferred path to clang.
A second approach would be to blindly use the `CC` value but keep
going through the clang version check when `--enable-clang` is turned
on. But on Travis (a Linux host), the `gcc` invocation won't print a
clang version, so we would not get past the CLANG_VERSION check in
that context.
A third approach would be to never run the CLANG_VERSION check if `CC`
is explicitly set. That is not a terrible idea; but if the user uses
`CC` to pass in a path to some other version of clang that they want
to test, probably should still send that through the `CLANG_VERSION`
check.
So in the end I (pnkfelix) took a fourth approach: do the
CLANG_VERSION check if `CC` is unset *or* if `CC` is set to a string
ending with `clang`. This way setting `CC` to things like
`path/to/clang` or `ccache clang` will still go through the
CLANG_VERSION check, while setting `CC` to `gcc` or some unknown
compiler will skip the CLANG_VERSION check (regardless of whether the
user passed --enable-clang to `configure`).
----
Drive-by fixes:
* The call that sets `CFG_CLANG_VERSION` was quoting `"$CFG_CC"` in
its invocation, but that does not play nicely with someone who sets
`$CFG_CC` to e.g. `ccache clang`, since you do not want to intepret
that whole string as a command.
(On the other hand, a path with spaces might need the quoted
invocation. Not sure which one of these corner use-cases is more
important to support.)
* Fix chk_cc error message to point user at `gcc` not `cc`.
Two line summary: Distinguish HOST_RPATH and TARGET_RPATH; added
RPATH_LINK_SEARCH; skip tests broken in stage1; general cleanup.
`HOST_RPATH_VAR$(1)_T_$(2)_H_$(3)` and `TARGET_RPATH_VAR$(1)_T_$(2)_H_$(3)`
both match the format of the old `RPATH_VAR$(1)_T_$(2)_H_$(3)` (which
is still being set the same way that it was before, to one of either
HOST/TARGET depending on what stage we are building). Namely, the format
is <XXX>_RPATH_VAR = "<LD_LIB_PATH_ENVVAR>=<COLON_SEP_PATH_ENTRIES>"
What this commit does:
* Pass both of the (newly introduced) HOST and TARGET rpath setup vars
to `maketest.py`
* Update `maketest.py` to no longer update the LD_LIBRARY_PATH itself
Instead, it passes along the HOST and TARGET rpath setup vars in
environment variables `HOST_RPATH_ENV` and `TARGET_RPATH_ENV`
* Also, pass the current stage number to maketest.py; it in turn
passes it (via an env var) to run-make tests.
This allows the run-make tests to selectively change behavior
(e.g. turn themselves off) to deal with incompatibilities with
e.g. stage1.
* Cleanup: Distinguish in tools.mk between the command to run (`RUN`)
and the file to generate to drive that command (`RUN_BINFILE`). The
main thing this enables is that `RUN` can now setup the
`TARGET_RPATH_ENV` without having to dirty up the runner code in
each of the `run-make` Makefiles.
* Cleanup: Factored out commands to delete dylib/rlib into
REMOVE_DYLIBS/REMOVE_RLIBS.
There were places where we were only calling `rm $(call DYLIB,foo)`
even though we really needed to get rid of the whole glob (at least
based on alex's findings on #13753 that removing the symlink does not
suffice).
Therefore rather than peppering the code with the awkward
`rm $(TMPDIR)/$(call DYLIB_GLOB,foo)`, I instead introduced a common
`REMOVE_DYLIBS` user function that expands into that when called.
After I adding an analogous `REMOVE_RLIBS`, I changed all of the
existing calls that rm dylibs or rlibs to use these routines
instead.
Note that the latter is not a true refactoring since I may have
changed cases where it was our intent to only remove the sym-link.
(But if that is the case, then we need to more deeply investigate
alex's findings on #13753 where the system was still dynamically
loading up the non-symlinked libraries that it finds on the load
path.)
* Added RPATH_LINK_SEARCH command and use it on Linux.
On some platforms, namely Linux, when you have libboot.so that has
its internal rpath set (to e.g. $(ORIGIN)/path/to/HOSTDIR), the
linker still complains when you do the link step and it does not
know where to find libraries that libboot.so depends upon that live
in HOSTDIR (think e.g. librustuv.so).
As far as I can tell, the GNU linker will consult the
LD_LIBRARY_PATH as part of the linking process to find such
libraries. But if you want to be more careful and not override
LD_LIBRARY_PATH for the `gcc` invocation, then you need some other
way to tell the linker where it can find the libraries that
libboot.so needs. The solution to this on Linux is the
`-Wl,-rpath-link` command line option.
However, this command line option does not exist on Mac OS X, (which
appears to be figuring out how to resolve the libboot.dylib
dependency by some other means, perhaps by consulting the rpath
setting within libboot.dylib).
So, in order to abstract over this distinction, I added the
RPATH_LINK_SEARCH macro to the run-make infrastructure and added
calls to it where necessary to get Linux working. On architectures
other than Linux, the macro expands to nothing.
* Disable miscellaneous tests atop stage1.
* An especially interesting instance of the previous bullet point:
Excuse regex from doing rustdoc tests atop stage1.
This was a (nearly-) final step to get `make check-stage1` working
again.
The use of a special-case check for regex here is ugly but is
analogous other similar checks for regex such as the one that landed
in PR #13844.
The way this is written, the user will get a reminder that
doc-crate-regex is being skipped whenever their rules attempt to do
the crate documentation tests. This is deliberate: I want people
running `make check-stage1` to be reminded about which cases are
being skipped. (But if such echo noise is considered offensive, it
can obviously be removed.)
* Got windows working with the above changes.
This portion of the commit is a cleanup revision of the (previously
mentioned on try builds) re-architecting of how the LD_LIBRARY_PATH
setup and extension is handled in order to accommodate Windows' (1.)
use of `$PATH` for that purpose and (2.) use of spaces in `$PATH`
entries (problematic for make and for interoperation with tools at
the shell).
* In addition, since the code has been rearchitected to pass the
HOST_RPATH_DIR/TARGET_RPATH_DIR rather than a whole sh
environment-variable setting command, there is no need to for the
convert_path_spec calls in maketest.py, which in fact were put in
place to placate Windows but were now causing the Windows builds to
fail. Instead we just convert the paths to absolute paths just like
all of the other path arguments.
Also, note for makefile hackers: apparently you cannot quote operands
to `ifeq` in Makefile (or at least, you need to be careful about
adding them, e.g. to only one side).
This commit is part of the libstd facade RFC, issue #13851. This creates a new
library, liballoc, which is intended to be the core allocation library for all
of Rust. It is pinned on the basic assumption that an allocation failure is an
abort or failure.
This module has inherited the heap/libc_heap modules from std::rt, the owned/rc
modules from std, and the arc module from libsync. These three pointers are
currently the three most core pointer implementations in Rust.
The UnsafeArc type in std::sync should be considered deprecated and replaced by
Arc<Unsafe<T>>. This commit does not currently migrate to this type, but future
commits will continue this refactoring.
Use sync:1️⃣:Once to fetch the mach_timebase_info only once when
running precise_time_ns(). This helps because mach_timebase_info() is
surprisingly inefficient. Also fix the order of operations when applying
the timebase to the mach absolute time value.
This improves the time on my machine from
```
test tests::bench_precise_time_ns ... bench: 157 ns/iter (+/- 4)
```
to
```
test tests::bench_precise_time_ns ... bench: 38 ns/iter (+/- 3)
```
and it will get even faster once #14174 lands.
By default, jemalloc is building itself with -g3 if the local compiler supports
it. It looks like this is generating a good deal of debug info that windows
isn't optimizing out (on the order of 18MB). Windows gcc/ld is also not
optimizing this data away, causing hello world to be 18MB in size.
There's no current real need for debugging jemalloc to a great extent, so this
commit manually passes -g1 to override -g3 which jemalloc is using. This is
confirmed to drop the size of executables on windows back to a more reasonable
size (2.0MB, as they were before).
Closes#14144
By default, jemalloc is building itself with -g3 if the local compiler supports
it. It looks like this is generating a good deal of debug info that windows
isn't optimizing out (on the order of 18MB). Windows gcc/ld is also not
optimizing this data away, causing hello world to be 18MB in size.
There's no current real need for debugging jemalloc to a great extent, so this
commit manually passes -g1 to override -g3 which jemalloc is using. This is
confirmed to drop the size of executables on windows back to a more reasonable
size (2.0MB, as they were before).
Closes#14144
Passing `--pretty flowgraph=<NODEID>` makes rustc print a control flow graph.
In pratice, you will also need to pass the additional option:
`-o <FILE>` to emit output to a `.dot` file for graphviz.
(You can only print the flow-graph for a particular block in the AST.)
----
An interesting implementation detail is the way the code puts both the
node index (`cfg::CFGIndex`) and a reference to the payload
(`cfg::CFGNode`) into the single `Node` type that is used for
labelling and walking the graph. I had once mistakenly thought that I
only wanted the `cfg::CFGNode`, but for labelling, you really want the
cfg index too, rather than e.g. trying to use the `ast::NodeId` as the
label (which breaks down e.g. due to `ast::DUMMY_NODE_ID`).
----
As a drive-by fix, I had to fix `rustc::middle::cfg::construct`
interface to reflect changes that have happened on the master branch
while I was getting this integrated into the compiler. (The next
commit actually adds tests of the `--pretty flowgraph` functionality,
so that should ensure that the `rustc::middle::cfg` code does not go
stale again.)
The core library in theory has 0 dependencies, but in practice it has some in
order for it to be efficient. These dependencies are in the form of the basic
memory operations provided by libc traditionally, such as memset, memcmp, etc.
These functions are trivial to implement and themselves have 0 dependencies.
This commit adds a new crate, librlibc, which will serve the purpose of
providing these dependencies. The crate is never linked to by default, but is
available to be linked to by downstream consumers. Normally these functions are
provided by the system libc, but in other freestanding contexts a libc may not
be available. In these cases, librlibc will suffice for enabling execution with
libcore.
cc #10116
The current suite of benchmarks for the standard distribution take a significant
amount of time to run, but it's unclear whether we're gaining any benefit from
running them. Some specific pain points:
* No one is looking at the data generated by the benchmarks. We have no graphs
or analysis of what's happening, so all the data is largely being cast into
the void.
* No benchmark has ever uncovered a bug, they have always run successfully.
* Benchmarks not only take a significant amount of time to run, but also take a
significant amount of time to compile. I don't think we should mitigate this
for now because it's useful to ensure that they do indeed still compile.
This commit disables --bench test runs by default as part of `make check`,
flipping the NO_BENCH environment variable to a PLEASE_BENCH variable which will
manually enable benchmarking. If and when a dedicated bot is set up for
benchmarking, this flag can be enabled on that bot.
There's no need to include this specific flag just for android. We can
already deal with what it tries to solve by using -C linker=/path/to/cc
and -C ar=/path/to/ar. The Makefiles for rustc already set this up when
we're crosscompiling.
I did add the flag to compiletest though so it can find gdb. Though, I'm
pretty sure we don't run debuginfo tests on android anyways right now.
[breaking-change]
This adds a `std::rt::heap` module with a nice allocator API. It's a
step towards fixing #13094 and is a starting point for working on a
generic allocator trait.
The revision used for the jemalloc submodule is the stable 3.6.0 release.
Closes#11807
This code does not belong in libstd, and rather belongs in a dedicated crate. In
the future, the syntax::ext::format module should move to the fmt_macros crate
(hence the name of the crate), but for now the fmt_macros crate will only
contain the format string parser.
The entire fmt_macros crate is marked #[experimental] because it is not meant
for general consumption, only the format!() interface is officially supported,
not the internals.
This is a breaking change for anyone using the internals of std::fmt::parse.
Some of the flags have moved to std::fmt::rt, while the actual parsing support
has all moved to the fmt_macros library.
[breaking-change]
This code does not belong in libstd, and rather belongs in a dedicated crate. In
the future, the syntax::ext::format module should move to the fmt_macros crate
(hence the name of the crate), but for now the fmt_macros crate will only
contain the format string parser.
The entire fmt_macros crate is marked #[experimental] because it is not meant
for general consumption, only the format!() interface is officially supported,
not the internals.
This is a breaking change for anyone using the internals of std::fmt::parse.
Some of the flags have moved to std::fmt::rt, while the actual parsing support
has all moved to the fmt_macros library.
[breaking-change]
This pull request contains preparations for adding LLDB autotests:
+ the debuginfo tests are split into debuginfo-gdb and debuginfo-lldb
+ the `compiletest` tool is updated to support the debuginfo-lldb mode
+ tests.mk is modified to provide debuginfo-gdb and debuginfo-lldb make targets
+ GDB test cases are moved from `src/test/debug-info` to `src/test/debuginfo-gdb`
+ configure will now look for LLDB and set the appropriate CFG variables
+ the `lldb_batchmode.py` script is added to `src/etc`. It emulates GDB's batch mode
The LLDB autotests themselves are not part of this PR. Those will probable require some manual work on the test bots to make them work for the first time. Better to get these unproblematic preliminaries out of the way in a separate step.
This is the second step in implementing #13851. This PR cannot currently land until a snapshot exists with #13892, but I imagine that this review will take longer.
This PR refactors a large amount of functionality outside of the standard library into a new library, libcore. This new library has 0 dependencies (in theory). In practice, this library currently depends on these symbols being available:
* `rust_begin_unwind` and `rust_fail_bounds_check` - These are the two entry points of failure in libcore. The symbols are provided by libstd currently. In the future (see the bullets on #13851) this will be officially supported with nice error mesages. Additionally, there will only be one failure entry point once `std::fmt` migrates to libcore.
* `memcpy` - This is often generated by LLVM. This is also quite trivial to implement for any platform, so I'm not too worried about this.
* `memcmp` - This is required for comparing strings. This function is quite common *everywhere*, so I don't feel to bad about relying on a consumer of libcore to define it.
* `malloc` and `free` - This is quite unfortunate, and is a temporary stopgap until we deal with the `~` situation. More details can be found in the module `core::should_not_exist`
* `fmod` and `fmodf` - These exist because the `Rem` trait is defined in libcore, so the `Rem` implementation for floats must also be defined in libcore. I imagine that any platform using floating-point modulus will have these symbols anyway, and otherwise they will be optimized out.
* `fdim` and `fdimf` - Like `fmod`, these are from the `Signed` trait being defined in libcore. I don't expect this to be much of a problem
These dependencies all "Just Work" for now because libcore only exists as an rlib, not as a dylib.
The commits themselves are organized to show that the overall diff of this extraction is not all that large. Most modules were able to be moved with very few modifications. The primary module left out of this iteration is `std::fmt`. I plan on migrating the `fmt` module to libcore, but I chose to not do so at this time because it had implications on the `Writer` trait that I wanted to deal with in isolation. There are a few breaking changes in these commits, but they are fairly minor, and are all labeled with `[breaking-change]`.
The nastiest parts of this movement come up with `~[T]` and `~str` being language-defined types today. I believe that much of this nastiness will get better over time as we migrate towards `Vec<T>` and `Str` (or whatever the types will be named). There will likely always be some extension traits, but the situation won't be as bad as it is today.
Known deficiencies:
* rustdoc will get worse in terms of readability. This is the next issue I will tackle as part of #13851. If others think that the rustdoc change should happen first, I can also table this to fix rustdoc first.
* The compiler reveals that all these types are reexports via error messages like `core::option::Option`. This is filed as #13065, and I believe that issue would have a higher priority now. I do not currently plan on fixing that as part of #13851. If others believe that this issue should be fixed, I can also place it on the roadmap for #13851.
I recommend viewing these changes on a commit-by-commit basis. The overall change is likely too overwhelming to take in.
Compile-fail tests for syntax extensions belong in this suite which has correct
dependencies on all artifacts rather than just the target artifacts.
Closes#13818
This primary fix brought on by this upgrade is the proper matching of the ```
and ~~~ doc blocks. This also moves hoedown to a git submodule rather than a
bundled repository.
Additionally, hoedown is stricter about code blocks, so this ended up fixing a
lot of invalid code blocks (ending with " ```" instead of "```", or ending with
"~~~~" instead of "~~~").
Closes#12776
These fonts were moved into place by rust's makefiles, but rustdoc is widely
used outside of rustc itself. This moves the fonts into the rustdoc binary,
similarly to the other static assets, and writes them to the output location
whenever rustdoc generates documentation.
Closes#13593Closes#13787
