Keep track of position when deleting from a BTreeMap
This improves the performance of drain_filter and is needed for future Cursor support for BTreeMap.
cc @ssomers
r? @Mark-Simulacrum
Don't import integer and float modules, use assoc consts
Stop importing the standard library integer and float modules to reach the `MIN`, `MAX` and other constants. They are available directly on the primitive types now.
This PR is a follow up of #69860 which made sure we use the new constants in documentation.
This type of change touches a lot of files, and previously all my assoc int consts PRs had collisions and were accepted only after a long delay. So I'd prefer to do it in smaller steps now. Just removing these imports seem like a good next step.
r? @dtolnay
clarify comment in RawVec::into_box
On first reading I almost thought "len <= cap" would be all that there is to check here. Expand the comment to clarify that that is not the case.
Fix some aliasing issues in Vec
`Vec::extend` and `Vec::truncate` invalidated references into the vector even without reallocation, because they (implicitly) created a mutable reference covering the *entire* initialized part of the vector.
Fixes https://github.com/rust-lang/rust/issues/70301
I verified the fix by adding some new tests here that I ran in Miri.
This commit changes some usage of mem::forget into mem::ManuallyDrop
in some Vec, VecDeque, BTreeMap and Box methods.
Before the commit, the generated IR for some of the methods was
longer, and even after optimization, some unwinding artifacts were
still present.
Use associated numeric consts in documentation
Now when the associated constants on int/float types are stabilized and the recommended way of accessing said constants (#68952). We can start using it in this repository, and recommend it via documentation example code.
This PR is the reincarnation of #67913 minus the actual adding + stabilization of said constants. (EDIT: Now it's only changing the documentation. So users will see the new consts, but we don't yet update the internal code)
Because of how fast bit rot happens to PRs that touch this many files, it does not try to replace 100% of the old usage of the constants in the entire repo, but a good chunk of them.
Overhaul of the `AllocRef` trait to match allocator-wg's latest consens; Take 2
GitHub won't let me reopen#69889 so I make a new PR.
In addition to #69889 this fixes the unsoundness of `RawVec::into_box` when using allocators supporting overallocating. Also it uses `MemoryBlock` in `AllocRef` to unify `_in_place` methods by passing `&mut MemoryBlock`. Additionally, `RawVec` now checks for `size_of::<T>()` again and ignore every ZST. The internal capacity of `RawVec` isn't used by ZSTs anymore, as `into_box` now requires a length to be specified.
r? @Amanieu
fixesrust-lang/wg-allocators#38fixesrust-lang/wg-allocators#41fixesrust-lang/wg-allocators#44fixesrust-lang/wg-allocators#51
expand vec![] to Vec::new()
The current expansion of `vec![]` calls `into_vec` on a boxed slice, which results in longer IR, and even after optimization, some unwinding artifacts are still present in the IR. This PR uses `Vec::new()` for `vec![]`.
This also allows `vec![]` to be used in const expressions.
BTreeMap/BTreeSet: implement drain_filter
Provide an implementation of drain_filter for BTreeMap and BTreeSet. Should be optimal when the predicate picks only elements in leaf nodes with at least MIN_LEN remaining elements, which is a common case, at least when draining only a fraction of the map/set, and also when the predicate picks elements stored in internal nodes where the right subtree can easily let go of a replacement element.
The first commit adds benchmarks with an external, naive implementation. to compare how much this claimed optimality-in-some-cases is actually worth.
Optimize strip_prefix and strip_suffix with str patterns
As mentioned in https://github.com/rust-lang/rust/issues/67302#issuecomment-585639226.
I'm not sure whether adding these methods to `Pattern` is desirable—but they have default implementations so the change is backwards compatible. Plus it seems like they're slated for wholesale replacement soon anyway? #56345
----
Constructing a Searcher in strip_prefix and strip_suffix is
unnecessarily slow when the pattern is a fixed-length string. Add
strip_prefix and strip_suffix methods to the Pattern trait, and add
optimized implementations of these methods in the str implementation.
The old implementation is retained as the default for these methods.
Constructing a Searcher in strip_prefix and strip_suffix is
unnecessarily slow when the pattern is a fixed-length string. Add
strip_prefix and strip_suffix methods to the Pattern trait, and add
optimized implementations of these methods in the str implementation.
The old implementation is retained as the default for these methods.
BTreeMap testing: introduce symbolic constants and use height consistently
Doesn't change what or how much is tested, except for some exact integer types, just for convenience and because `node::CAPACITY` is a usize.
r? @RalfJung
They used to be covered by `optin_builtin_traits` but negative impls
are now applicable to all traits, not just auto traits.
This also adds docs in the unstable book for the current state of auto traits.
Currently, constructing a waker requires calling the unsafe
`Waker::from_raw` API. This API requires the user to manually construct
a vtable for the waker themself - which is both cumbersome and very
error prone. This API would provide an ergonomic, straightforward and
guaranteed memory-safe way of constructing a waker.
It has been our longstanding intention that the `Waker` type essentially
function as an `Arc<dyn Wake>`, with a `Wake` trait as defined here. Two
considerations prevented the original API from being shipped as simply
an `Arc<dyn Wake>`:
- We want to support futures on embedded systems, which may not have an
allocator, and in optimized executors for which this API may not be
best-suited. Therefore, we have always explicitly supported the
maximally-flexible (but also memory-unsafe) `RawWaker` API, and
`Waker` has always lived in libcore.
- Because `Waker` lives in libcore and `Arc` lives in liballoc, it has
not been feasible to provide a constructor for `Waker` from `Arc<dyn
Wake>`.
Therefore, the Wake trait was left out of the initial version of the
task waker API.
However, as Rust 1.41, it is possible under the more flexible orphan
rules to implement `From<Arc<W>> for Waker where W: Wake` in liballoc.
Therefore, we can now define this constructor even though `Waker` lives
in libcore.
This PR adds these APIs:
- A `Wake` trait, which contains two methods
- A required method `wake`, which is called by `Waker::wake`
- A provided method `wake_by_ref`, which is called by
`Waker::wake_by_ref` and which implementors can override if they
can optimize this use case.
- An implementation of `From<Arc<W>> for Waker where W: Wake + Send +
Sync + 'static`
- A similar implementation of `From<Arc<W>> for RawWaker`.
couple more clippy fixes (let_and_return, if_same_then_else)
* summarize if-else-code with identical blocks (clippy::if_same_then_else)
* don't create variable bindings just to return the bound value immediately (clippy::let_and_return)
Amend Rc/Arc::from_raw() docs regarding unsafety
[This](https://stackoverflow.com/questions/59671647/is-it-safe-to-clone-a-type-erased-arc-via-raw-pointer) question on SO boils down to "is it safe to `::from_raw()` a `Rc<T>`/`Arc<T>` using a dummy `T` even if `T` is never dereferenced via the new `Rc`/`Arc`?". It almost never is.
This PR amends the docs of `from_raw()` regarding this point.
BTreeMap: remove shared root
This replaces the shared root with `Option`s in the BTreeMap code, and then slightly cleans up the node manipulation code taking advantage of the removal of the shared root. I expect that further simplification is possible, but wanted to get this posted for initial review.
Note that `BTreeMap::new()` continues to not allocate.
Benchmarks seem within the margin of error/unaffected, as expected for an entirely predictable branch.
```
name alloc-bench-a ns/iter alloc-bench-b ns/iter diff ns/iter diff % speedup
btree::map::iter_mut_20 20 21 1 5.00% x 0.95
btree::set::clone_100 1,360 1,439 79 5.81% x 0.95
btree::set::clone_100_and_into_iter 1,319 1,434 115 8.72% x 0.92
btree::set::clone_10k 143,515 150,991 7,476 5.21% x 0.95
btree::set::clone_10k_and_clear 142,792 152,916 10,124 7.09% x 0.93
btree::set::clone_10k_and_into_iter 146,019 154,561 8,542 5.85% x 0.94
```
This makes ensure_root_is_owned return a reference to the (now guaranteed to
exist) root, allowing callers to operate on it without going through another
unwrap.
Unfortunately this is only rarely useful as it's frequently the case that both
the length and the root need to be accessed and field-level borrows in methods
don't yet exist.
The memory fences used previously in Arc implementation are not properly
understood by ThreadSanitizer as synchronization primitives. This had
unfortunate effect where running any non-trivial program compiled with
`-Z sanitizer=thread` would result in numerous false positives.
Replace acquire fences with acquire loads when using ThreadSanitizer to
address the issue.
Implement From<&mut str> for String
I ran into this missing impl when trying to do `String::from` on the result returned from this API in the `uuid` crate:
https://docs.rs/uuid/0.8.1/uuid/adapter/struct.Hyphenated.html#method.encode_lower
I wasn't sure what to put in the stability annotation. I'd appreciate some help with that :)
Implement Error for TryReserveError
I noticed that the Error trait wasn't implemented for TryReserveError. (#48043)
Not sure if the error messages and code style are 100% correct, it's my first time contributing to the Rust std.
Allow ZSTs in `AllocRef`
Allows ZSTs in all `AllocRef` methods. The implementation of `AllocRef` for `Global` and `System` were adjusted to reflect those changes.
This is the second item on the roadmap to support ZSTs in `AllocRef`: https://github.com/rust-lang/wg-allocators/issues/38#issuecomment-595861542
After this has landed, I will adapt `RawVec`, but since this will be a pretty big overhaul, it makes sense to do a different PR for it.
~~Requires #69794 to land first~~
r? @Amanieu
More documentation and simplification of BTreeMap's internals
Salvage the documentation and simplification from #67980, without changing the type locked down by debuginfo.
r? @rkruppe
cleanup more iterator usages (and other things)
* Improve weird formatting by moving comment inside else-code block.
* Use .any(x) instead of .find(x).is_some() on iterators.
* Use .nth(x) instead of .skip(x).next() on iterators.
* Simplify conditions like x + 1 <= y to x < y
* Use let instead of match to get value of enum with single variant.
Remove `usable_size` APIs
This removes the usable size APIs:
- remove `usable_size` (obv)
- change return type of allocating methods to include the allocated size
- remove `_excess` API
r? @Amanieu
closesrust-lang/wg-allocators#17
Clarify explanation of Vec<T> 'fn resize'
1. Clarified on what should implement `Clone` trait.
2. Minor grammar fix:
to be able clone => to be able **to** clone
BTreeMap navigation done safer & faster
It turns out that there was a faster way to do the tree navigation code bundled in #67073, by moving from edge to KV and from KV to next edge separately. It extracts most of the code as safe functions, and contains the duplication of handles within the short wrapper functions.
This somehow hits a sweet spot in the compiler because it reports boosts all over the board:
```
>cargo benchcmp pre3.txt posz4.txt --threshold 5
name pre3.txt ns/iter posz4.txt ns/iter diff ns/iter diff % speedup
btree::map::first_and_last_0 40 37 -3 -7.50% x 1.08
btree::map::first_and_last_100 58 44 -14 -24.14% x 1.32
btree::map::iter_1000 8,920 3,419 -5,501 -61.67% x 2.61
btree::map::iter_100000 1,069,290 411,615 -657,675 -61.51% x 2.60
btree::map::iter_20 169 58 -111 -65.68% x 2.91
btree::map::iter_mut_1000 8,701 3,303 -5,398 -62.04% x 2.63
btree::map::iter_mut_100000 1,034,560 405,975 -628,585 -60.76% x 2.55
btree::map::iter_mut_20 165 58 -107 -64.85% x 2.84
btree::set::clone_100 1,831 1,562 -269 -14.69% x 1.17
btree::set::clone_100_and_clear 1,831 1,565 -266 -14.53% x 1.17
btree::set::clone_100_and_into_iter 1,917 1,541 -376 -19.61% x 1.24
btree::set::clone_100_and_pop_all 2,609 2,441 -168 -6.44% x 1.07
btree::set::clone_100_and_remove_all 4,598 3,927 -671 -14.59% x 1.17
btree::set::clone_100_and_remove_half 2,765 2,551 -214 -7.74% x 1.08
btree::set::clone_10k 191,610 164,616 -26,994 -14.09% x 1.16
btree::set::clone_10k_and_clear 192,003 164,616 -27,387 -14.26% x 1.17
btree::set::clone_10k_and_into_iter 200,037 163,010 -37,027 -18.51% x 1.23
btree::set::clone_10k_and_pop_all 267,023 250,913 -16,110 -6.03% x 1.06
btree::set::clone_10k_and_remove_all 536,230 464,100 -72,130 -13.45% x 1.16
btree::set::clone_10k_and_remove_half 453,350 430,545 -22,805 -5.03% x 1.05
btree::set::difference_random_100_vs_100 1,787 801 -986 -55.18% x 2.23
btree::set::difference_random_100_vs_10k 2,978 2,696 -282 -9.47% x 1.10
btree::set::difference_random_10k_vs_100 111,075 54,734 -56,341 -50.72% x 2.03
btree::set::difference_random_10k_vs_10k 246,380 175,980 -70,400 -28.57% x 1.40
btree::set::difference_staggered_100_vs_100 1,789 951 -838 -46.84% x 1.88
btree::set::difference_staggered_100_vs_10k 2,798 2,606 -192 -6.86% x 1.07
btree::set::difference_staggered_10k_vs_10k 176,452 97,401 -79,051 -44.80% x 1.81
btree::set::intersection_100_neg_vs_10k_pos 34 32 -2 -5.88% x 1.06
btree::set::intersection_100_pos_vs_100_neg 30 27 -3 -10.00% x 1.11
btree::set::intersection_random_100_vs_100 1,537 613 -924 -60.12% x 2.51
btree::set::intersection_random_100_vs_10k 2,793 2,649 -144 -5.16% x 1.05
btree::set::intersection_random_10k_vs_10k 222,127 147,166 -74,961 -33.75% x 1.51
btree::set::intersection_staggered_100_vs_100 1,447 622 -825 -57.01% x 2.33
btree::set::intersection_staggered_100_vs_10k 2,606 2,382 -224 -8.60% x 1.09
btree::set::intersection_staggered_10k_vs_10k 143,620 58,790 -84,830 -59.07% x 2.44
btree::set::is_subset_100_vs_100 1,349 488 -861 -63.83% x 2.76
btree::set::is_subset_100_vs_10k 1,720 1,428 -292 -16.98% x 1.20
btree::set::is_subset_10k_vs_10k 135,984 48,527 -87,457 -64.31% x 2.80
```
The `first_and_last` ones are noise (they don't do iteration), the others seem genuine.
As always, approved by Miri.
Also, a separate commit with some more benchmarks of mutable behaviour (which also benefit).
r? @cuviper
Audit liballoc for leaks in `Drop` impls when user destructor panics
Inspired by https://github.com/rust-lang/rust/pull/67243 and https://github.com/rust-lang/rust/pull/67235, this audits and hopefully fixes the remaining `Drop` impls in liballoc for resource leaks in the presence of panics in destructors called by the affected `Drop` impl.
This does not touch `Hash{Map,Set}` since they live in hashbrown. They have similar issues though.
r? @KodrAus
Implement split_inclusive for slice and str
# Overview
* Implement `split_inclusive` for `slice` and `str` and `split_inclusive_mut` for `slice`
* `split_inclusive` is a substring/subslice splitting iterator that includes the matched part in the iterated substrings as a terminator.
* EDIT: The behaviour has now changed, as per @KodrAus 's input, to the same semantics with the `split_terminator` function. I updated the examples below.
* Two examples below:
```Rust
let data = "\nMäry häd ä little lämb\nLittle lämb\n";
let split: Vec<&str> = data.split_inclusive('\n').collect();
assert_eq!(split, ["\n", "Märy häd ä little lämb\n", "Little lämb\n"]);
```
```Rust
let uppercase_separated = "SheePSharKTurtlECaT";
let mut first_char = true;
let split: Vec<&str> = uppercase_separated.split_inclusive(|c: char| {
let split = !first_char && c.is_uppercase();
first_char = split;
split
}).collect();
assert_eq!(split, ["SheeP", "SharK", "TurtlE", "CaT"]);
```
# Justification for the API
* I was surprised to find that stdlib currently only has splitting iterators that leave out the matched part. In my experience, wanting to leave a substring terminator as a part of the substring is a pretty common usecase.
* This API is strictly more expressive than the standard `split` API: it's easy to get the behaviour of `split` by mapping a subslicing operation that drops the terminator. On the other hand it's impossible to derive this behaviour from `split` without using hacky and brittle `unsafe` code. The normal way to achieve this functionality would be implementing the iterator yourself.
* Especially when dealing with mutable slices, the only way currently is to use `split_at_mut`. This API provides an ergonomic alternative that plays to the strengths of the iterating capabilities of Rust. (Using `split_at_mut` iteratively used to be a real pain before NLL, fortunately the situation is a bit better now.)
# Discussion items
* <s>Does it make sense to mimic `split_terminator` in that the final empty slice would be left off in case of the string/slice ending with a terminator? It might do, as this use case is naturally geared towards considering the matching part as a terminator instead of a separator.</s>
* EDIT: The behaviour was changed to mimic `split_terminator`.
* Does it make sense to have `split_inclusive_mut` for `&mut str`?
Add LinkedList::remove()
LinkedList::remove() removes the element at the specified index and returns it.
I added this because I think having a remove function would be useful to have, and similar functions are in other containers, like Vec and HashMap.
I'm not sure if adding a feature like this requires an RFC or not, so I'm sorry if this PR is premature.
Preparation for allocator aware `Box`
This cleans up the `Box` code a bit, and uses `Box::from_raw(ptr)` instead of `Box(ptr)`.
Additionally, `box_free` and `exchange_malloc` now uses the `AllocRef` trait and a comment was added on how `box_free` is tied to `Box`.
This a preparation for an upcoming PR, which makes `Box` aware of an allocator.
r? @Amanieu
Derive Clone + Eq for std::string::FromUtf8Error
Implement `Clone` and `Eq` for `std::string::FromUtf8Error`.
Both the inner `Vec<u8>` and `std::str::Utf8Error` are also `Clone + Eq`, so I don't see why we shouldn't derive them on `FromUtf8Error` as well.
(impl are insta-stable, requiring FCP from T-libs.)
Fix and test implementation of BTreeMap's first/last_entry, pop_first/last
Properly implement and test `first_entry` & `last_entry` to fix problem report #68829
BtreeMap range_search spruced up
#39457 created a lower level entry point for `range_search` to operate on, but it's really not hard to move it up a level of abstraction, making it somewhat shorter and reusing existing unsafe code (`new_edge` is unsafe although it is currently not tagged as such).
Benchmark added. Comparison says there's no real difference:
```
>cargo benchcmp old3.txt new3.txt --threshold 5
name old3.txt ns/iter new3.txt ns/iter diff ns/iter diff % speedup
btree::map::find_seq_100 19 21 2 10.53% x 0.90
btree::map::range_excluded_unbounded 3,117 2,838 -279 -8.95% x 1.10
btree::map::range_included_unbounded 1,768 1,871 103 5.83% x 0.94
btree::set::intersection_10k_neg_vs_10k_pos 35 37 2 5.71% x 0.95
btree::set::intersection_staggered_100_vs_10k 2,488 2,314 -174 -6.99% x 1.08
btree::set::is_subset_10k_vs_100 3 2 -1 -33.33% x 1.50
```
r? @Mark-Simulacrum
Added upper bound of what vecs and boxes can allocate
Fixed issue #68593
I added a line of documentation to these two files to reflect that vectors and boxes ensure that they never allocate more than `isize::MAX` bytes.
r? @steveklabnik
BTreeMap: tag and explain unsafe internal functions or assert preconditions
#68418 concluded that it's not desirable to tag all internal functions with preconditions as being unsafe. This PR does it to some functions, documents why, and elsewhere enforces the preconditions with asserts.
Implement clone_from for BTreeMap and BTreeSet
See #28481. This results in up to 90% speedups on simple data types when `self` and `other` are the same size, and is generally comparable or faster. Some concerns:
1. This implementation requires an `Ord` bound on the `Clone` implementation for `BTreeMap` and `BTreeSet`. Since these structs can only be created externally for keys with `Ord` implemented, this should be fine? If not, there's certainly a less safe way to do this.
2. Changing `next_unchecked` on `RangeMut` to return mutable key references allows for replacing the entire overlapping portion of both maps without changing the external interface in any way. However, if `clone_from` fails it can leave the `BTreeMap` in an invalid state, which might be unacceptable.
~This probably needs an FCP since it changes a trait bound, but (as far as I know?) that change cannot break any external code.~
Rename `Alloc` to `AllocRef`
The allocator-wg has decided to merge this change upstream in https://github.com/rust-lang/wg-allocators/issues/8#issuecomment-577122958.
This renames `Alloc` to `AllocRef` because types that implement `Alloc` are a reference, smart pointer, or ZSTs. It is not possible to have an allocator like `MyAlloc([u8; N])`, that owns the memory and also implements `Alloc`, since that would mean, that moving a `Vec<T, MyAlloc>` would need to correct the internal pointer, which is not possible as we don't have move constructors.
For further explanation please see https://github.com/rust-lang/wg-allocators/issues/8#issuecomment-489464843 and the comments after that one.
Additionally it clarifies the semantics of `Clone` on an allocator. In the case of `AllocRef`, it is clear that the cloned handle still points to the same allocator instance, and that you can free data allocated from one handle with another handle.
The initial proposal was to rename `Alloc` to `AllocHandle`, but `Ref` expresses the semantics better than `Handle`. Also, the only appearance of `Handle` in `std` are for windows specific resources, which might be confusing.
Blocked on https://github.com/rust-lang/miri/pull/1160
Mainly for API parity with HashMap.
- Add BTreeMap::remove_entry
- Rewrite BTreeMap::remove to use remove_entry
- Use btreemap_remove_entry feature in doc comment
Stabilize ptr::slice_from_raw_parts[_mut]
Closes#36925, the tracking issue.
Initial impl: #60667
r? @rust-lang/libs
In addition to stabilizing, I've adjusted the example of `ptr::slice_from_raw_parts` to use `slice_from_raw_parts` instead of `slice_from_raw_parts_mut`, which was unnecessary for the example as written.
Use pointer offset instead of deref for A/Rc::into_raw
Internals thread: https://internals.rust-lang.org/t/rc-and-internal-mutability/11463/2?u=cad97
The current implementation of (`A`)`Rc::into_raw` uses the `Deref::deref` implementation to get the pointer-to-data that is returned. This is problematic in the proposed Stacked Borrow rules, as this only gets shared provenance over the data location. (Note that the strong/weak counts are `UnsafeCell` (`Cell`/`Atomic`) so shared provenance can still mutate them, but the data itself is not.) When promoted back to a real reference counted pointer, the restored pointer can be used for mutation through `::get_mut` (if it is the only surviving reference). However, this mutates through a pointer ultimately derived from a `&T` borrow, violating the Stacked Borrow rules.
There are three known potential solutions to this issue:
- Stacked Borrows is wrong, liballoc is correct.
- Fully admit (`A`)`Rc` as an "internal mutability" type and store the data payload in an `UnsafeCell` like the strong/weak counts are. (Note: this is not needed generally since the `RcBox`/`ArcInner` is stored behind a shared `NonNull` which maintains shared write provenance as a raw pointer.)
- Adjust `into_raw` to do direct manipulation of the pointer (like `from_raw`) so that it maintains write provenance and doesn't derive the pointer from a reference.
This PR implements the third option, as recommended by @RalfJung.
Potential future work: provide `as_raw` and `clone_raw` associated functions to allow the [`&T` -> (`A`)`Rc<T>` pattern](https://internals.rust-lang.org/t/rc-and-internal-mutability/11463/2?u=cad97) to be used soundly without creating (`A`)`Rc` from references.
Stein Somers
IgorPerikov
Dylan DPC
Amanieu d'Antras
Ralf Jung
Linus Färnstrand
Trevor Spiteri
Mazdak Farrokhzad
Yoshua Wuyts
DutchGhost
bors
Tim Diekmann
Bastian Kauschke
Nikhil Benesch
John Kåre Alsaker
Tim Diekmann
Niko Matsakis
Kornel
Saoirse Shipwreckt
Matthias Krüger
Mark Rousskov
Tomasz Miąsko
Jonas Schievink
Yuki Okushi
Lena Wildervanck
Ryan Lopopolo
Joshua Nelson
Christopher Durham
Dylan Nugent
TrolledWoods
Youngsuk Kim
Peter Todd
Bijan Tabatabai
Pyry Kontio
Lukas Lueg
kennytm
hman523
Oliver Middleton
Charles Gleason
Charles Gleason
Pedro de Brito
Billy Rieger
Charles Lew