Commit Graph

924 Commits

Author SHA1 Message Date
bors
c00a4648a4 Auto merge of #78833 - CDirkx:parse_prefix, r=dtolnay
Refactor and fix `parse_prefix` on Windows

This PR is an extension of #78692 as well as a general refactor of `parse_prefix`:

**Fixes**:
There are two errors in the current implementation of `parse_prefix`:

Firstly, in the current implementation only `\` is recognized as a separator character in device namespace prefixes. This behavior is only correct for verbatim paths; `"\\.\C:/foo"` should be parsed as `"C:"` instead of `"C:/foo"`.

Secondly, the current implementation only handles single separator characters. In non-verbatim paths a series of separator characters should be recognized as a single boundary, e.g. the UNC path `"\\localhost\\\\\\C$\foo"` should be parsed as `"\\localhost\\\\\\C$"` and then `UNC(server: "localhost", share: "C$")`, but currently it is not parsed at all, because it starts being parsed as `\\localhost\` and then has an invalid empty share location.

Paths like `"\\.\C:/foo"` and `"\\localhost\\\\\\C$\foo"` are valid on Windows, they are equivalent to just `"C:\foo"`.

**Refactoring**:
All uses of `&[u8]` within `parse_prefix` are extracted to helper functions and`&OsStr` is used instead. This reduces the number of places unsafe is used:
- `get_first_two_components` is adapted to the more general `parse_next_component` and used in more places
- code for parsing drive prefixes is extracted to `parse_drive`
2020-12-16 00:47:50 +00:00
bors
fa41639427 Auto merge of #77618 - fusion-engineering-forks:windows-parker, r=Amanieu
Add fast futex-based thread parker for Windows.

This adds a fast futex-based thread parker for Windows. It either uses WaitOnAddress+WakeByAddressSingle or NT Keyed Events (NtWaitForKeyedEvent+NtReleaseKeyedEvent), depending on which is available. Together, this makes this thread parker work for Windows XP and up. Before this change, park()/unpark() did not work on Windows XP: it needs condition variables, which only exist since Windows Vista.

---

Unfortunately, NT Keyed Events are an undocumented Windows API. However:
- This API is relatively simple with obvious behaviour, and there are several (unofficial) articles documenting the details. [1]
- parking_lot has been using this API for years (on Windows versions before Windows 8). [2] Many big projects extensively use parking_lot, such as servo and the Rust compiler itself.
- It is the underlying API used by Windows SRW locks and Windows critical sections. [3] [4]
- The source code of the implementations of Wine, ReactOs, and Windows XP are available and match the expected behaviour.
- The main risk with an undocumented API is that it might change in the future. But since we only use it for older versions of Windows, that's not a problem.
- Even if these functions do not block or wake as we expect (which is unlikely, see all previous points), this implementation would still be memory safe. The NT Keyed Events API is only used to sleep/block in the right place.

[1]\: http://www.locklessinc.com/articles/keyed_events/
[2]\: https://github.com/Amanieu/parking_lot/commit/43abbc964e
[3]\: https://docs.microsoft.com/en-us/archive/msdn-magazine/2012/november/windows-with-c-the-evolution-of-synchronization-in-windows-and-c
[4]\: Windows Internals, Part 1, ISBN 9780735671300

---

The choice of fallback API is inspired by parking_lot(_core), but the implementation of this thread parker is different. While parking_lot has no use for a fast path (park() directly returning if unpark() was already called), this implementation has a fast path that returns without even checking which waiting/waking API to use, as the same atomic variable with compatible states is used in all cases.
2020-12-14 16:41:14 +00:00
Guillaume Gomez
5d8b2a5bf1
Rollup merge of #79918 - woodruffw-forks:ww/doc-initializer-side-effects, r=dtolnay
doc(array,vec): add notes about side effects when empty-initializing

Copying some context from a conversation in the Rust discord:

* Both `vec![T; 0]` and `[T; 0]` are syntactically valid, and produce empty containers of their respective types

* Both *also* have side effects:

```rust
fn side_effect() -> String {
    println!("side effect!");

    "foo".into()
}

fn main() {
    println!("before!");

    let x = vec![side_effect(); 0];

    let y = [side_effect(); 0];

    println!("{:?}, {:?}", x, y);
}
```

produces:

```
before!
side effect!
side effect!
[], []
```

This PR just adds two small notes to each's documentation, warning users that side effects can occur.

I've also submitted a clippy proposal: https://github.com/rust-lang/rust-clippy/issues/6439
2020-12-14 14:43:44 +01:00
Yuki Okushi
d559bb6707
Rollup merge of #79398 - pickfire:keyword, r=Dylan-DPC
Link loop/for keyword

Even though the reference already have all of these, I am just adding related keywords in the see also to let others easily click on the related keyword.
2020-12-13 11:05:30 +09:00
William Woodruff
d986924eb1
doc: apply suggestions 2020-12-11 10:09:40 -05:00
bors
a2e29d67c2 Auto merge of #79893 - RalfJung:forget-windows, r=oli-obk
Windows TLS: ManuallyDrop instead of mem::forget

The Windows TLS implementation still used `mem::forget` instead of `ManuallyDrop`, leading to the usual problem of "using" the `Box` when it should not be used any more.
2020-12-11 07:54:35 +00:00
Tyler Mandry
17ec4b8258
Rollup merge of #79809 - Eric-Arellano:split-once, r=matklad
Dogfood `str_split_once()`

Part of https://github.com/rust-lang/rust/issues/74773.

Beyond increased clarity, this fixes some instances of a common confusion with how `splitn(2)` behaves: the first element will always be `Some()`, regardless of the delimiter, and even if the value is empty.

Given this code:

```rust
fn main() {
    let val = "...";
    let mut iter = val.splitn(2, '=');
    println!("Input: {:?}, first: {:?}, second: {:?}", val, iter.next(), iter.next());
}
```

We get:

```
Input: "no_delimiter", first: Some("no_delimiter"), second: None
Input: "k=v", first: Some("k"), second: Some("v")
Input: "=", first: Some(""), second: Some("")
```

Using `str_split_once()` makes more clear what happens when the delimiter is not found.
2020-12-10 21:33:08 -08:00
Tyler Mandry
a8c19e1b48
Rollup merge of #79375 - vext01:kernel-copy-temps, r=bjorn3
Make the kernel_copy tests more robust/concurrent.

These tests write to the same filenames in /tmp and in some cases these files don't get cleaned up properly. This caused issues for us when different users run the tests on the same system, e.g.:

```
---- sys::unix::kernel_copy::tests::bench_file_to_file_copy stdout ----
thread 'sys::unix::kernel_copy::tests::bench_file_to_file_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs:71:10
---- sys::unix::kernel_copy::tests::bench_file_to_socket_copy stdout ----
thread 'sys::unix::kernel_copy::tests::bench_file_to_socket_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs💯10
```

Use `std::sys_common::io__test::tmpdir()` to solve this.

CC ``@the8472.``
2020-12-10 21:33:02 -08:00
Tyler Mandry
1b4ffe4705
Rollup merge of #77027 - termhn:mul_add_doc_change, r=m-ou-se
Improve documentation for `std::{f32,f64}::mul_add`

Makes it more clear that performance improvement is not guaranteed when using FMA, even when the target architecture supports it natively.
2020-12-10 21:32:59 -08:00
bors
8cef65fde3 Auto merge of #77801 - fusion-engineering-forks:pin-mutex, r=Mark-Simulacrum
Enforce no-move rule of ReentrantMutex using Pin and fix UB in stdio

A `sys_common::ReentrantMutex` may not be moved after initializing it with `.init()`. This was not enforced, but only stated as a requirement in the comments on the unsafe functions. This change enforces this no-moving rule using `Pin`, by changing `&self` to a `Pin` in the `init()` and `lock()` functions.

This uncovered a bug I introduced in #77154: stdio.rs (the only user of ReentrantMutex) called `init()` on its ReentrantMutexes while constructing them in the intializer of `SyncOnceCell::get_or_init`, which would move them afterwards. Interestingly, the ReentrantMutex unit tests already had the same bug, so this invalid usage has been tested on all (CI-tested) platforms for a long time. Apparently this doesn't break badly on any of the major platforms, but it does break the rules.\*

To be able to keep using SyncOnceCell, this adds a `SyncOnceCell::get_or_init_pin` function, which makes it possible to work with pinned values inside a (pinned) SyncOnceCell. Whether this function should be public or not and what its exact behaviour and interface should be if it would be public is something I'd like to leave for a separate issue or PR. In this PR, this function is internal-only and marked with `pub(crate)`.

\* Note: That bug is now included in 1.48, while this patch can only make it to ~~1.49~~ 1.50. We should consider the implications of 1.48 shipping with a wrong usage of `pthread_mutex_t` / `CRITICAL_SECTION` / .. which technically invokes UB according to their specification. The risk is very low, considering the objects are not 'used' (locked) before the move, and the ReentrantMutex unit tests have verified this works fine in practice.

Edit: This has been backported and included in 1.48. And soon 1.49 too.

---

In future changes, I want to push this usage of Pin further inside `sys` instead of only `sys_common`, and apply it to all 'unmovable' objects there (`Mutex`, `Condvar`, `RwLock`). Also, while `sys_common`'s mutexes and condvars are already taken care of by #77147 and #77648, its `RwLock` should still be made movable or get pinned.
2020-12-10 23:43:20 +00:00
William Woodruff
9cf2516251
doc(array,vec): add notes about side effects when empty-initializing 2020-12-10 17:47:28 -05:00
Ralf Jung
594b451ccc Windows TLS: ManuallyDrop instead of mem::forget 2020-12-10 11:07:39 +01:00
bors
e413d89aa7 Auto merge of #79274 - the8472:probe-eperm, r=nagisa
implement better availability probing for copy_file_range

Followup to https://github.com/rust-lang/rust/pull/75428#discussion_r469616547

Previously syscall detection was overly pessimistic. Any attempt to copy to an immutable file (EPERM) would disable copy_file_range support for the whole process.

The change tries to copy_file_range on invalid file descriptors which will never run into the immutable file case and thus we can clearly distinguish syscall availability.
2020-12-10 03:11:27 +00:00
The8472
7647d03c33 Improve comment grammar 2020-12-09 21:31:37 +01:00
The8472
028754a2f7 implement better availability probing for copy_file_range
previously any attempt to copy to an immutable file (EPERM) would disable
copy_file_range support for the whole process.
2020-12-09 21:31:37 +01:00
bors
f0f68778f7 Auto merge of #77611 - oli-obk:atomic_miri_leakage, r=nagisa
Directly use raw pointers in `AtomicPtr` store/load

I was unable to find any reason for this limitation in the latest source of LLVM or in the documentation [here](http://llvm.org/docs/Atomics.html#libcalls-atomic).

fixes https://github.com/rust-lang/miri/issues/1574
2020-12-09 19:53:23 +00:00
bors
c16d52db77 Auto merge of #79387 - woodruffw-forks:ww/peer-cred-pid-macos, r=Amanieu
ext/ucred: Support PID in peer creds on macOS

This is a follow-up to https://github.com/rust-lang/rust/pull/75148 (RFC: https://github.com/rust-lang/rust/issues/42839).

The original PR used `getpeereid` on macOS and the BSDs, since they don't (generally) support the `SO_PEERCRED` mechanism that Linux supplies.

This PR splits the macOS/iOS implementation of `peer_cred()` from that of the BSDs, since macOS supplies the `LOCAL_PEERPID` sockopt as a source of the missing PID. It also adds a `cfg`-gated tests that ensures that platforms with support for PIDs in `UCred` have the expected data.
2020-12-09 17:27:35 +00:00
bors
2c56ea38b0 Auto merge of #78768 - mzabaluev:optimize-buf-writer, r=cramertj
Use is_write_vectored to optimize the write_vectored implementation for BufWriter

In case when the underlying writer does not have an efficient implementation `write_vectored`, the present implementation of
`write_vectored` for `BufWriter` may still forward vectored writes directly to the writer depending on the total length of the data. This misses the advantage of buffering, as the actually written slice may be small.

Provide an alternative code path for the non-vectored case, where the slices passed to `BufWriter` are coalesced in the buffer before being flushed to the underlying writer with plain `write` calls. The buffer is only bypassed if an individual slice's length is at least as large as the buffer.

Remove a FIXME comment referring to #72919 as the issue has been closed with an explanation provided.
2020-12-09 01:54:08 +00:00
Mara Bos
67c18fdec5 Use Pin for the 'don't move' requirement of ReentrantMutex.
The code in io::stdio before this change misused the ReentrantMutexes,
by calling init() on them and moving them afterwards. Now that
ReentrantMutex requires Pin for init(), this mistake is no longer easy
to make.
2020-12-08 22:57:57 +01:00
Mara Bos
8fe90966e1 Add (internal-only) SyncOnceCell::get_or_init_pin. 2020-12-08 22:57:50 +01:00
Mara Bos
9dc7f13c39 Remove unnecessary import of crate::marker in std::sys_common::remutex.
It was used for marker::Send, but Send is already in scope.
2020-12-08 22:57:49 +01:00
Mara Bos
2bc5d44ca9 Fix outdated comment about not needing to flush stderr. 2020-12-08 22:57:49 +01:00
Eric Arellano
a3174de9ff Fix net.rs - rsplitn() returns a reverse iterator 2020-12-07 18:47:10 -07:00
Eric Arellano
d2de69da2e Dogfood 'str_split_once()` in the std lib 2020-12-07 14:24:05 -07:00
Jethro Beekman
9703bb8192 Fix SGX CI, take 3
Broken in #79038
2020-12-07 15:22:34 +01:00
bors
ddafcc0b66 Auto merge of #79650 - the8472:fix-take, r=dtolnay
Fix incorrect io::Take's limit resulting from io::copy specialization

The specialization introduced in #75272 fails to update `io::Take` wrappers after performing the copy syscalls which bypass those wrappers. The buffer flushing before the copy does update them correctly, but the bytes copied after the initial flush weren't subtracted.

The fix is to subtract the bytes copied from each `Take` in the chain of wrappers, even when an error occurs during the syscall loop. To do so the `CopyResult` enum now has to carry the bytes copied so far in the error case.
2020-12-06 01:15:37 +00:00
bors
a5fbaed6c3 Auto merge of #79673 - ijackson:intoinnerintoinnererror, r=m-ou-se
Provide IntoInnerError::into_parts

Hi.  This is an updated version of the IntoInnerError bits of my previous portmanteau MR #78689.  Thanks to `@jyn514` and `@m-ou-se` for helpful comments there.

I have made this insta-stable since it seems like it will probably be uncontroversial, but that is definitely something that someone from the libs API team should be aware of and explicitly consider.

I included a tangentially-related commit providing documentation of the buffer full behaviiour of `&mut [u8] as Write`; the behaviour I am documenting is relied on by the doctest for `into_parts`.
2020-12-04 22:30:19 +00:00
Ian Jackson
b777552167 IntoInnerError: Provide into_error
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2020-12-04 18:43:02 +00:00
Ian Jackson
19c7619dcd IntoInnerError: Provide into_parts
In particular, IntoIneerError only currently provides .error() which
returns a reference, not an owned value.  This is not helpful and
means that a caller of BufWriter::into_inner cannot acquire an owned
io::Error which seems quite wrong.

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2020-12-04 18:43:02 +00:00
Ian Jackson
db5d697004 std: impl of Write for &mut [u8]: document the buffer full error
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
2020-12-04 18:38:44 +00:00
Tim Diekmann
9274b37d99 Rename AllocRef to Allocator and (de)alloc to (de)allocate 2020-12-04 14:47:15 +01:00
Dylan DPC
88f0c72dc6
Rollup merge of #79611 - poliorcetics:use-std-in-docs, r=jyn514
Use more std:: instead of core:: in docs for consistency

``@rustbot`` label T-doc

Some cleanup work to use `std::` instead of `core::` in docs as much as possible. This helps with terminology and consistency, especially for newcomers from other languages that have often heard of `std` to describe the standard library but not of `core`.

Edit: I also added more intra doc links when I saw the opportunity.
2020-12-04 03:30:27 +01:00
Dylan DPC
6b42900e40
Rollup merge of #79602 - jethrogb:sgx-fix-79038, r=Mark-Simulacrum
Fix SGX CI

Broken in #79038
2020-12-04 03:30:25 +01:00
Guillaume Gomez
50eb3a89f8 Only deny doc_keyword in std and set it as "allow" by default 2020-12-03 16:48:17 +01:00
Edd Barrett
87c1fdbcfb Make the kernel_copy tests more robust/concurrent.
These tests write to the same filenames in /tmp and in some cases these
files don't get cleaned up properly. This caused issues for us when
different users run the tests on the same system, e.g.:

```
---- sys::unix::kernel_copy::tests::bench_file_to_file_copy stdout ----
thread 'sys::unix::kernel_copy::tests::bench_file_to_file_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs:71:10
---- sys::unix::kernel_copy::tests::bench_file_to_socket_copy stdout ----
thread 'sys::unix::kernel_copy::tests::bench_file_to_socket_copy' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }', library/std/src/sys/unix/kernel_copy/tests.rs💯10
```

Use `std::sys_common::io__test::tmpdir()` to solve this.
2020-12-03 13:49:24 +00:00
The8472
a9b1381b8d fix copy specialization not updating Take wrappers 2020-12-03 00:02:01 +01:00
The8472
9b390e73db update test to check Take limits after copying 2020-12-02 23:34:59 +01:00
bors
af69066aa6 Auto merge of #69864 - LinkTed:master, r=Amanieu
unix: Extend UnixStream and UnixDatagram to send and receive file descriptors

Add the functions `recv_vectored_fds` and `send_vectored_fds` to `UnixDatagram` and `UnixStream`. With this functions `UnixDatagram` and `UnixStream` can send and receive file descriptors, by using `recvmsg` and `sendmsg` system call.
2020-12-02 17:36:29 +00:00
Alexis Bourget
4eb76fcc8e Use more std:: instead of core:: in docs for consistency, add more intra doc links 2020-12-02 00:41:53 +01:00
Guillaume Gomez
9e26fc60b1
Rollup merge of #79600 - nicokoch:kernel_copy_unixstream, r=m-ou-se
std::io: Use sendfile for UnixStream

`UnixStream` was forgotten in #75272 .

Benchmark yields the following results.
Before:
`running 1 test
test sys::unix::kernel_copy::tests::bench_file_to_uds_copy        ... bench:      54,399 ns/iter (+/- 6,817) = 2409 MB/s`

After:
`running 1 test
test sys::unix::kernel_copy::tests::bench_file_to_uds_copy        ... bench:      18,627 ns/iter (+/- 6,007) = 7036 MB/s`
2020-12-01 23:46:13 +01:00
Jethro Beekman
b787d723fc Fix SGX CI
Broken in #79038
2020-12-01 18:15:00 +01:00
Nicolas Koch
59874516fa Leverage kernel copy for UnixStream
UDS can be a sendfile destination, just like TCP sockets.
2020-12-01 14:45:36 +01:00
Nicolas Koch
eda4c63fdc Add benchmark for File to UnixStream copy 2020-12-01 14:44:40 +01:00
Mara Bos
2404409c6c
Rollup merge of #79444 - sasurau4:test/move-const-ip, r=matklad
Move const ip in ui test to unit test

Helps with #76268

r? ``@matklad``
2020-12-01 10:50:15 +00:00
Christiaan Dirkx
be554c4101 Make ui test that are run-pass and do not test the compiler itself library tests 2020-11-30 02:47:32 +01:00
oli
79fb037cc5 Remove now-unnecessary miri_static_root invocation 2020-11-28 17:13:47 +00:00
Jonas Schievink
13375864ed
Rollup merge of #79383 - abdnh:patch-1, r=shepmaster
Fix bold code formatting in keyword docs
2020-11-28 15:58:21 +01:00
Jonas Schievink
772b1a6d79
Rollup merge of #78086 - poliorcetics:as-placeholder, r=Mark-Simulacrum
Improve doc for 'as _'

Fix #78042.

`@rustbot` modify labels: A-coercions T-doc
2020-11-28 15:58:13 +01:00
LinkTed
8983752c12 Add comment for the previous android bug fix 2020-11-26 18:54:13 +01:00
Daiki Ihara
d4ee2f6dc5 Move const ip in ui test to unit test 2020-11-26 23:15:32 +09:00