OpenE2K
/
rust
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Auto merge of #81238 - RalfJung:copy-intrinsics, r=m-ou-se 

directly expose copy and copy_nonoverlapping intrinsics

This effectively un-does https://github.com/rust-lang/rust/pull/57997. That should help with `ptr::read` codegen in debug builds (and any other of these low-level functions that bottoms out at `copy`/`copy_nonoverlapping`), where the wrapper function will not get inlined. See the discussion in https://github.com/rust-lang/rust/pull/80290 and https://github.com/rust-lang/rust/issues/81163.

Cc `@bjorn3` `@therealprof`
This commit is contained in:
bors 2021-02-13 20:30:07 +00:00
parent 3c10a880ec 1a806352e4
commit 8e54a21139
4 changed files with 177 additions and 220 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

337
library/core/src/intrinsics.rs
View File

@ -1730,6 +1730,157 @@ extern "rust-intrinsic" {
/// Allocate at compile time. Should not be called at runtime.
#[rustc_const_unstable(feature = "const_heap", issue = "79597")]
pub fn const_allocate(size: usize, align: usize) -> *mut u8;
/// Copies `count * size_of::<T>()` bytes from `src` to `dst`. The source
/// and destination must *not* overlap.
///
/// For regions of memory which might overlap, use [`copy`] instead.
///
/// `copy_nonoverlapping` is semantically equivalent to C's [`memcpy`], but
/// with the argument order swapped.
///
/// [`memcpy`]: https://en.cppreference.com/w/c/string/byte/memcpy
///
/// # Safety
///
/// Behavior is undefined if any of the following conditions are violated:
///
/// * `src` must be [valid] for reads of `count * size_of::<T>()` bytes.
///
/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes.
///
/// * Both `src` and `dst` must be properly aligned.
///
/// * The region of memory beginning at `src` with a size of `count *
/// size_of::<T>()` bytes must *not* overlap with the region of memory
/// beginning at `dst` with the same size.
///
/// Like [`read`], `copy_nonoverlapping` creates a bitwise copy of `T`, regardless of
/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using *both* the values
/// in the region beginning at `*src` and the region beginning at `*dst` can
/// [violate memory safety][read-ownership].
///
/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
/// `0`, the pointers must be non-NULL and properly aligned.
///
/// [`read`]: crate::ptr::read
/// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value
/// [valid]: crate::ptr#safety
///
/// # Examples
///
/// Manually implement [`Vec::append`]:
///
/// ```
/// use std::ptr;
///
/// /// Moves all the elements of `src` into `dst`, leaving `src` empty.
/// fn append<T>(dst: &mut Vec<T>, src: &mut Vec<T>) {
/// let src_len = src.len();
/// let dst_len = dst.len();
///
/// // Ensure that `dst` has enough capacity to hold all of `src`.
/// dst.reserve(src_len);
///
/// unsafe {
/// // The call to offset is always safe because `Vec` will never
/// // allocate more than `isize::MAX` bytes.
/// let dst_ptr = dst.as_mut_ptr().offset(dst_len as isize);
/// let src_ptr = src.as_ptr();
///
/// // Truncate `src` without dropping its contents. We do this first,
/// // to avoid problems in case something further down panics.
/// src.set_len(0);
///
/// // The two regions cannot overlap because mutable references do
/// // not alias, and two different vectors cannot own the same
/// // memory.
/// ptr::copy_nonoverlapping(src_ptr, dst_ptr, src_len);
///
/// // Notify `dst` that it now holds the contents of `src`.
/// dst.set_len(dst_len + src_len);
/// }
/// }
///
/// let mut a = vec!['r'];
/// let mut b = vec!['u', 's', 't'];
///
/// append(&mut a, &mut b);
///
/// assert_eq!(a, &['r', 'u', 's', 't']);
/// assert!(b.is_empty());
/// ```
///
/// [`Vec::append`]: ../../std/vec/struct.Vec.html#method.append
#[doc(alias = "memcpy")]
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")]
pub fn copy_nonoverlapping<T>(src: *const T, dst: *mut T, count: usize);
/// Copies `count * size_of::<T>()` bytes from `src` to `dst`. The source
/// and destination may overlap.
///
/// If the source and destination will *never* overlap,
/// [`copy_nonoverlapping`] can be used instead.
///
/// `copy` is semantically equivalent to C's [`memmove`], but with the argument
/// order swapped. Copying takes place as if the bytes were copied from `src`
/// to a temporary array and then copied from the array to `dst`.
///
/// [`memmove`]: https://en.cppreference.com/w/c/string/byte/memmove
///
/// # Safety
///
/// Behavior is undefined if any of the following conditions are violated:
///
/// * `src` must be [valid] for reads of `count * size_of::<T>()` bytes.
///
/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes.
///
/// * Both `src` and `dst` must be properly aligned.
///
/// Like [`read`], `copy` creates a bitwise copy of `T`, regardless of
/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using both the values
/// in the region beginning at `*src` and the region beginning at `*dst` can
/// [violate memory safety][read-ownership].
///
/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
/// `0`, the pointers must be non-NULL and properly aligned.
///
/// [`read`]: crate::ptr::read
/// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value
/// [valid]: crate::ptr#safety
///
/// # Examples
///
/// Efficiently create a Rust vector from an unsafe buffer:
///
/// ```
/// use std::ptr;
///
/// /// # Safety
/// ///
/// /// * `ptr` must be correctly aligned for its type and non-zero.
/// /// * `ptr` must be valid for reads of `elts` contiguous elements of type `T`.
/// /// * Those elements must not be used after calling this function unless `T: Copy`.
/// # #[allow(dead_code)]
/// unsafe fn from_buf_raw<T>(ptr: *const T, elts: usize) -> Vec<T> {
/// let mut dst = Vec::with_capacity(elts);
///
/// // SAFETY: Our precondition ensures the source is aligned and valid,
/// // and `Vec::with_capacity` ensures that we have usable space to write them.
/// ptr::copy(ptr, dst.as_mut_ptr(), elts);
///
/// // SAFETY: We created it with this much capacity earlier,
/// // and the previous `copy` has initialized these elements.
/// dst.set_len(elts);
/// dst
/// }
/// ```
#[doc(alias = "memmove")]
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")]
pub fn copy<T>(src: *const T, dst: *mut T, count: usize);
}
// Some functions are defined here because they accidentally got made
@ -1755,192 +1906,6 @@ pub(crate) fn is_nonoverlapping<T>(src: *const T, dst: *const T, count: usize) -
diff >= size
}
/// Copies `count * size_of::<T>()` bytes from `src` to `dst`. The source
/// and destination must *not* overlap.
///
/// For regions of memory which might overlap, use [`copy`] instead.
///
/// `copy_nonoverlapping` is semantically equivalent to C's [`memcpy`], but
/// with the argument order swapped.
///
/// [`memcpy`]: https://en.cppreference.com/w/c/string/byte/memcpy
///
/// # Safety
///
/// Behavior is undefined if any of the following conditions are violated:
///
/// * `src` must be [valid] for reads of `count * size_of::<T>()` bytes.
///
/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes.
///
/// * Both `src` and `dst` must be properly aligned.
///
/// * The region of memory beginning at `src` with a size of `count *
/// size_of::<T>()` bytes must *not* overlap with the region of memory
/// beginning at `dst` with the same size.
///
/// Like [`read`], `copy_nonoverlapping` creates a bitwise copy of `T`, regardless of
/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using *both* the values
/// in the region beginning at `*src` and the region beginning at `*dst` can
/// [violate memory safety][read-ownership].
///
/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
/// `0`, the pointers must be non-NULL and properly aligned.
///
/// [`read`]: crate::ptr::read
/// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value
/// [valid]: crate::ptr#safety
///
/// # Examples
///
/// Manually implement [`Vec::append`]:
///
/// ```
/// use std::ptr;
///
/// /// Moves all the elements of `src` into `dst`, leaving `src` empty.
/// fn append<T>(dst: &mut Vec<T>, src: &mut Vec<T>) {
/// let src_len = src.len();
/// let dst_len = dst.len();
///
/// // Ensure that `dst` has enough capacity to hold all of `src`.
/// dst.reserve(src_len);
///
/// unsafe {
/// // The call to offset is always safe because `Vec` will never
/// // allocate more than `isize::MAX` bytes.
/// let dst_ptr = dst.as_mut_ptr().offset(dst_len as isize);
/// let src_ptr = src.as_ptr();
///
/// // Truncate `src` without dropping its contents. We do this first,
/// // to avoid problems in case something further down panics.
/// src.set_len(0);
///
/// // The two regions cannot overlap because mutable references do
/// // not alias, and two different vectors cannot own the same
/// // memory.
/// ptr::copy_nonoverlapping(src_ptr, dst_ptr, src_len);
///
/// // Notify `dst` that it now holds the contents of `src`.
/// dst.set_len(dst_len + src_len);
/// }
/// }
///
/// let mut a = vec!['r'];
/// let mut b = vec!['u', 's', 't'];
///
/// append(&mut a, &mut b);
///
/// assert_eq!(a, &['r', 'u', 's', 't']);
/// assert!(b.is_empty());
/// ```
///
/// [`Vec::append`]: ../../std/vec/struct.Vec.html#method.append
#[doc(alias = "memcpy")]
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")]
#[inline]
pub const unsafe fn copy_nonoverlapping<T>(src: *const T, dst: *mut T, count: usize) {
extern "rust-intrinsic" {
#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")]
fn copy_nonoverlapping<T>(src: *const T, dst: *mut T, count: usize);
}
// FIXME: Perform these checks only at run time
/*if cfg!(debug_assertions)
&& !(is_aligned_and_not_null(src)
&& is_aligned_and_not_null(dst)
&& is_nonoverlapping(src, dst, count))
{
// Not panicking to keep codegen impact smaller.
abort();
}*/
// SAFETY: the safety contract for `copy_nonoverlapping` must be
// upheld by the caller.
unsafe { copy_nonoverlapping(src, dst, count) }
}
/// Copies `count * size_of::<T>()` bytes from `src` to `dst`. The source
/// and destination may overlap.
///
/// If the source and destination will *never* overlap,
/// [`copy_nonoverlapping`] can be used instead.
///
/// `copy` is semantically equivalent to C's [`memmove`], but with the argument
/// order swapped. Copying takes place as if the bytes were copied from `src`
/// to a temporary array and then copied from the array to `dst`.
///
/// [`memmove`]: https://en.cppreference.com/w/c/string/byte/memmove
///
/// # Safety
///
/// Behavior is undefined if any of the following conditions are violated:
///
/// * `src` must be [valid] for reads of `count * size_of::<T>()` bytes.
///
/// * `dst` must be [valid] for writes of `count * size_of::<T>()` bytes.
///
/// * Both `src` and `dst` must be properly aligned.
///
/// Like [`read`], `copy` creates a bitwise copy of `T`, regardless of
/// whether `T` is [`Copy`]. If `T` is not [`Copy`], using both the values
/// in the region beginning at `*src` and the region beginning at `*dst` can
/// [violate memory safety][read-ownership].
///
/// Note that even if the effectively copied size (`count * size_of::<T>()`) is
/// `0`, the pointers must be non-NULL and properly aligned.
///
/// [`read`]: crate::ptr::read
/// [read-ownership]: crate::ptr::read#ownership-of-the-returned-value
/// [valid]: crate::ptr#safety
///
/// # Examples
///
/// Efficiently create a Rust vector from an unsafe buffer:
///
/// ```
/// use std::ptr;
///
/// /// # Safety
/// ///
/// /// * `ptr` must be correctly aligned for its type and non-zero.
/// /// * `ptr` must be valid for reads of `elts` contiguous elements of type `T`.
/// /// * Those elements must not be used after calling this function unless `T: Copy`.
/// # #[allow(dead_code)]
/// unsafe fn from_buf_raw<T>(ptr: *const T, elts: usize) -> Vec<T> {
/// let mut dst = Vec::with_capacity(elts);
///
/// // SAFETY: Our precondition ensures the source is aligned and valid,
/// // and `Vec::with_capacity` ensures that we have usable space to write them.
/// ptr::copy(ptr, dst.as_mut_ptr(), elts);
///
/// // SAFETY: We created it with this much capacity earlier,
/// // and the previous `copy` has initialized these elements.
/// dst.set_len(elts);
/// dst
/// }
/// ```
#[doc(alias = "memmove")]
#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")]
#[inline]
pub const unsafe fn copy<T>(src: *const T, dst: *mut T, count: usize) {
extern "rust-intrinsic" {
#[rustc_const_unstable(feature = "const_intrinsic_copy", issue = "80697")]
fn copy<T>(src: *const T, dst: *mut T, count: usize);
}
// FIXME: Perform these checks only at run time
/*if cfg!(debug_assertions) && !(is_aligned_and_not_null(src) && is_aligned_and_not_null(dst)) {
// Not panicking to keep codegen impact smaller.
abort();
}*/
// SAFETY: the safety contract for `copy` must be upheld by the caller.
unsafe { copy(src, dst, count) }
}
/// Sets `count * size_of::<T>()` bytes of memory starting at `dst` to
/// `val`.
///

7
library/core/src/ptr/mod.rs
View File

@ -881,17 +881,12 @@ pub const unsafe fn read_unaligned<T>(src: *const T) -> T {
#[inline]
#[stable(feature = "rust1", since = "1.0.0")]
pub unsafe fn write<T>(dst: *mut T, src: T) {
// We are calling the intrinsics directly to avoid function calls in the generated code
// as `intrinsics::copy_nonoverlapping` is a wrapper function.
extern "rust-intrinsic" {
fn copy_nonoverlapping<T>(src: *const T, dst: *mut T, count: usize);
}
// SAFETY: the caller must guarantee that `dst` is valid for writes.
// `dst` cannot overlap `src` because the caller has mutable access
// to `dst` while `src` is owned by this function.
unsafe {
copy_nonoverlapping(&src as *const T, dst, 1);
// We are calling the intrinsic directly to avoid function calls in the generated code.
intrinsics::forget(src);
}
}

49
src/test/ui/const-ptr/out_of_bounds_read.stderr
View File

@ -1,13 +1,12 @@
error: any use of this value will cause an error
--> $SRC_DIR/core/src/intrinsics.rs:LL:COL
--> $SRC_DIR/core/src/ptr/mod.rs:LL:COL
|
LL | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| memory access failed: pointer must be in-bounds at offset 8, but is outside bounds of alloc6 which has size 4
| inside `copy_nonoverlapping::<u32>` at $SRC_DIR/core/src/intrinsics.rs:LL:COL
| inside `std::ptr::read::<u32>` at $SRC_DIR/core/src/ptr/mod.rs:LL:COL
| inside `_READ` at $DIR/out_of_bounds_read.rs:13:33
LL | copy_nonoverlapping(src, tmp.as_mut_ptr(), 1);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| memory access failed: pointer must be in-bounds at offset 8, but is outside bounds of alloc6 which has size 4
| inside `std::ptr::read::<u32>` at $SRC_DIR/core/src/ptr/mod.rs:LL:COL
| inside `_READ` at $DIR/out_of_bounds_read.rs:13:33
|
::: $DIR/out_of_bounds_read.rs:13:5
|
@ -19,16 +18,15 @@ LL | const _READ: u32 = unsafe { ptr::read(PAST_END_PTR) };
= note: for more information, see issue #71800 <https://github.com/rust-lang/rust/issues/71800>
error: any use of this value will cause an error
--> $SRC_DIR/core/src/intrinsics.rs:LL:COL
--> $SRC_DIR/core/src/ptr/mod.rs:LL:COL
|
LL | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| memory access failed: pointer must be in-bounds at offset 8, but is outside bounds of alloc6 which has size 4
| inside `copy_nonoverlapping::<u32>` at $SRC_DIR/core/src/intrinsics.rs:LL:COL
| inside `std::ptr::read::<u32>` at $SRC_DIR/core/src/ptr/mod.rs:LL:COL
| inside `ptr::const_ptr::<impl *const u32>::read` at $SRC_DIR/core/src/ptr/const_ptr.rs:LL:COL
| inside `_CONST_READ` at $DIR/out_of_bounds_read.rs:14:39
LL | copy_nonoverlapping(src, tmp.as_mut_ptr(), 1);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| memory access failed: pointer must be in-bounds at offset 8, but is outside bounds of alloc6 which has size 4
| inside `std::ptr::read::<u32>` at $SRC_DIR/core/src/ptr/mod.rs:LL:COL
| inside `ptr::const_ptr::<impl *const u32>::read` at $SRC_DIR/core/src/ptr/const_ptr.rs:LL:COL
| inside `_CONST_READ` at $DIR/out_of_bounds_read.rs:14:39
|
::: $DIR/out_of_bounds_read.rs:14:5
|
@ -39,16 +37,15 @@ LL | const _CONST_READ: u32 = unsafe { PAST_END_PTR.read() };
= note: for more information, see issue #71800 <https://github.com/rust-lang/rust/issues/71800>
error: any use of this value will cause an error
--> $SRC_DIR/core/src/intrinsics.rs:LL:COL
--> $SRC_DIR/core/src/ptr/mod.rs:LL:COL
|
LL | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| memory access failed: pointer must be in-bounds at offset 8, but is outside bounds of alloc6 which has size 4
| inside `copy_nonoverlapping::<u32>` at $SRC_DIR/core/src/intrinsics.rs:LL:COL
| inside `std::ptr::read::<u32>` at $SRC_DIR/core/src/ptr/mod.rs:LL:COL
| inside `ptr::mut_ptr::<impl *mut u32>::read` at $SRC_DIR/core/src/ptr/mut_ptr.rs:LL:COL
| inside `_MUT_READ` at $DIR/out_of_bounds_read.rs:15:37
LL | copy_nonoverlapping(src, tmp.as_mut_ptr(), 1);
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| memory access failed: pointer must be in-bounds at offset 8, but is outside bounds of alloc6 which has size 4
| inside `std::ptr::read::<u32>` at $SRC_DIR/core/src/ptr/mod.rs:LL:COL
| inside `ptr::mut_ptr::<impl *mut u32>::read` at $SRC_DIR/core/src/ptr/mut_ptr.rs:LL:COL
| inside `_MUT_READ` at $DIR/out_of_bounds_read.rs:15:37
|
::: $DIR/out_of_bounds_read.rs:15:5
|

4
src/tools/clippy/clippy_lints/src/utils/paths.rs
View File

@ -20,8 +20,8 @@ pub const CLONE_TRAIT: [&str; 3] = ["core", "clone", "Clone"];
pub const CLONE_TRAIT_METHOD: [&str; 4] = ["core", "clone", "Clone", "clone"];
pub const CMP_MAX: [&str; 3] = ["core", "cmp", "max"];
pub const CMP_MIN: [&str; 3] = ["core", "cmp", "min"];
pub const COPY: [&str; 3] = ["core", "intrinsics", "copy_nonoverlapping"];
pub const COPY_NONOVERLAPPING: [&str; 3] = ["core", "intrinsics", "copy"];
pub const COPY: [&str; 4] = ["core", "intrinsics", "", "copy_nonoverlapping"];
pub const COPY_NONOVERLAPPING: [&str; 4] = ["core", "intrinsics", "", "copy"];
pub const COW: [&str; 3] = ["alloc", "borrow", "Cow"];
pub const CSTRING_AS_C_STR: [&str; 5] = ["std", "ffi", "c_str", "CString", "as_c_str"];
pub const DEFAULT_TRAIT: [&str; 3] = ["core", "default", "Default"];