Make some `usize`-typed masks definitions agnostic to the size of `usize`
Some masks where defined as
```rust
const NONASCII_MASK: usize = 0x80808080_80808080u64 as usize;
```
where it was assumed that `usize` is never wider than 64, which is currently true.
To make those constants valid in a hypothetical 128-bit target, these constants have been redefined in an `usize`-width-agnostic way
```rust
const NONASCII_MASK: usize = usize::from_ne_bytes([0x80; size_of::<usize>()]);
```
There are already some cases where Rust anticipates the possibility of supporting 128-bit targets, such as not implementing `From<usize>` for `u64`.
docs: add link from zip to unzip
The docs for `Iterator::unzip` explain that it is kind of an inverse operation to `Iterator::zip` and guide the reader to the `zip` docs, but the `zip` docs don't let the user know that they can undo the `zip` operation with `unzip`. This change modifies the docs to help the user find `unzip`.
Create (unstable) 2024 edition
[On Zulip](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Deprecating.20macro.20scoping.20shenanigans/near/272860652), there was a small aside regarding creating the 2024 edition now as opposed to later. There was a reasonable amount of support and no stated opposition.
This change creates the 2024 edition in the compiler and creates a prelude for the 2024 edition. There is no current difference between the 2021 and 2024 editions. Cargo and other tools will need to be updated separately, as it's not in the same repository. This change permits the vast majority of work towards the next edition to proceed _now_ instead of waiting until 2024.
For sanity purposes, I've merged the "hello" UI tests into a single file with multiple revisions. Otherwise we'd end up with a file per edition, despite them being essentially identical.
````@rustbot```` label +T-lang +S-waiting-on-review
Not sure on the relevant team, to be honest.
Stabilize `derive_default_enum`
This stabilizes `#![feature(derive_default_enum)]`, as proposed in [RFC 3107](https://github.com/rust-lang/rfcs/pull/3107) and tracked in #87517. In short, it permits you to `#[derive(Default)]` on `enum`s, indicating what the default should be by placing a `#[default]` attribute on the desired variant (which must be a unit variant in the interest of forward compatibility).
```````@rustbot``````` label +S-waiting-on-review +T-lang
Some masks where defined as
```rust
const NONASCII_MASK: usize = 0x80808080_80808080u64 as usize;
```
where it was assumed that `usize` is never wider than 64, which is currently true.
To make those constants valid in a hypothetical 128-bit target, these constants have been redefined in an `usize`-width-agnostic way
```rust
const NONASCII_MASK: usize = usize::from_ne_bytes([0x80; size_of::<usize>()]);
```
There are already some cases where Rust anticipates the possibility of supporting 128-bit targets, such as not implementing `From<usize>` for `u64`.
The docs for `Iterator::unzip` explain that it is kind of an inverse operation to `Iterator::zip` and guide the reader to the `zip` docs, but the `zip` docs don't let the user know that they can undo the `zip` operation with `unzip`. This change modifies the docs to help the user find `unzip`.
Implement tuples using recursion
Because it is c00l3r™, requires less repetition and can be used as a reference for external people.
This change is non-essential and I am not sure about potential performance impacts so feel free to close this PR if desired.
r? `@petrochenkov`
Faster parsing for lower numbers for radix up to 16 (cont.)
( Continuation of https://github.com/rust-lang/rust/pull/83371 )
With LingMan's change I think this is potentially ready.
Clarify str::from_utf8_unchecked's invariants
Specifically, make it clear that it is immediately UB to pass ill-formed UTF-8 into the function. The previous wording left space to interpret that the UB only occurred when calling another function, which "assumes that `&str`s are valid UTF-8."
This does not change whether str being UTF-8 is a safety or a validity invariant. (As per previous discussion, it is a safety invariant, not a validity invariant.) It just makes it clear that valid UTF-8 is a precondition of str::from_utf8_unchecked, and that emitting an Abstract Machine fault (e.g. UB or a sanitizer error) on invalid UTF-8 is a valid thing to do.
If user code wants to create an unsafe `&str` pointing to ill-formed UTF-8, it must be done via transmutes. Also, just, don't.
Zulip discussion: https://rust-lang.zulipchat.com/#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/str.3A.3Afrom_utf8_unchecked.20Safety.20requirement
Update binary_search example to instead redirect to partition_point
Inspired by discussion in the tracking issue for `Result::into_ok_or_err`: https://github.com/rust-lang/rust/issues/82223#issuecomment-1067098167
People are surprised by us not providing a `Result<T, T> -> T` conversion, and the main culprit for this confusion seems to be the `binary_search` API. We should instead redirect people to the equivalent API that implicitly does that `Result<T, T> -> T` conversion internally which should obviate the need for the `into_ok_or_err` function and give us time to work towards a more general solution that applies to all enums rather than just `Result` such as making or_patterns usable for situations like this via postfix `match`.
I choose to duplicate the example rather than simply moving it from `binary_search` to partition point because most of the confusion seems to arise when people are looking at `binary_search`. It makes sense to me to have the example presented immediately rather than requiring people to click through to even realize there is an example. If I had to put it in only one place I'd leave it in `binary_search` and remove it from `partition_point` but it seems pretty obviously relevant to `partition_point` so I figured the best option would be to duplicate it.
Specifically, make it clear that it is immediately UB to pass ill-formed UTF-8 into the function. The previous wording left space to interpret that the UB only occurred when calling another function, which "assumes that `&str`s are valid UTF-8."
This does not change whether str being UTF-8 is a safety or a validity invariant. (As per previous discussion, it is a safety invariant, not a validity invariant.) It just makes it clear that valid UTF-8 is a precondition of str::from_utf8_unchecked, and that emitting an Abstract Machine fault (e.g. UB or a sanitizer error) on invalid UTF-8 is a valid thing to do.
If user code wants to create an unsafe `&str` pointing to ill-formed UTF-8, it must be done via transmutes. Also, just, don't.
Avoid duplication of doc comments in `std::char` constants and functions
For those consts and functions, only the summary is kept and a reference to the `char` associated const/method is included.
Additionaly, re-exported functions have been converted to function definitions that call the previously re-exported function. This makes it easier to add a deprecated attribute to these functions in the future.
Use bitwise XOR in to_ascii_uppercase
This saves an instruction compared to the previous approach, which
was to unset the fifth bit with bitwise OR.
Comparison of generated assembly on x86: https://godbolt.org/z/GdfvdGs39
This can also affect autovectorization, saving SIMD instructions as well: https://godbolt.org/z/cnPcz75T9
Not sure if `u8::to_ascii_lowercase` should also be changed, since using bitwise OR for that function does not require an extra bitwise negate since the code is setting a bit rather than unsetting a bit. `char::to_ascii_uppercase` already uses XOR, so no change seems to be required there.
Left overs of #95761
These are just nits. Feel free to close this PR if all modifications are not worth merging.
* `#![feature(decl_macro)]` is not needed anymore in `rustc_expand`
* `tuple_impls` does not require `$Tuple:ident`. I guess it is there to enhance readability?
r? ```@petrochenkov```
Make non-power-of-two alignments a validity error in `Layout`
Inspired by the zulip conversation about how `Layout` should better enforce `size <= isize::MAX as usize`, this uses an N-variant enum on N-bit platforms to require at the validity level that the existing invariant of "must be a power of two" is upheld.
This was MIRI can catch it, and means there's a more-specific type for `Layout` to store than just `NonZeroUsize`.
It's left as `pub(crate)` here; a future PR could consider giving it a tracking issue for non-internal usage.
Inspired by the zulip conversation about how `Layout` should better enforce `size < isize::MAX as usize`, this uses an N-variant enum on N-bit platforms to require at the validity level that the existing invariant of "must be a power of two" is upheld.
This was MIRI can catch it, and means there's a more-specific type for `Layout` to store than just `NonZeroUsize`.
Correct safety reasoning in `str::make_ascii_{lower,upper}case()`
I don't understand why the previous comment was used (it was inserted in #66564), but it doesn't explain why these functions are safe, only why `str::as_bytes{_mut}()` are safe.
If someone thinks they make perfect sense, I'm fine with closing this PR.
Bump bootstrap compiler to 1.61.0 beta
This PR bumps the bootstrap compiler to the 1.61.0 beta. The first commit changes the stage0 compiler, the second commit applies the "mechanical" changes and the third and fourth commits apply changes explained in the relevant comments.
r? `@Mark-Simulacrum`
Mention `std::env::var` in `env!`
When searching for how to read an environment variable, I first encountered the `env!` macro. It would have been useful to me if the documentation had included a link to `std::env::var`, which is what I was actually looking for.
When searching for how to read an environment variable, I first encountered the `env!` macro. It would have been useful to me if the documentation had included a link to `std::env::var`, which is what I was actually looking for.
caution against ptr-to-int transmutes
I don't know how strong of a statement we want to make here, but I am very concerned that the current docs could be interpreted as saying that ptr-to-int transmutes are just as okay as transmuting `*mut T` into an `&mut T`.
Examples [like this](https://github.com/rust-lang/unsafe-code-guidelines/issues/286#issuecomment-1085144431) show that ptr-to-int transmutes are deeply suspicious -- they are either UB, or they don't round-trip properly, or we have to basically say that `transmute` will actively look for pointers and do all the things a ptr-to-int cast does (which includes a global side-effect of marking the pointed-to allocation as 'exposed').
Another alternative might be to simply not talk about them... but we *do* want people to use casts rather than transmutes for this.
Cc `@rust-lang/lang`
Update panic docs to make it clearer when to use panic vs Result
This is based on a question that came up in one of my [error handling office hours](https://twitter.com/yaahc_/status/1506376624509374467?s=20&t=Sp-cEjrx5kpMdNsAGPOo9w) meetings. I had a user who was fairly familiar with error type design, thiserror and anyhow, and rust in general, but who was still confused about when to use panics vs when to use Result and `Error`.
This will also be cross referenced in an error handling FAQ that I will be creating in the https://github.com/rust-lang/project-error-handling repo shortly.
explicitly distinguish pointer::addr and pointer::expose_addr
``@bgeron`` pointed out that the current docs promise that `ptr.addr()` and `ptr as usize` are equivalent. I don't think that is a promise we want to make. (Conceptually, `ptr as usize` might 'escape' the provenance to enable future `usize as ptr` casts, but `ptr.addr()` dertainly does not do that.)
So I propose we word the docs a bit more carefully here. ``@Gankra`` what do you think?
Mention implementers of unsatisfied trait
When encountering an unsatisfied trait bound, if there are no other
suggestions, mention all the types that *do* implement that trait:
```
error[E0277]: the trait bound `f32: Foo` is not satisfied
--> $DIR/impl_wf.rs:22:6
|
LL | impl Baz<f32> for f32 { }
| ^^^^^^^^ the trait `Foo` is not implemented for `f32`
|
= help: the trait `Foo` is implemented for `i32`
note: required by a bound in `Baz`
--> $DIR/impl_wf.rs:18:31
|
LL | trait Baz<U: ?Sized> where U: Foo { }
| ^^^ required by this bound in `Baz`
```
```
error[E0277]: the trait bound `u32: Foo` is not satisfied
--> $DIR/associated-types-path-2.rs:29:5
|
LL | f1(2u32, 4u32);
| ^^ the trait `Foo` is not implemented for `u32`
|
= help: the trait `Foo` is implemented for `i32`
note: required by a bound in `f1`
--> $DIR/associated-types-path-2.rs:13:14
|
LL | pub fn f1<T: Foo>(a: T, x: T::A) {}
| ^^^ required by this bound in `f1`
```
Suggest dereferencing in more cases.
Fix#87437, fix#90970.
When encountering an unsatisfied trait bound, if there are no other
suggestions, mention all the types that *do* implement that trait:
```
error[E0277]: the trait bound `f32: Foo` is not satisfied
--> $DIR/impl_wf.rs:22:6
|
LL | impl Baz<f32> for f32 { }
| ^^^^^^^^ the trait `Foo` is not implemented for `f32`
|
= help: the following other types implement trait `Foo`:
Option<T>
i32
str
note: required by a bound in `Baz`
--> $DIR/impl_wf.rs:18:31
|
LL | trait Baz<U: ?Sized> where U: Foo { }
| ^^^ required by this bound in `Baz`
```
Mention implementers of traits in `ImplObligation`s.
Do not mention other `impl`s for closures, ranges and `?`.
Add SyncUnsafeCell.
This adds `SyncUnsafeCell`, which is just `UnsafeCell` except it implements `Sync`.
This was first proposed under the name `RacyUnsafeCell` here: https://github.com/rust-lang/rust/issues/53639#issuecomment-415515748 and here: https://github.com/rust-lang/rust/issues/53639#issuecomment-432741659 and here: https://github.com/rust-lang/rust/issues/53639#issuecomment-888435728
It allows you to create an UnsafeCell that is Sync without having to wrap it in a struct first (and then implement Sync for that struct).
E.g. `static X: SyncUnsafeCell<i32>`. Using a regular `UnsafeCell` as `static` is not possible, because it isn't `Sync`. We have a language workaround for it called `static mut`, but it's nice to be able to use the proper type for such unsafety instead.
It also makes implementing synchronization primitives based on unsafe cells slightly less verbose, because by using `SyncUnsafeCell` for `UnsafeCell`s that are shared between threads, you don't need a separate `impl<..> Sync for ..`. Using this type also clearly documents that the cell is expected to be accessed from multiple threads.
Mark Location::caller() as #[inline]
This function gets compiled to a single register move as it actually gets it's return value passed in as argument.
Fix &mut invalidation in ptr::swap doctest
Under Stacked Borrows with raw pointer tagging, the previous code was UB
because the code which creates the the second pointer borrows the array
through a tag in the borrow stacks below the Unique tag that our first
pointer is based on, thus invalidating the first pointer.
This is not definitely a bug and may never be real UB, but I desperately
want people to write code that conforms to SB with raw pointer tagging
so that I can write good diagnostics. The alternative aliasing models
aren't possible to diagnose well due to state space explosion.
Therefore, it would be super cool if the standard library nudged people
towards writing code that is valid with respect to SB with raw pointer
tagging.
The diagnostics that I want to write are implemented in a branch of Miri and the one for this case is below:
```
error: Undefined Behavior: attempting a read access using <2170> at alloc1068[0x0], but that tag does not exist in the borrow stack for this location
--> /home/ben/rust/library/core/src/intrinsics.rs:2103:14
|
2103 | unsafe { copy_nonoverlapping(src, dst, count) }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| attempting a read access using <2170> at alloc1068[0x0], but that tag does not exist in the borrow stack for this location
| this error occurs as part of an access at alloc1068[0x0..0x8]
|
= help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
= help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <2170> was created due to a retag at offsets [0x0..0x10]
--> ../libcore/src/ptr/mod.rs:640:9
|
8 | let x = array[0..].as_mut_ptr() as *mut [u32; 2]; // this is `array[0..2]`
| ^^^^^^^^^^^^^^^^^^^^^^^
help: <2170> was later invalidated due to a retag at offsets [0x0..0x10]
--> ../libcore/src/ptr/mod.rs:641:9
|
9 | let y = array[2..].as_mut_ptr() as *mut [u32; 2]; // this is `array[2..4]`
| ^^^^^
= note: inside `std::intrinsics::copy_nonoverlapping::<[u32; 2]>` at /home/ben/rust/library/core/src/intrinsics.rs:2103:14
= note: inside `std::ptr::swap::<[u32; 2]>` at /home/ben/rust/library/core/src/ptr/mod.rs:685:9
note: inside `main::_doctest_main____libcore_src_ptr_mod_rs_635_0` at ../libcore/src/ptr/mod.rs:12:5
--> ../libcore/src/ptr/mod.rs:644:5
|
12 | ptr::swap(x, y);
| ^^^^^^^^^^^^^^^
note: inside `main` at ../libcore/src/ptr/mod.rs:15:3
--> ../libcore/src/ptr/mod.rs:647:3
|
15 | } _doctest_main____libcore_src_ptr_mod_rs_635_0() }
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace
error: aborting due to previous error
```
Under Stacked Borrows with raw pointer tagging, the previous code was UB
because the code which creates the the second pointer borrows the array
through a tag in the borrow stacks below the Unique tag that our first
pointer is based on, thus invalidating the first pointer.
This is not definitely a bug and may never be real UB, but I desperately
want people to write code that conforms to SB with raw pointer tagging
so that I can write good diagnostics. The alternative aliasing models
aren't possible to diagnose well due to state space explosion.
Therefore, it would be super cool if the standard library nudged people
towards writing code that is valid with respect to SB with raw pointer
tagging.
Improve doc example of DerefMut
It is more illustrative, after using `*x` to modify the field, to show
in the assertion that the field has indeed been modified.
Add debug assertions to some unsafe functions
As suggested by https://github.com/rust-lang/rust/issues/51713
~~Some similar code calls `abort()` instead of `panic!()` but aborting doesn't work in a `const fn`, and the intrinsic for doing dispatch based on whether execution is in a const is unstable.~~
This picked up some invalid uses of `get_unchecked` in the compiler, and fixes them.
I can confirm that they do in fact pick up invalid uses of `get_unchecked` in the wild, though the user experience is less-than-awesome:
```
Running unittests (target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50)
running 6 tests
error: test failed, to rerun pass '--lib'
Caused by:
process didn't exit successfully: `/home/ben/rle-decode-helper/target/x86_64-unknown-linux-gnu/debug/deps/rle_decode_fast-04b7918da2001b50` (signal: 4, SIGILL: illegal instruction)
```
~~As best I can tell these changes produce a 6% regression in the runtime of `./x.py test` when `[rust] debug = true` is set.~~
Latest commit (6894d559bd) brings the additional overhead from this PR down to 0.5%, while also adding a few more assertions. I think this actually covers all the places in `core` that it is reasonable to check for safety requirements at runtime.
Thoughts?
Implement provenance preserving methods on NonNull
### Description
Add the `addr`, `with_addr`, `map_addr` methods to the `NonNull` type, and map the address type to `NonZeroUsize`.
### Motivation
The `NonNull` type is useful for implementing pointer types which have the 0-niche. It is currently possible to implement these provenance preserving functions by calling `NonNull::as_ptr` and `new_unchecked`. The adding these methods makes it more ergonomic.
### Testing
Added a unit test of a non-null tagged pointer type. This is based on some real code I have elsewhere, that currently routes the pointer through a `NonZeroUsize` and back out to produce a usable pointer. I wanted to produce an ideal version of the same tagged pointer struct that preserved pointer provenance.
### Related
Extension of APIs proposed in #95228 . I can also split this out into a separate tracking issue if that is better (though I may need some pointers on how to do that).
Handle rustc_const_stable attribute in library feature collector
The library feature collector in [compiler/rustc_passes/src/lib_features.rs](551b4fa395/compiler/rustc_passes/src/lib_features.rs) has only been looking at `#[stable(…)]`, `#[unstable(…)]`, and `#[rustc_const_unstable(…)]` attributes, while ignoring `#[rustc_const_stable(…)]`. The consequences of this were:
- When any const feature got stabilized (changing one or more `rustc_const_unstable` to `rustc_const_stable`), users who had previously enabled that unstable feature using `#![feature(…)]` would get told "unknown feature", rather than rustc's nicer "the feature … has been stable since … and no longer requires an attribute to enable".
This can be seen in the way that https://github.com/rust-lang/rust/pull/93957#issuecomment-1079794660 failed after rebase:
```console
error[E0635]: unknown feature `const_ptr_offset`
--> $DIR/offset_from_ub.rs:1:35
|
LL | #![feature(const_ptr_offset_from, const_ptr_offset)]
| ^^^^^^^^^^^^^^^^
```
- We weren't enforcing that a particular feature is either stable everywhere or unstable everywhere, and that a feature that has been stabilized has the same stabilization version everywhere, both of which we enforce for the other stability attributes.
This PR updates the library feature collector to handle `rustc_const_stable`, and fixes places in the standard library and test suite where `rustc_const_stable` was being used in a way that does not meet the rules for a stability attribute.
For those consts and functions, only the summary is kept and a reference to the `char` associated const/method is included.
Additionaly, re-exported functions have been converted to function definitions that call the previously re-exported function. This makes it easier to add a deprecated attribute to these functions in the future.
add notes about alignment-altering reallocations to Allocator docs
As I said in https://github.com/rust-lang/wg-allocators/issues/97, the fact that calls to `grow`, `grow_zeroed`, and `shrink` may request altered alignments is surprising and may be a pitfall for implementors of `Allocator` if it's left implicit. This pull request adds a note to the "Safety" section of each function's docs making it explicit.
skip slow int_log tests in Miri
Iterating over i16::MAX many things takes a long time in Miri, let's not do that.
I added https://github.com/rust-lang/miri/pull/2044 on the Miri side to still give us some test coverage.
ptr_metadata test: avoid ptr-to-int transmutes
Pointers can have provenance, integers don't, so transmuting pointers to integers creates "non-standard" values and it is unclear how well those can be supported (https://github.com/rust-lang/unsafe-code-guidelines/issues/286).
So for this test let's take the safer option and use a pointer type instead. That also makes Miri happy. :)
**Description**
Add the `addr`, `with_addr, `map_addr` methods to the `NonNull` type,
and map the address type to `NonZeroUsize`.
**Motiviation**
The `NonNull` type is useful for implementing pointer types which have
the 0-niche. It is currently possible to implement these provenance
preserving functions by calling `NonNull::as_ptr` and `new_unchecked`.
The addition of these methods simply make it more ergonomic to use.
**Testing**
Added a unit test of a nonnull tagged pointer type. This is based on
some real code I have elsewhere, that currently routes the pointer
through a `NonZeroUsize` and back out to produce a usable pointer.
