Allow writing of incomplete UTF-8 sequences to the Windows console via stdout/stderr
# Problem
Writes of just an incomplete UTF-8 byte sequence (e.g. `b"\xC3"` or `b"\xF0\x9F"`) to stdout/stderr with a Windows console attached error with `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` even though further writes could complete the codepoint. This is currently a rare occurence since the [linewritershim](2c56ea38b0/library/std/src/io/buffered/linewritershim.rs) implementation flushes complete lines immediately and buffers up to 1024 bytes for incomplete lines. It can still happen as described in #83258.
The problem will become more pronounced once the developer can switch stdout/stderr from line-buffered to block-buffered or immediate when the changes in the "Switchable buffering for Stdout" pull request (#78515) get merged.
# Patch description
If there is at least one valid UTF-8 codepoint all valid UTF-8 is passed through to the extracted `write_valid_utf8_to_console()` fn. The new code only comes into play if `write()` is being passed a short byte slice comprising an incomplete UTF-8 codepoint. In this case up to three bytes are buffered in the `IncompleteUtf8` struct associated with `Stdout` / `Stderr`. The bytes are accepted one at a time. As soon as an error can be detected `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` is returned. Once a complete UTF-8 codepoint is received it is passed to the `write_valid_utf8_to_console()` and the buffer length is set to zero.
Calling `flush()` will neither error nor write anything if an incomplete codepoint is present in the buffer.
# Tests
Currently there are no Windows-specific tests for console writing code at all. Writing (regression) tests for this problem is a bit challenging since unit tests and UI tests don't run in a console and suddenly popping up another console window might be surprising to developers running the testsuite and it might not work at all in CI builds. To just test the new functionality in unit tests the code would need to be refactored. Some guidance on how to proceed would be appreciated.
# Public API changes
* `std::str::verifications::utf8_char_width()` would be exposed as `std::str::utf8_char_width()` behind the "str_internals" feature gate.
# Related issues
* Fixes#83258.
* PR #78515 will exacerbate the problem.
# Open questions
* Add tests?
* Squash into one commit with better commit message?
Stabilize `UnsafeCell::raw_get()`
This PR stabilizes the associated function `UnsafeCell::raw_get()`. The FCP has [already completed](https://github.com/rust-lang/rust/issues/66358#issuecomment-899095068). While there was some discussion about the naming after the close of the FCP, it looks like people have agreed on this name. Still, it would probably be best if a `libs-api` member had a look at this and stated whether more discussion is needed.
While I was at it, I added some tests for `UnsafeCell`, because there were barely any.
Closes#66358.
Add carrying_add, borrowing_sub, widening_mul, carrying_mul methods to integers
This comes in part from my own attempts to make (crude) big integer implementations, and also due to the stalled discussion in [RFC 2417](https://github.com/rust-lang/rfcs/pull/2417). My understanding is that changes like these are best offered directly as code and then an RFC can be opened if there needs to be more discussion before stabilisation. Since all of these methods are unstable from the start, I figured I might as well offer them now.
I tried looking into intrinsics, messed around with a few different implementations, and ultimately concluded that these are "good enough" implementations for now to at least put up some code and maybe start bikeshedding on a proper API for these.
For the `carrying_add` and `borrowing_sub`, I tried looking into potential architecture-specific code and realised that even using the LLVM intrinsics for `addcarry` and `subborrow` on x86 specifically, I was getting exactly the same assembly as the naive implementation using `overflowing_add` and `overflowing_sub`, although the LLVM IR did differ because of the architecture-specific code. Longer-term I think that they would be best suited to specific intrinsics as that would make optimisations easier (instructions like add-carry tend to use implicit flags, and thus can only be optimised if they're done one-after-another, and thus it would make the most sense to have compact intrinsics that can be merged together easily).
For `widening_mul` and `carrying_mul`, for now at least, I simply cast to the larger type and perform arithmetic that way, since we currently have no intrinsic that would work better for 128-bit integers. In the future, I also think that some form of intrinsic would work best to cover that case, but for now at least, I think that they're "good enough" for now.
The main reasoning for offering these directly to the standard library even though they're relatively niche optimisations is to help ensure that the code generated for them is optimal. Plus, these operations alone aren't enough to create big integer implementations, although they could help simplify the code required to do so and make it a bit more accessible for the average implementor.
That said, I 100% understand if any or all of these methods are not desired simply because of how niche they are. Up to you. 🤷🏻
fix: move test that require mut to another
Adding TODOs for Option::take and Option::copied
TODO to FIXME + moving const stability under normal
Moving const stability attr under normal stab attr
move more rustc stability attributes
Add Saturating type (based on Wrapping type)
Tracking #87920
### Unresolved Questions
<!--
Include any open questions that need to be answered before the feature can be
stabilised.
-->
- [x] ~`impl Div for Saturating<T>` falls back on inner integer division - which seems alright?~
- [x] add `saturating_div`? (to respect division by `-1`)
- [x] There is no `::saturating_shl` and `::saturating_shr`. (How to) implement `Shl`, `ShlAssign`, `Shr` and `ShrAssign`?
- [naively](3f7d2ce28f)
- [x] ~`saturating_neg` is only implemented on [signed integer types](https://doc.rust-lang.org/std/?search=saturating_n)~
- [x] Is the implementation copied over from the `Wrapping`-type correct for `Saturating`?
- [x] `Saturating::rotate_left`
- [x] `Saturating::rotate_right`
- [x] `Not`
- [x] `BitXorOr` and `BitXorOrAssign`
- [x] `BitOr` and `BitOrAssign`
- [x] `BitAnd` and `BitAndAssign`
- [x] `Saturating::swap_bytes`
- [x] `Saturating::reverse_bits`
Use if-let guards in the codebase and various other pattern cleanups
Dogfooding if-let guards as experimentation for the feature.
Tracking issue #51114. Conflicts with #87937.
Add SAFETY comments to core::slice::sort::partition_in_blocks
A few more SAFETY comments for #66219. There are still a few more in this module.
`@rustbot` label T-libs T-compiler C-cleanup
Fix references to `ControlFlow` in docs
The `Iterator::for_each` method previously stated that it was not possible to use `break` and `continue` in it — this has been updated to acknowledge the stabilization of `ControlFlow`. Additionally, `ControlFlow` was referred to as `crate::ops::ControlFlow` which is not the correct path for an end user.
r? `@jyn514`
add Cell::as_array_of_cells, similar to Cell::as_slice_of_cells
I'd like to propose adding `Cell::as_array_of_cells`, as a natural analog to `Cell::as_slice_of_cells`. I don't have a specific use case in mind, other than that supporting slices but not arrays feels like a gap. Do other folks agree with that intuition? Would this addition be substantial enough to need an RFC?
---
Previously, converting `&mut [T; N]` to `&[Cell<T>; N]` looks like this:
```rust
let array = &mut [1, 2, 3];
let cells: &[Cell<i32>; 3] = Cell::from_mut(&mut array[..])
.as_slice_of_cells()
.try_into()
.unwrap();
```
With this new helper method, it looks like this:
```rust
let array = &mut [1, 2, 3];
let cells = Cell::from_mut(array).as_array_of_cells();
```
Get piece unchecked in `write`
We already use specialized `zip`, but it seems like we can do a little better by not checking `pieces` length at all.
`Arguments` constructors are now unsafe. So the `format_args!` expansion now includes an `unsafe` block.
<details>
<summary>Local Bench Diff</summary>
```text
name before ns/iter after ns/iter diff ns/iter diff % speedup
fmt::write_str_macro1 22,967 19,718 -3,249 -14.15% x 1.16
fmt::write_str_macro2 35,527 32,654 -2,873 -8.09% x 1.09
fmt::write_str_macro_debug 571,953 575,973 4,020 0.70% x 0.99
fmt::write_str_ref 9,579 9,459 -120 -1.25% x 1.01
fmt::write_str_value 9,573 9,572 -1 -0.01% x 1.00
fmt::write_u128_max 176 173 -3 -1.70% x 1.02
fmt::write_u128_min 138 134 -4 -2.90% x 1.03
fmt::write_u64_max 139 136 -3 -2.16% x 1.02
fmt::write_u64_min 129 135 6 4.65% x 0.96
fmt::write_vec_macro1 24,401 22,273 -2,128 -8.72% x 1.10
fmt::write_vec_macro2 37,096 35,602 -1,494 -4.03% x 1.04
fmt::write_vec_macro_debug 588,291 589,575 1,284 0.22% x 1.00
fmt::write_vec_ref 9,568 9,732 164 1.71% x 0.98
fmt::write_vec_value 9,516 9,625 109 1.15% x 0.99
```
</details>
Previously, converting `&mut [T; N]` to `&[Cell<T>; N]` looks like this:
let array = &mut [1, 2, 3];
let cells: &[Cell<i32>; 3] = Cell::from_mut(&mut array[..])
.as_slice_of_cells()
.try_into()
.unwrap();
With this new helper method, it looks like this:
let array = &mut [1, 2, 3];
let cells: &[Cell<i32>; 3] = Cell::from_mut(array).as_array_of_cells();
Fix example in `Extend<(A, B)>` impl
After looking over the examples in my last PR (#85835) on doc.rust-lang.org/nightly I realized that the example didn't actually show what I wanted it to show 😅
So here's the better example
For some reason, I always forget which variants are smaller and which
are larger when you derive PartialOrd on an enum. And the wording in the
current docs is not entirely clear to me.
So, I often end up making a small enum, deriving PartialOrd on it, and
then writing a `#[test]` with an assert that the top one is smaller than
the bottom one (or the other way around) to figure out which way the
deriving goes.
So then I figured, it would be great if the standard library docs just
had that example, so if I keep forgetting, at least I can figure it out
quickly by looking at std's docs.
The libs-api team agrees to allow const_trait_impl to appear in the
standard library as long as stable code cannot be broken (they are
properly gated) this means if the compiler teams thinks it's okay, then
it's okay.
My priority on constifying would be:
1. Non-generic impls (e.g. Default) or generic impls with no
bounds
2. Generic functions with bounds (that use const impls)
3. Generic impls with bounds
4. Impls for traits with associated types
For people opening constification PRs: please cc me and/or oli-obk.
Uplift the invalid_atomic_ordering lint from clippy to rustc
This is mostly just a rebase of https://github.com/rust-lang/rust/pull/79654; I've copy/pasted the text from that PR below.
r? `@lcnr` since you reviewed the last one, but feel free to reassign.
---
This is an implementation of https://github.com/rust-lang/compiler-team/issues/390.
As mentioned, in general this turns an unconditional runtime panic into a (compile time) lint failure. It has no false positives, and the only false negatives I'm aware of are if `Ordering` isn't specified directly and is comes from an argument/constant/whatever.
As a result of it having no false positives, and the alternative always being strictly wrong, it's on as deny by default. This seems right.
In the [zulip stream](https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Uplift.20the.20.60invalid_atomic_ordering.60.20lint.20from.20clippy/near/218483957) `@joshtriplett` suggested that lang team should FCP this before landing it. Perhaps libs team cares too?
---
Some notes on the code for reviewers / others below
## Changes from clippy
The code is changed from [the implementation in clippy](68cf94f6a6/clippy_lints/src/atomic_ordering.rs) in the following ways:
1. Uses `Symbols` and `rustc_diagnostic_item`s instead of string literals.
- It's possible I should have just invoked Symbol::intern for some of these instead? Seems better to use symbol, but it did require adding several.
2. The functions are moved to static methods inside the lint struct, as a way to namespace them.
- There's a lot of other code in that file — which I picked as the location for this lint because `@jyn514` told me that seemed reasonable.
3. Supports unstable AtomicU128/AtomicI128.
- I did this because it was almost easier to support them than not — not supporting them would have (ideally) required finding a way not to give them a `rustc_diagnostic_item`, which would have complicated an already big macro.
- These don't have tests since I wasn't sure if/how I should make tests conditional on whether or not the target has the atomic... This is to a certain extent an issue of 64bit atomics too, but 128-bit atomics are much less common. Regardless, the existing tests should be *more* than thorough enough here.
4. Minor changes like:
- grammar tweaks ("loads cannot have `Release` **and** `AcqRel` ordering" => "loads cannot have `Release` **or** `AcqRel` ordering")
- function renames (`match_ordering_def_path` => `matches_ordering_def_path`),
- avoiding clippy-specific helper methods that don't exist in rustc_lint and didn't seem worth adding for this case (for example `cx.struct_span_lint` vs clippy's `span_lint_and_help` helper).
## Potential issues
(This is just about the code in this PR, not conceptual issues with the lint or anything)
1. I'm not sure if I should have used a diagnostic item for `Ordering` and its variants (I couldn't figure out how really, so if I should do this some pointers would be appreciated).
- It seems possible that failing to do this might possibly mean there are more cases this lint would miss, but I don't really know how `match_def_path` works and if it has any pitfalls like that, so maybe not.
2. I *think* I deprecated the lint in clippy (CC `@flip1995` who asked to be notified about clippy changes in the future in [this comment](https://github.com/rust-lang/rust/pull/75671#issuecomment-718731659)) but I'm not sure if I need to do anything else there.
- I'm kind of hoping CI will catch if I missed anything, since `x.py test src/tools/clippy` fails with a lot of errors with and without my changes (and is probably a nonsense command regardless). Running `cargo test` from src/tools/clippy also fails with unrelated errors that seem like refactorings that didnt update clippy? So, honestly no clue.
3. I wasn't sure if the description/example I gave good. Hopefully it is. The example is less thorough than the one from clippy here: https://rust-lang.github.io/rust-clippy/master/index.html#invalid_atomic_ordering. Let me know if/how I should change it if it needs changing.
4. It pulls in the `if_chain` crate. This crate was already used in clippy, and seems like it's used elsewhere in rustc, but I'm willing to rewrite it to not use this if needed (I'd prefer not to, all things being equal).
- Deprecate clippy::invalid_atomic_ordering
- Use rustc_diagnostic_item for the orderings in the invalid_atomic_ordering lint
- Reduce code duplication
- Give up on making enum variants diagnostic items and just look for
`Ordering` instead
I ran into tons of trouble with this because apparently the change to
store HIR attrs in a side table also gave the DefIds of the
constructor instead of the variant itself. So I had to change
`matches_ordering` to also check the grandparent of the defid as well.
- Rename `atomic_ordering_x` symbols to just the name of the variant
- Fix typos in checks - there were a few places that said "may not be
Release" in the diagnostic but actually checked for SeqCst in the lint.
- Make constant items const
- Use fewer diagnostic items
- Only look at arguments after making sure the method matches
This prevents an ICE when there aren't enough arguments.
- Ignore trait methods
- Only check Ctors instead of going through `qpath_res`
The functions take values, so this couldn't ever be anything else.
- Add if_chain to allowed dependencies
- Fix grammar
- Remove unnecessary allow
Test and fix `size_hint` for slice’s [r]split* iterators
Adds extensive test (of `size_hint`) for all the _[r]split*_ iterators.
Fixes `size_hint` upper bound for _split_inclusive*_ iterators which was one higher than necessary for non-empty slices.
Fixes `size_hint` lower bound for _[r]splitn*_ iterators when _n == 0_, which was one too high.
**Lower bound being one too high was a logic error, violating the correctness condition of `size_hint`.**
_Edit:_ I’ve opened an issue for that bug, so this PR fixes#87978
Implement `black_box` using intrinsic
Introduce `black_box` intrinsic, as suggested in https://github.com/rust-lang/rust/pull/87590#discussion_r680468700.
This is still codegenned as empty inline assembly for LLVM. For MIR interpretation and cranelift it's treated as identity.
cc `@Amanieu` as this is related to inline assembly
cc `@bjorn3` for rustc_codegen_cranelift changes
cc `@RalfJung` as this affects MIRI
r? `@nagisa` I suppose
Adds extensive test for all the [r]split* iterators.
Fixes size_hint upper bound for split_inclusive* iterators which was one higher than necessary for non-empty slices.
Fixes size_hint lower bound for [r]splitn* iterators when n==0, which was one too high.
The new implementation allows some `memcpy`s to be optimized away,
so the uninit value in ui/sanitize/memory.rs is constructed directly
onto the return place. Therefore the sanitizer now says that the
value is allocated by `main` rather than `random`.
Implement Extend<(A, B)> for (Extend<A>, Extend<B>)
I oriented myself at the implementation of `Iterator::unzip` and also rewrote the impl in terms of `(A, B)::extend` after that.
Since (A, B) now also implements Extend we could also mention in the documentation of unzip that it can do "nested unzipping" (you could unzip `Iterator<Item=(A, (B, C))>` into `(Vec<A>, (Vec<B>, Vec<C>))` for example) but I'm not sure of that so I'm asking here 🙂
(P.S. I saw a couple of people asking if there is an unzip3 but there isn't. So this could be a way to get equivalent functionality)
Added the `Option::unzip()` method
* Adds the `Option::unzip()` method to turn an `Option<(T, U)>` into `(Option<T>, Option<U>)` under the `unzip_option` feature
* Adds tests for both `Option::unzip()` and `Option::zip()`, I noticed that `.zip()` didn't have any
* Adds `#[inline]` to a few of `Option`'s methods that were missing it
Constify implementations of `(Try)From` for int types
I believe this to be one of the (many?) things blocking const (Range) iterators.
~~If this is to be merged maybe that should wait until `#![feature(const_trait_impl)]` no longer needs `#![allow(incomplete_features)]`?~~ - Done
The existing documentation felt a little unhelpfully concise, so this
change tries to improve it by using longer sentences, each of which
specifies which kinds of types it applies to as early as possible. In
particular, the third item starts with “Structs ...” instead of
saying “Foo is a struct” later.
Also, the previous list items “Only the last field has a type
involving `T`” and “`T` is not part of the type of any other
fields” are, as far as I see, redundant with each other, so I removed
the latter.
Avoid using the `copy_nonoverlapping` wrapper through `mem::replace`.
This is a much simpler way to achieve the pre-#86003 behavior of `mem::replace` not needing dynamically-sized `memcpy`s (at least before inlining), than re-doing #81238 (which needs #86699 or something similar).
I didn't notice it until recently, but `ptr::write` already explicitly avoided using the wrapper, while `ptr::read` just called the wrapper (and was the reason for us observing any behavior change from #86003 in Rust-GPU).
<hr/>
The codegen test I've added fails without the change to `core::ptr::read` like this (ignore the `v0` mangling, I was using a worktree with it turned on by default, for this):
```llvm
13: ; core::intrinsics::copy_nonoverlapping::<u8>
14: ; Function Attrs: inlinehint nonlazybind uwtable
15: define internal void `@_RINvNtCscK5tvALCJol_4core10intrinsics19copy_nonoverlappinghECsaS4X3EinRE8_25mem_replace_direct_memcpy(i8*` %src, i8* %dst, i64 %count) unnamed_addr #0 {
16: start:
17: %0 = mul i64 %count, 1
18: call void `@llvm.memcpy.p0i8.p0i8.i64(i8*` align 1 %dst, i8* align 1 %src, i64 %0, i1 false)
not:17 !~~~~~~~~~~~~~~~~~~~~~ error: no match expected
19: ret void
20: }
```
With the `core::ptr::read` change, `core::intrinsics::copy_nonoverlapping` doesn't get instantiated and the test passes.
<hr/>
r? `@m-ou-se` cc `@nagisa` (codegen test) `@oli-obk` / `@RalfJung` (miri diagnostic changes)
rustc: Fill out remaining parts of C-unwind ABI
This commit intends to fill out some of the remaining pieces of the
C-unwind ABI. This has a number of other changes with it though to move
this design space forward a bit. Notably contained within here is:
* On `panic=unwind`, the `extern "C"` ABI is now considered as "may
unwind". This fixes a longstanding soundness issue where if you
`panic!()` in an `extern "C"` function defined in Rust that's actually
UB because the LLVM representation for the function has the `nounwind`
attribute, but then you unwind.
* Whether or not a function unwinds now mainly considers the ABI of the
function instead of first checking the panic strategy. This fixes a
miscompile of `extern "C-unwind"` with `panic=abort` because that ABI
can still unwind.
* The aborting stub for non-unwinding ABIs with `panic=unwind` has been
reimplemented. Previously this was done as a small tweak during MIR
generation, but this has been moved to a separate and dedicated MIR
pass. This new pass will, for appropriate functions and function
calls, insert a `cleanup` landing pad for any function call that may
unwind within a function that is itself not allowed to unwind. Note
that this subtly changes some behavior from before where previously on
an unwind which was caught-to-abort it would run active destructors in
the function, and now it simply immediately aborts the process.
* The `#[unwind]` attribute has been removed and all users in tests and
such are now using `C-unwind` and `#![feature(c_unwind)]`.
I think this is largely the last piece of the RFC to implement.
Unfortunately I believe this is still not stabilizable as-is because
activating the feature gate changes the behavior of the existing `extern
"C"` ABI in a way that has no replacement. My thinking for how to enable
this is that we add support for the `C-unwind` ABI on stable Rust first,
and then after it hits stable we change the behavior of the `C` ABI.
That way anyone straddling stable/beta/nightly can switch to `C-unwind`
safely.
#[inline] slice::Iter::advance_by
https://github.com/rust-lang/rust/pull/87387#issuecomment-891942661 was marked as a regression. One of the methods in the PR was missing an inline annotation unlike all the other methods on slice iterators.
Let's see if that makes a difference.
Make wrapping_neg() use wrapping_sub(), #[inline(always)]
This is a follow-up change to the fix for #75598. It simplifies the implementation of wrapping_neg() for all integer types by just calling 0.wrapping_sub(self) and always inlines it. This leads to much less assembly code being emitted for opt-level≤1 and thus much better performance for debug-compiled code.
Background is [this discussion on the internals forum](https://internals.rust-lang.org/t/why-does-rust-generate-10x-as-much-unoptimized-assembly-as-gcc/14930).
Add `core::stream::from_iter`
_Tracking issue: https://github.com/rust-lang/rust/issues/81798_
This_ PR implements `std::stream::from_iter`, as outlined in the _"Converting an Iterator to a Stream"_ section of the [Stream RFC](https://github.com/nellshamrell/rfcs/blob/add-async-stream-rfc/text/0000-async-stream.md#converting-an-iterator-to-a-stream). This function enables converting an `Iterator` to a `Stream` by wrapping each item in the iterator with a `Poll::Ready` instance.
r? `@tmandry`
cc/ `@rust-lang/libs` `@rust-lang/wg-async-foundations`
## Example
Being able to convert from an iterator into a stream is useful when refactoring from iterative loops into a more functional adapter-based style. This is fairly common when using more complex `filter` / `map` / `find` chains. In its basic form this conversion looks like this:
**before**
```rust
let mut output = vec![];
for item in my_vec {
let out = do_io(item).await?;
output.push(out);
}
```
**after**
```rust
use std::stream;
let output = stream::from_iter(my_vec.iter())
.map(async |item| do_io(item).await)
.collect()?;
```
Having a way to convert an `Iterator` to a `Stream` is essential in enabling this flow.
## Implementation Notes
This PR makes use of `unsafe {}` to pin an item. Currently we're having conversations on the libs stream in Zulip how to bring `pin-project` in as a dependency to `core` so we can omit the `unsafe {}`.
This PR also includes a documentation block which references `Stream::next` which currently doesn't exist in the stdlib (originally included in the RFC and PR, but later omitted because of an unresolved issue). `stream::from_iter` can't stabilize before `Stream` does, and there's still a chance we may stabilize `Stream` with a `next` method. So this PR includes documentation referencing that method, which we can remove as part of stabilization if by any chance we don't have `Stream::next`.
## Alternatives Considered
### `impl IntoStream for T: IntoIterator`
An obvious question would be whether we could make it so every iterator can automatically be converted into a stream by calling `into_stream` on it. The answer is: "perhaps, but it could cause type issues". Types like `std::collections` may want to opt to create manual implementations for `IntoStream` and `IntoIter`, which wouldn't be possible if it was implemented through a catch-all trait.
Possibly an alternative such as `impl IntoStream for T: Iterator` could work, but it feels somewhat restrictive. In the end, converting an iterator to a stream is likely to be a bit of a niche case. And even then, **adding a standalone function to convert an `Iterator` into a `Stream` would not be mutually exclusive with a blanket implementation**.
### Naming
The exact name can be debated in the period before stabilization. But I've chosen `stream::from_iter` rather than `stream::iter` because we are _creating a stream from an iterator_ rather than _iterating a stream_. We also expect to add a stream counterpart to `iter::from_fn` later on (blocked on async closures), and having `stream::from_fn` and `stream::from_iter` would feel like a consistent pair. It also has prior art in `async_std::stream::from_iter`.
## Future Directions
### Stream conversions for collections
This is a building block towards implementing `stream/stream_mut/into_stream` methods for `std::collections`, `std::vec`, and more. This would allow even quicker refactorings from using loops to using iterator adapters by omitting the import altogether:
**before**
```rust
use std::stream;
let output = stream::from_iter(my_vec.iter())
.map(async |item| do_io(item).await)
.collect()?;
```
**after**
```rust
let output = my_vec
.stream()
.map(async |item| do_io(item).await)
.collect()?;
```
This commit intends to fill out some of the remaining pieces of the
C-unwind ABI. This has a number of other changes with it though to move
this design space forward a bit. Notably contained within here is:
* On `panic=unwind`, the `extern "C"` ABI is now considered as "may
unwind". This fixes a longstanding soundness issue where if you
`panic!()` in an `extern "C"` function defined in Rust that's actually
UB because the LLVM representation for the function has the `nounwind`
attribute, but then you unwind.
* Whether or not a function unwinds now mainly considers the ABI of the
function instead of first checking the panic strategy. This fixes a
miscompile of `extern "C-unwind"` with `panic=abort` because that ABI
can still unwind.
* The aborting stub for non-unwinding ABIs with `panic=unwind` has been
reimplemented. Previously this was done as a small tweak during MIR
generation, but this has been moved to a separate and dedicated MIR
pass. This new pass will, for appropriate functions and function
calls, insert a `cleanup` landing pad for any function call that may
unwind within a function that is itself not allowed to unwind. Note
that this subtly changes some behavior from before where previously on
an unwind which was caught-to-abort it would run active destructors in
the function, and now it simply immediately aborts the process.
* The `#[unwind]` attribute has been removed and all users in tests and
such are now using `C-unwind` and `#![feature(c_unwind)]`.
I think this is largely the last piece of the RFC to implement.
Unfortunately I believe this is still not stabilizable as-is because
activating the feature gate changes the behavior of the existing `extern
"C"` ABI in a way that has no replacement. My thinking for how to enable
this is that we add support for the `C-unwind` ABI on stable Rust first,
and then after it hits stable we change the behavior of the `C` ABI.
That way anyone straddling stable/beta/nightly can switch to `C-unwind`
safely.
Add missing "allocated object" doc link to `<*mut T>::add`
The portion of the documentation expecting the link was already there, but it was rendered as "[allocated object]". The added reference is just copied from the documentation for `<*const T>::add`.
rfc3052 followup: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
Partially stabilize `const_slice_first_last`
This stabilizes the non-`mut` methods of `const_slice_first_last` as `const`. These methods are trivial to implement and have no blockers that I am aware of.
`@rustbot` label +A-const-fn +S-waiting-on-review +T-libs-api
Move UnwindSafe, RefUnwindSafe, AssertUnwindSafe to core
They were previously only available in std::panic, not core::panic.
- https://doc.rust-lang.org/1.51.0/std/panic/trait.UnwindSafe.html
- https://doc.rust-lang.org/1.51.0/std/panic/trait.RefUnwindSafe.html
- https://doc.rust-lang.org/1.51.0/std/panic/struct.AssertUnwindSafe.html
Where this is relevant: trait objects! Inside a `#![no_std]` library it's otherwise impossible to have a struct holding a trait object, and at the same time can be used from downstream std crates in a way that doesn't interfere with catch_unwind.
```rust
// common library
#![no_std]
pub struct Thing {
pub(crate) x: &'static (dyn SomeTrait + Send + Sync),
}
pub(crate) trait SomeTrait {...}
```
```rust
// downstream application
fn main() {
let thing: library::Thing = ...;
let _ = std::panic::catch_unwind(|| { let _ = thing; }); // does not work :(
}
```
See a4131708e2/src/gradient.rs (L7-L15) for a real life example of needing to work around this problem. In particular that workaround would not even be viable if implementors of the trait were provided externally by a caller, as the `feature = "std"` would become non-additive in that case.
What happens without the UnwindSafe constraints:
```rust
fn main() {
let gradient = colorous::VIRIDIS;
let _ = std::panic::catch_unwind(|| { let _ = gradient; });
}
```
```console
error[E0277]: the type `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
--> src/main.rs:3:13
|
3 | let _ = std::panic::catch_unwind(|| { let _ = gradient; });
| ^^^^^^^^^^^^^^^^^^^^^^^^ `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
|
::: .rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:430:40
|
430 | pub fn catch_unwind<F: FnOnce() -> R + UnwindSafe, R>(f: F) -> Result<R> {
| ---------- required by this bound in `catch_unwind`
|
= help: within `Gradient`, the trait `RefUnwindSafe` is not implemented for `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)`
= note: required because it appears within the type `&'static (dyn colorous::gradient::EvalGradient + Send + Sync + 'static)`
= note: required because it appears within the type `Gradient`
= note: required because of the requirements on the impl of `UnwindSafe` for `&Gradient`
= note: required because it appears within the type `[closure@src/main.rs:3:38: 3:62]`
```
Implement advance_by, advance_back_by for slice::{Iter, IterMut}
Part of #77404.
Picking up where #77633 was closed.
I have addressed https://github.com/rust-lang/rust/pull/77633#issuecomment-771842599 by restoring `nth` and `nth_back`. So according to that comment this should already be r=m-ou-se, but it has been sitting for a while.
Add docs about performance and `Iterator::map` to `[T; N]::map`
This suboptimal code gen for some usages of array::map got a bit of
attention by multiple people throughout the community. Some cases:
- https://github.com/rust-lang/rust/issues/75243#issuecomment-866051086
- https://github.com/rust-lang/rust/issues/75243#issuecomment-874732134
- https://www.reddit.com/r/rust/comments/oeqqf7/unexpected_high_stack_usage/
My *guess* is that this gets the attention it gets because in JavaScript
(and potentially other languages), a `map` function on arrays is very
commonly used since in those languages, arrays basically take the role
of Rust's iterator. I considered explicitly naming JavaScript in the
first paragraph I added, but I couldn't find precedence of mentioning
other languages in standard library doc, so I didn't add it.
When array::map was stabilized, we still wanted to add docs, but that
somehow did not happen in time. So here we are. Not sure if this sounds
crazy but maybe it is worth considering beta backporting this? Only if
it's not a lot of work, of course! But yeah, stabilized array::map is
already in beta and if this problem is really as big as it sometimes seems,
might be worth having the docs in place when 1.55 is released.
CC ``@CryZe``
r? ``@m-ou-se`` (since you were involved in that discussion and the stabilization)
Optimize fmt::PadAdapter::wrap
After adding the first `write!` usage to my project and printing the result to the console, I noticed, that my binary contains the strings "called `Option::unwrap()` on a `None` value`" and more importantly "C:\Users\Patrick Fischer\.rustup\toolchains\nightly-x86_64-pc-windows-msvc\lib\rustlib\src\rust\library\core\src\fmt\builders.rs", with my release build being configured as follows:
```
[profile.release]
panic = "abort"
codegen-units = 1
strip = "symbols" # the important bit
lto = true
```
I am in a no_std environment and my custom panic handler is a simple `loop {}`. I did not expect the above information to be preserved. I heavily suspect the edited function to be the culprit. It contains the only direct use of `Option::unwrap` in the entire file and I tracked the symbols in the assembly to be used from the section `_ZN68_$LT$core..fmt..builders..PadAdapter$u20$as$u20$core..fmt..Write$GT$9write_str17ha1d5e5efe167202aE`.
Aside from me suspecting this function to be the culprit, the replaced code performs the same operation as `Option::insert`, but without the `unreachable_unchecked` optimization `Option::insert` provides. Therefore, it makes sense to me to use the more optimized version, instead.
As I don't change any semantics, I hope a simple pull request suffices.
Fix may not to appropriate might not or must not
I went through and changed occurrences of `may not` to be more explicit with `might not` and `must not`.
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
Add #[track_caller] for some function in core::mem.
These functions can panic for some types. This makes the panic point to the code that calls e.g. mem::uninitialized(), instead of inside the definition of mem::uninitialized.
Make const panic!("..") work in Rust 2021.
During const eval, this replaces calls to core::panicking::panic_fmt and std::panicking::being_panic_fmt with a call to a new const fn: core::panicking::const_panic_fmt. That function uses fmt::Arguments::as_str() to get the str and calls panic_str with that instead.
panic!() invocations with formatting arguments are still not accepted, as the creation of such a fmt::Arguments cannot be done in constant functions right now.
r? `@RalfJung`
Remove unsound TrustedRandomAccess implementations
Removes the implementations that depend on the user-definable trait `Copy`.
Fixes#85873 in the most straightforward way.
<hr>
_Edit:_ This PR now contains additional trait infrastructure to avoid performance regressions around in-place collect, see the discussion in this thread starting from the codegen test failure at https://github.com/rust-lang/rust/pull/85874#issuecomment-872327577.
With this PR, `TrustedRandomAccess` gains additional documentation that specifically allows for and specifies the safety conditions around subtype coercions – those coercions can happen in safe Rust code with the `Zip` API’s usage of `TrustedRandomAccess`. This PR introduces a new supertrait of `TrustedRandomAccess`(currently named `TrustedRandomAccessNoCoerce`) that _doesn’t allow_ such coercions, which means it can be still be useful for optimizing cases such as in-place collect where no iterator is handed out to a user (who could do coercions) after a `get_unchecked` call; the benefit of the supertrait is that it doesn’t come with the additional safety conditions around supertraits either, so it can be implemented for more types than `TrustedRandomAccess`.
The `TrustedRandomAccess` implementations for `vec::IntoIter`, `vec_deque::IntoIter`, and `array::IntoIter` are removed as they don’t conform with the newly documented safety conditions, this way unsoundness is removed. But this PR in turn (re-)adds a `TrustedRandomAccessNoCoerce` implementation for `vec::IntoIter` to avoid performance regressions from stable in a case of in-place collecting of `Vec`s [the above-mentioned codegen test failure]. Re-introducing the (currently nightly+beta-only) impls for `VecDeque`’s and `[T; N]`’s iterators is technically possible, but goes beyond the scope of this PR (i.e. it can happen in a future PR).
Remove P: Unpin bound on impl Future for Pin
We can safely produce a `Pin<&mut P::Target>` without moving out of the `Pin` by using `Pin::as_mut` directly.
The `Unpin` bound was originally added in #56939 following the recommendation of ``@withoutboats`` in https://github.com/rust-lang/rust/issues/55766#issue-378417538
That comment does not give explicit justification for why the bound should be added. The relevant context was:
> [ ] Remove `impl<P> Unpin for Pin<P>`
>
> This impl is not justified by our standard justification for unpin impls: there is no pointer direction between `Pin<P>` and `P`. Its usefulness is covered by the impls for pointers themselves.
>
> This futures impl (link to the impl changed in this PR) will need to change to add a `P: Unpin` bound.
The decision to remove the unconditional impl of `Unpin for Pin` is sound (these days there is just an auto-impl for when `P: Unpin`). But, I think the decision to also add the `Unpin` bound for `impl Future` may have been unnecessary. Or if that's not the case, I'd be very interested to have the argument for why written down somewhere. The bound _appears_ to not be needed, as demonstrated by the change requiring no unsafe code and by the existence of `Pin::as_mut`.
Stabilize core::task::ready!
_Tracking issue: https://github.com/rust-lang/rust/issues/70922_
This PR stabilizes the `task::ready!` macro. Similar to https://github.com/rust-lang/rust/pull/80886, this PR was waiting on https://github.com/rust-lang/rust/issues/74355 to be fixed.
The `task::ready!` API has existed in the futures ecosystem for several years, and was added on nightly last year in https://github.com/rust-lang/rust/pull/70817. The motivation for this macro is the same as it was back then: virtually every single manual future implementation makes use of this; so much so that it's one of the few things included in the [futures-core](https://docs.rs/futures-core/0.3.12/futures_core) library.
r? ``@tmandry``
cc/ ``@rust-lang/wg-async-foundations`` ``@rust-lang/libs``
## Example
```rust
use core::task::{Context, Poll};
use core::future::Future;
use core::pin::Pin;
async fn get_num() -> usize {
42
}
pub fn do_poll(cx: &mut Context<'_>) -> Poll<()> {
let mut f = get_num();
let f = unsafe { Pin::new_unchecked(&mut f) };
let num = ready!(f.poll(cx));
// ... use num
Poll::Ready(())
}
```
