Rwlock downgrade
Tracking Issue: #128203
This PR adds a `downgrade` method for `RwLock` / `RwLockWriteGuard` on all currently supported platforms.
Outstanding questions:
- [x] ~~Does the `futex.rs` change affect performance at all? It doesn't seem like it will but we can't be certain until we bench it...~~
- [x] ~~Should the SOLID platform implementation [be ported over](https://github.com/rust-lang/rust/pull/128219#discussion_r1693470090) to the `queue.rs` implementation to allow it to support downgrades?~~
mark is_val_statically_known intrinsic as stably const-callable
The intrinsic doesn't actually "do" anything in terms of language semantics, and we are already using it in stable const fn. So let's just properly mark it as stably const-callable to avoid needing `rustc_allow_const_fn_unstable` (and thus reducing noise and keeping the remaining `rustc_allow_const_fn_unstable` as a more clear signal).
Cc `@rust-lang/lang` usually you have to approve exposing intrinsics in const, but this intrinsic is basically just a compiler implementation detail. So FCP doesn't seem necessary.
Cc `@rust-lang/wg-const-eval`
This commit fixes a memory ordering bug in the futex implementation
(`Relaxed` -> `Release` on `downgrade`).
This commit also removes a badly written test that deadlocked and
replaces it with a more reasonable test based on an already-tested
`downgrade` test from the parking-lot crate.
This commit adds the `downgrade` method onto the inner `RwLock` queue
implementation.
There are also a few other style patches included in this commit.
Co-authored-by: Jonas Böttiger <jonasboettiger@icloud.com>
This commit only has documentation changes and a few things moved around
the file. The very few code changes are cosmetic: changes like turning a
`match` statement into an `if let` statement or reducing indentation for
long if statements.
This commit also adds several safety comments on top of `unsafe` blocks
that might not be immediately obvious to a first-time reader.
Code "changes" are in:
- `add_backlinks_and_find_tail`
- `lock_contended`
A majority of the changes are just expanding the comments from 80
columns to 100 columns.
Always inline functions signatures containing `f16` or `f128`
There are a handful of tier 2 and tier 3 targets that cause a LLVM crash or linker error when generating code that contains `f16` or `f128`. The cranelift backend also does not support these types. To work around this, every function in `std` or `core` that contains these types must be marked `#[inline]` in order to avoid sending any code to the backend unless specifically requested.
However, this is inconvenient and easy to forget. Introduce a check for these types in the frontend that automatically inlines any function signatures that take or return `f16` or `f128`.
Note that this is not a perfect fix because it does not account for the types being passed by reference or as members of aggregate types, but this is sufficient for what is currently needed in the standard library.
Fixes: https://github.com/rust-lang/rust/issues/133035
Closes: https://github.com/rust-lang/rust/pull/133037
[illumos] use pipe2 to create anonymous pipes
pipe2 allows the newly-created pipe to atomically be CLOEXEC.
pipe2 was added to illumos a long time ago:
5dbfd19ad5. I've verified that this change passes all of std's tests on illumos.
Fix compilation error on Solaris due to flock usage
PR 130999 added the file_lock feature, but libc does not define flock() for the Solaris platform leading to a compilation error.
Additionally, I went through all the Tier 2 platforms and read through their documentation to see whether flock was implemented. This turned up 5 more Unix platforms where flock is not supported, even though it may exist in the libc crate.
Fixes https://github.com/rust-lang/rust/issues/132921
Related to #130999
These types are currently passed by reference, which does not avoid the
backend crashes. Change these back to being passed by value, which makes
the types easier to detect for automatic inlining.
Fix a copy-paste issue in the NuttX raw type definition
This file is copied from the rtems as initial implementation, and forgot to change the OS name in the comment.
Rollup of 7 pull requests
Successful merges:
- #131304 (float types: move copysign, abs, signum to libcore)
- #132907 (Change intrinsic declarations to new style)
- #132971 (Handle infer vars in anon consts on stable)
- #133003 (Make `CloneToUninit` dyn-compatible)
- #133004 (btree: simplify the backdoor between set and map)
- #133008 (update outdated comment about test-float-parse)
- #133012 (Add test cases for #125918)
r? `@ghost`
`@rustbot` modify labels: rollup
This file is copied from the rtems as initial implementation, and
forgot to change the OS name in the comment.
Signed-off-by: Huang Qi <huangqi3@xiaomi.com>
btree: simplify the backdoor between set and map
The internal `btree::Recover` trait acted as a private API between
`BTreeSet` and `BTreeMap`, but we can use `pub(_)` restrictions these
days, and some of the methods don't need special handling anymore.
* `BTreeSet::get` can use `BTreeMap::get_key_value`
* `BTreeSet::take` can use `BTreeMap::remove_entry`
* `BTreeSet::replace` does need help, but this now uses a `pub(super)`
method on `BTreeMap` instead of the trait.
* `btree::Recover` is now removed.
Make `CloneToUninit` dyn-compatible
Make `CloneToUninit` dyn-compatible, by making `clone_to_uninit`'s `dst` parameter `*mut u8` instead of `*mut Self`, so the method does not reference `Self` except in the `self` parameter and is thus dispatchable from a trait object.
This allows, among other things, adding `CloneToUninit` as a supertrait bound for `trait Foo` to allow cloning `dyn Foo` in some containers. Currently, this means that `Rc::make_mut` and `Arc::make_mut` can work with `dyn Foo` where `trait Foo: CloneToUninit`.
<details><summary>Example</summary>
```rs
#![feature(clone_to_uninit)]
use std::clone::CloneToUninit;
use std::rc::Rc;
use std::fmt::Debug;
use std::borrow::BorrowMut;
trait Foo: BorrowMut<u32> + CloneToUninit + Debug {}
impl<T: BorrowMut<u32> + CloneToUninit + Debug> Foo for T {}
fn main() {
let foo: Rc<dyn Foo> = Rc::new(42_u32);
let mut bar = foo.clone();
*Rc::make_mut(&mut bar).borrow_mut() = 37;
dbg!(foo, bar); // 42, 37
}
```
</details>
Eventually, `Box::<T>::clone` is planned to be converted to use `T::clone_to_uninit`, which when combined with this change, will allow cloning `Box<dyn Foo>` where `trait Foo: CloneToUninit` without any additional `unsafe` code for the author of `trait Foo`.[^1]
This PR should have no stable side-effects, as `CloneToUninit` is unstable so cannot be mentioned on stable, and `CloneToUninit` is not used as a supertrait anywhere in the stdlib.
This change removes some length checks that could only fail if library UB was already hit (e.g. calling `<[T]>::clone_to_uninit` with a too-small-length `dst` is library UB and was previously detected[^2]; since `dst` does not have a length anymore, this now cannot be detected[^3]).
r? libs-api
-----
I chose to make the parameter `*mut u8` instead of `*mut ()` because that might make it simpler to pass the result of `alloc` to `clone_to_uninit`, but `*mut ()` would also make sense, and any `*mut ConcreteType` would *work*. The original motivation for [using specifically `*mut ()`](https://github.com/rust-lang/rust/pull/116113#discussion_r1335303908) appears to be `std::ptr::from_raw_parts_mut`, but that now [takes `*mut impl Thin`](https://doc.rust-lang.org/nightly/std/ptr/fn.from_raw_parts.html) instead of `*mut ()`. I have another branch where the parameter is `*mut ()`, if that is preferred.
It *could* also take something like `&mut [MaybeUninit<u8>]` to be dyn-compatible but still allow size-checking and in some cases safe writing, but this is already an `unsafe` API where misuse is UB, so I'm not sure how many guardrails it's worth adding here, and `&mut [MaybeUninit<u8>]` might be overly cumbersome to construct for callers compared to `*mut u8`
[^1]: Note that `impl<T: CloneToUninit + ?Sized> Clone for Box` must be added before or at the same time as when `CloneToUninit` becomes stable, due to `Box` being `#[fundamental]`, as if there is any stable gap between the stabilization of `CloneToUninit` and `impl<T: CloneToUninit + ?Sized> Clone for Box`, then users could implement both `CloneToUninit for dyn LocalTrait` and separately `Clone for Box<dyn LocalTrait>` during that gap, and be broken by the introduction of `impl<T: CloneToUninit + ?Sized> Clone for Box`.
[^2]: Using a `debug_assert_eq` in [`core::clone::uninit::CopySpec::clone_slice`](https://doc.rust-lang.org/nightly/src/core/clone/uninit.rs.html#28).
[^3]: This PR just uses [the metadata (length) from `self`](e0c1c8bc50/library/core/src/clone.rs (L286)) to construct the `*mut [T]` to pass to `CopySpec::clone_slice` in `<[T]>::clone_to_uninit`.
Change intrinsic declarations to new style
Pr is for issue #132735
This changes the first `extern "rust-intrinsic"` block to the new style.
r? `@RalfJung`
float types: move copysign, abs, signum to libcore
These operations are explicitly specified to act "bitwise", i.e. they just act on the sign bit and do not even quiet signaling NaNs. We also list them as ["non-arithmetic operations"](https://doc.rust-lang.org/nightly/std/primitive.f32.html#nan-bit-patterns), and all the other non-arithmetic operations are in libcore. There's no reason to expect them to require any sort of runtime support, and from [these experiments](https://github.com/rust-lang/rust/issues/50145#issuecomment-997301250) it seems like LLVM indeed compiles them in a way that does not require any sort of runtime support.
Nominating for `@rust-lang/libs-api` since this change takes immediate effect on stable.
Part of https://github.com/rust-lang/rust/issues/50145.
improve codegen of fmt_num to delete unreachable panic
it seems LLVM doesn't realize that `curr` is always decremented at least once in either loop formatting characters of the input string by their appropriate radix, and so the later `&buf[curr..]` generates a check for out-of-bounds access and panic. this is unreachable in reality as even for `x == T::zero()` we'll produce at least the character `Self::digit(T::zero())`, yielding at least one character output, and `curr` will always be at least one below `buf.len()`.
adjust `fmt_int` to make this fact more obvious to the compiler, which fortunately (or unfortunately) results in a measurable performance improvement for workloads heavy on formatting integers.
in the program i'd noticed this in, you can see the `cmp $0x80,%rdi; ja 7c` here, which branches to a slice index fail helper:
<img width="660" alt="before" src="https://github.com/rust-lang/rust/assets/4615790/ac482d54-21f8-494b-9c83-4beadc3ca0ef">
where after this change the function is broadly similar, but smaller, with one fewer registers updated in each pass through the loop in addition the never-taken `cmp/ja` being gone:
<img width="646" alt="after" src="https://github.com/rust-lang/rust/assets/4615790/1bee1d76-b674-43ec-9b21-4587364563aa">
this represents a ~2-3% difference in runtime in my [admittedly comically i32-formatting-bound](https://github.com/athre0z/disas-bench/blob/master/bench/yaxpeax/src/main.rs#L58-L67) use case (printing x86 instructions, including i32 displacements and immediates) as measured on a ryzen 9 3950x.
the impact on `<impl LowerHex for i8>::fmt` is both more dramatic and less impactful: it continues to have a loop that is evaluated at most twice, though the compiler doesn't know that to unroll it. the generated code there is identical to the impl for `i32`. there, the smaller loop body has less effect on runtime, and removing the never-taken slice bounds check is offset by whatever address recalculation is happening with the `lea/add/neg` at the end of the loop. it behaves about the same before and after.
---
i initially measured slightly better outcomes using `unreachable_unchecked()` here instead, but that was hacking on std and rebuilding with `-Z build-std` on an older rustc (nightly 5b377cece, 2023-06-30). it does not yield better outcomes now, so i see no reason to proceed with that approach at all.
<details>
<summary>initial notes about that, seemingly irrelevant on modern rustc</summary>
i went through a few tries at getting llvm to understand the bounds check isn't necessary, but i should mention the _best_ i'd seen here was actually from the existing `fmt_int` with a diff like
```diff
if x == zero {
// No more digits left to accumulate.
break;
};
}
}
+
+ if curr >= buf.len() {
+ unsafe { core::hint::unreachable_unchecked(); }
+ }
let buf = &buf[curr..];
```
posting a random PR to `rust-lang/rust` to do that without a really really compelling reason seemed a bit absurd, so i tried to work that into something that seems more palatable at a glance. but if you're interested, that certainly produced better (x86_64) code through LLVM. in that case with `buf.iter_mut().rev()` as the iterator, `<impl LowerHex for i8>::fmt` actually unrolls into something like
```
put_char(x & 0xf);
let mut len = 1;
if x > 0xf {
put_char((x >> 4) & 0xf);
len = 2;
}
pad_integral(buf[buf.len() - len..]);
```
it's pretty cool! `<impl LowerHex for i32>::fmt` also was slightly better. that all resulted in closer to an 6% difference in my use case.
</details>
---
i have not looked at formatters other than LowerHex/UpperHex with this change, though i'd be a bit shocked if any were _worse_.
(i have absolutely _no_ idea how you'd regression test this, but that might be just my not knowing what the right tool for that would be in rust-lang/rust. i'm of half a mind that this is small and fiddly enough to not be worth landing lest it quietly regress in the future anyway. but i didn't want to discard the idea without at least offering it upstream here)
The internal `btree::Recover` trait acted as a private API between
`BTreeSet` and `BTreeMap`, but we can use `pub(_)` restrictions these
days, and some of the methods don't need special handling anymore.
* `BTreeSet::get` can use `BTreeMap::get_key_value`
* `BTreeSet::take` can use `BTreeMap::remove_entry`
* `BTreeSet::replace` does need help, but this now uses a `pub(super)`
method on `BTreeMap` instead of the trait.
* `btree::Recover` is now removed.
