fix data race in thread::scope
Puts the `ScopeData` into an `Arc` so it sticks around as long as we need it.
This means one extra `Arc::clone` per spawned scoped thread, which I hope is fine.
Fixes https://github.com/rust-lang/rust/issues/98498
r? `````@m-ou-se`````
[core] add `Exclusive` to sync
(discussed here: https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Adding.20.60SyncWrapper.60.20to.20std)
`Exclusive` is a wrapper that exclusively allows mutable access to the inner value if you have exclusive access to the wrapper. It acts like a compile time mutex, and hold an unconditional `Sync` implementation.
## Justification for inclusion into std
- This wrapper unblocks actual problems:
- The example that I hit was a vector of `futures::future::BoxFuture`'s causing a central struct in a script to be non-`Sync`. To work around it, you either write really difficult code, or wrap the futures in a needless mutex.
- Easy to maintain: this struct is as simple as a wrapper can get, and its `Sync` implementation has very clear reasoning
- Fills a gap: `&/&mut` are to `RwLock` as `Exclusive` is to `Mutex`
## Public Api
```rust
// core::sync
#[derive(Default)]
struct Exclusive<T: ?Sized> { ... }
impl<T: ?Sized> Sync for Exclusive {}
impl<T> Exclusive<T> {
pub const fn new(t: T) -> Self;
pub const fn into_inner(self) -> T;
}
impl<T: ?Sized> Exclusive<T> {
pub const fn get_mut(&mut self) -> &mut T;
pub const fn get_pin_mut(Pin<&mut self>) -> Pin<&mut T>;
pub const fn from_mut(&mut T) -> &mut Exclusive<T>;
pub const fn from_pin_mut(Pin<&mut T>) -> Pin<&mut Exclusive<T>>;
}
impl<T: Future> Future for Exclusive { ... }
impl<T> From<T> for Exclusive<T> { ... }
impl<T: ?Sized> Debug for Exclusive { ... }
```
## Naming
This is a big bikeshed, but I felt that `Exclusive` captured its general purpose quite well.
## Stability and location
As this is so simple, it can be in `core`. I feel that it can be stabilized quite soon after it is merged, if the libs teams feels its reasonable to add. Also, I don't really know how unstable feature work in std/core's codebases, so I might need help fixing them
## Tips for review
The docs probably are the thing that needs to be reviewed! I tried my best, but I'm sure people have more experience than me writing docs for `Core`
### Implementation:
The API is mostly pulled from https://docs.rs/sync_wrapper/latest/sync_wrapper/struct.SyncWrapper.html (which is apache 2.0 licenesed), and the implementation is trivial:
- its an unsafe justification for pinning
- its an unsafe justification for the `Sync` impl (mostly reasoned about by ````@danielhenrymantilla```` here: https://github.com/Actyx/sync_wrapper/pull/2)
- and forwarding impls, starting with derivable ones and `Future`
Remove feature `const_option` from std
This is part of the effort to reduce the number of unstable features used by std. This one is easy as it's only used in one place.
attempt to optimise vectored write
benchmarked:
old:
```
test io::cursor::tests::bench_write_vec ... bench: 68 ns/iter (+/- 2)
test io::cursor::tests::bench_write_vec_vectored ... bench: 913 ns/iter (+/- 31)
```
new:
```
test io::cursor::tests::bench_write_vec ... bench: 64 ns/iter (+/- 0)
test io::cursor::tests::bench_write_vec_vectored ... bench: 747 ns/iter (+/- 27)
```
More unsafe than I wanted (and less gains) in the end, but it still does the job
These calls allow detecting whether a symlink is a file or a directory,
a distinction Windows maintains, and one important to software that
wants to do further operations on the symlink (e.g. removing it).
Update `std::alloc::System` doc example code style
`return` on the last line of a block is unidiomatic so I don't think the example should be using that here
std: use an event-flag-based thread parker on SOLID
`Mutex` and `Condvar` are being replaced by more efficient implementations, which need thread parking themselves (see #93740). Therefore, the generic `Parker` needs to be replaced on all platforms where the new lock implementation will be used, which, after #96393, are SOLID, SGX and Hermit (more PRs coming soon).
SOLID, conforming to the [μITRON specification](http://www.ertl.jp/ITRON/SPEC/FILE/mitron-400e.pdf), has event flags, which are a thread parking primitive very similar to `Parker`. However, they do not make any atomic ordering guarantees (even though those can probably be assumed) and necessitate a system call even when the thread token is already available. Hence, this `Parker`, like the Windows parker, uses an extra atomic state variable.
I future-proofed the code by wrapping the event flag in a `WaitFlag` structure, as both SGX and Hermit can share the Parker implementation, they just have slightly different primitives (SGX uses signals and Hermit has a thread blocking API).
`````@kawadakk````` I assume you are the target maintainer? Could you test this for me?
Mitigate MMIO stale data vulnerability
Intel publicly disclosed the MMIO stale data vulnerability on June 14. To mitigate this vulnerability, compiler changes are required for the `x86_64-fortanix-unknown-sgx` target.
cc: ````@jethrogb````
Windows: Iterative `remove_dir_all`
This will allow better strategies for use of memory and File handles. However, fully taking advantage of that is left to future work.
Note to reviewer: It's probably best to view the `remove_dir_all_recursive` as a new function. The diff is not very helpful (imho).
Make RwLockReadGuard covariant
Hi, first time contributor here, if anything is not as expected, please let me know.
`RwLockReadGoard`'s type constructor is invariant. Since it behaves like a smart pointer to an immutable reference, there is no reason that it should not be covariant. Take e.g.
```
fn test_read_guard_covariance() {
fn do_stuff<'a>(_: RwLockReadGuard<'_, &'a i32>, _: &'a i32) {}
let j: i32 = 5;
let lock = RwLock::new(&j);
{
let i = 6;
do_stuff(lock.read().unwrap(), &i);
}
drop(lock);
}
```
where the compiler complains that &i doesn't live long enough. If `RwLockReadGuard` is covariant, then the above code is accepted because the lifetime can be shorter than `'a`.
In order for `RwLockReadGuard` to be covariant, it can't contain a full reference to the `RwLock`, which can never be covariant (because it exposes a mutable reference to the underlying data structure). By reducing the data structure to the required pieces of `RwLock`, the rest falls in place.
If there is a better way to do a test that tests successful compilation, please let me know.
Fixes#80392
Fix documentation for `with_capacity` and `reserve` families of methods
Fixes#95614
Documentation for the following methods
- `with_capacity`
- `with_capacity_in`
- `with_capacity_and_hasher`
- `reserve`
- `reserve_exact`
- `try_reserve`
- `try_reserve_exact`
was inconsistent and often not entirely correct where they existed on the following types
- `Vec`
- `VecDeque`
- `String`
- `OsString`
- `PathBuf`
- `BinaryHeap`
- `HashSet`
- `HashMap`
- `BufWriter`
- `LineWriter`
since the allocator is allowed to allocate more than the requested capacity in all such cases, and will frequently "allocate" much more in the case of zero-sized types (I also checked `BufReader`, but there the docs appear to be accurate as it appears to actually allocate the exact capacity).
Some effort was made to make the documentation more consistent between types as well.
Add a `is_known_utf8` flag to `Wtf8Buf`, which tracks whether the
string is known to contain UTF-8. This is efficiently computed in many
common situations, such as when a `Wtf8Buf` is constructed from a `String`
or `&str`, or with `Wtf8Buf::from_wide` which is already doing UTF-16
decoding and already checking for surrogates.
This makes `OsString::into_string` O(1) rather than O(N) on Windows in
common cases.
And, it eliminates the need to scan through the string for surrogates in
`Args::next` and `Vars::next`, because the strings are already being
translated with `Wtf8Buf::from_wide`.
Many things on Windows construct `OsString`s with `Wtf8Buf::from_wide`,
such as `DirEntry::file_name` and `fs::read_link`, so with this patch,
users of those functions can subsequently call `.into_string()` without
paying for an extra scan through the string for surrogates.
This allows implementing traits that require a raw FD on Arc and Box.
Previously, you'd have to add the function to the trait itself:
```rust
trait MyTrait {
fn as_raw_fd(&self) -> RawFd;
}
impl<T: MyTrait> MyTrait for Arc<T> {
fn as_raw_fd(&self) -> RawFd {
(**self).as_raw_fd()
}
}
```
Document Rust's stance on `/proc/self/mem`
Add documentation to `std::os::unix::io` describing Rust's stance on
`/proc/self/mem`, treating it as an external entity which is outside
the scope of Rust's safety guarantees.
`Stdio::makes_pipe`
Wrappers around `std::process::Command` may want to be able to override pipe creation. However, [`std::process::Stdio`](https://doc.rust-lang.org/std/process/struct.Stdio.html) is opaque so there's no way to tell if `Command` was told to create new pipes or not.
This is in some ways a more generic (and cross-platform) alternative to #97149. However, unlike that feature, this comes with the price of the user needing to actually create their own pipes rather than reusing the std one. So I think it stands (or not) on its own.
# Example
```rust
#![feature(stdio_makes_pipe)]
use std::process::Stdio;
let io = Stdio::piped();
assert_eq!(io.makes_pipe(), true);
```
Windows: `CommandExt::async_pipes`
Discussed in https://github.com/tokio-rs/tokio/issues/4670 was the need for third party crates to be able to force `process::Command::spawn` to create pipes as async.
This implements the suggestion for a `async_pipes` method that gives third party crates that option.
# Example:
```rust
use std::process::{Command, Stdio};
Command::new("cmd")
.async_pipes(true)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
```
Stabilize `Path::try_exists()` and improve doc
This stabilizes the `Path::try_exists()` method which returns
`Result<bool, io::Error>` instead of `bool` allowing handling of errors
unrelated to the file not existing. (e.g permission errors)
Along with the stabilization it also:
* Warns that the `exists()` method is error-prone and suggests to use
the newly stabilized one.
* Suggests it instead of `metadata()` to handle errors.
* Mentions TOCTOU bugs to avoid false assumption that `try_exists()` is
completely safe fixed version of `exists()`.
* Renames the feature of still-unstable `std::fs::try_exists()` to
`fs_try_exists` to avoid name conflict.
The tracking issue #83186 remains open to track `fs_try_exists`.
Documentation for the following methods
with_capacity
with_capacity_in
with_capacity_and_hasher
reserve
reserve_exact
try_reserve
try_reserve_exact
was inconsistent and often not entirely correct where they existed on the following types
Vec
VecDeque
String
OsString
PathBuf
BinaryHeap
HashSet
HashMap
BufWriter
LineWriter
since the allocator is allowed to allocate more than the requested capacity in all such cases, and will frequently "allocate" much more in the case of zero-sized types (I also checked BufReader, but there the docs appear to be accurate as it appears to actually allocate the exact capacity).
Some effort was made to make the documentation more consistent between types as well.
Fix with_capacity* methods for Vec
Fix *reserve* methods for Vec
Fix docs for *reserve* methods of VecDeque
Fix docs for String::with_capacity
Fix docs for *reserve* methods of String
Fix docs for OsString::with_capacity
Fix docs for *reserve* methods on OsString
Fix docs for with_capacity* methods on HashSet
Fix docs for *reserve methods of HashSet
Fix docs for with_capacity* methods of HashMap
Fix docs for *reserve methods on HashMap
Fix expect messages about OOM in doctests
Fix docs for BinaryHeap::with_capacity
Fix docs for *reserve* methods of BinaryHeap
Fix typos
Fix docs for with_capacity on BufWriter and LineWriter
Fix consistent use of `hasher` between `HashMap` and `HashSet`
Fix warning in doc test
Add test for capacity of vec with ZST
Fix doc test error
once cell renamings
This PR does the renamings proposed in https://github.com/rust-lang/rust/issues/74465#issuecomment-1153703128
- Move/rename `lazy::{OnceCell, Lazy}` to `cell::{OnceCell, LazyCell}`
- Move/rename `lazy::{SyncOnceCell, SyncLazy}` to `sync::{OnceLock, LazyLock}`
(I used `Lazy...` instead of `...Lazy` as it seems to be more consistent, easier to pronounce, etc)
```@rustbot``` label +T-libs-api -T-libs
Avoid `thread::panicking()` in non-poisoning methods of `Mutex` and `RwLock`
`Mutex::lock()` and `RwLock::write()` are poison-guarded against panics,
in that they set the poison flag if a panic occurs while they're locked.
But if we're already in a panic (`thread::panicking()`), they leave the
poison flag alone.
That check is a bit of a waste for methods that never set the poison
flag though, namely `get_mut()`, `into_inner()`, and `RwLock::read()`.
These use-cases are now split to avoid that unnecessary call.
Windows: No panic if function not (yet) available
In some situations (e.g. #97814) it is possible for required functions to be called before they've had a chance to be loaded. Therefore, we make it possible to recover from this situation simply by looking at error codes.
`@rustbot` label +O-windows
Add `#[inline]` to small fns of futex `RwLock`
The important methods like `read` and `write` were already inlined,
which can propagate all the way to inlining in user code, but these
small state functions were left behind as normal calls. They should
almost always be inlined as well, as they're just a few instructions.
Test NLL fix of bad lifetime inference for reference captured in closure.
This came up as a use-case for `thread::scope` API that only compiles successfully since `feature(nll)` got stabilized recently.
Closes#93203 which had been re-opened for tracking this very test case to be added.
Entry and_modify doc
This PR modifies the documentation for [HashMap](https://doc.rust-lang.org/std/collections/struct.HashMap.html#) and [BTreeMap](https://doc.rust-lang.org/std/collections/struct.BTreeMap.html#) by introducing examples for `and_modify`. `and_modify` is a function that tends to give more idiomatic rust code when dealing with these data structures -- yet it lacked examples and was hidden away. This PR adds that and addresses #98122.
I've made some choices which I tried to explain in my commits. This is my first time contributing to rust, so hopefully, I made the right choices.
os str capacity documentation
This is based on https://github.com/rust-lang/rust/pull/95394 , with expansion and consolidation
to address comments from `@dtolnay` and other `@rust-lang/libs-api` team members.
Add a `BorrowedFd::try_clone_to_owned` and accompanying documentation
Add a `BorrowedFd::try_clone_to_owned`, which returns a new `OwnedFd` sharing the underlying file description. And similar for `BorrowedHandle` and `BorrowedSocket` on WIndows.
This is similar to the existing `OwnedFd::try_clone`, but it's named differently to reflect that it doesn't return `Result<Self, ...>`. I'm open to suggestions for better names.
Also, extend the `unix::io` documentation to mention that `dup` is permitted on `BorrowedFd`.
This was originally requsted [here](https://github.com/rust-lang/rust/issues/88564#issuecomment-910786081). At the time I wasn't sure whether it was desirable, but it does have uses and it helps clarify the API. The documentation previously didn't rule out using `dup` on a `BorrowedFd`, but the API only offered convenient ways to do it from an `OwnedFd`. With this patch, the API allows one to do `try_clone` on any type where it's permitted.
The important methods like `read` and `write` were already inlined,
which can propagate all the way to inlining in user code, but these
small state functions were left behind as normal calls. They should
almost always be inlined as well, as they're just a few instructions.
STD support for the Nintendo 3DS
Rustc already supports compiling for the Nintendo 3DS using the `armv6k-nintendo-3ds` target (Tier 3). Until now though, only `core` and `alloc` were supported. This PR adds standard library support for the Nintendo 3DS. A notable exclusion is `std::thread` support, which will come in a follow-up PR as it requires more complicated changes.
This has been a joint effort by `@Meziu,` `@ian-h-chamberlain,` myself, and prior work by `@rust3ds` members.
### Background
The Nintendo 3DS (Horizon OS) is a mostly-UNIX looking system, with the caveat that it does not come with a full libc implementation out of the box. On the homebrew side (I'm not under NDA), the libc interface is partially implemented by the [devkitPro](https://devkitpro.org/wiki/devkitPro_pacman) toolchain and a user library like [`libctru`](https://github.com/devkitPro/libctru). This is important because there are [some possible legal barriers](https://github.com/rust-lang/rust/pull/88529#issuecomment-919938396) to linking directly to a library that uses the underlying platform APIs, since they might be considered a trade secret or under NDA.
To get around this, the standard library impl for the 3DS does not directly depend on any platform-level APIs. Instead, it expects standard libc functions to be linked in. The implementation of these libc functions is left to the user. Some functions are provided by the devkitPro toolchain, but in our testing, we used the following to fill in the other functions:
- [`libctru`] - provides more basic APIs, such as `nanosleep`. Linked in by way of [`ctru-sys`](https://github.com/Meziu/ctru-rs/tree/master/ctru-sys).
- [`pthread-3ds`](https://github.com/Meziu/pthread-3ds) - provides pthread APIs for `std::thread`. Implemented using [`libctru`].
- [`linker-fix-3ds`](https://github.com/Meziu/rust-linker-fix-3ds) - fulfills some other missing libc APIs. Implemented using [`libctru`].
For more details, see the `src/doc/rustc/src/platform-support/armv6k-nintendo-3ds.md` file added in this PR.
### Notes
We've already upstreamed changes to the [`libc`] crate to support this PR, as well as the upcoming threading PR. These changes have all been released as of 0.2.121, so we bump the crate version in this PR.
Edit: After some rebases, the version bump has already been merged so it doesn't appear in this PR.
A lot of the changes in this PR are straightforward, and follow in the footsteps of the ESP-IDF target: https://github.com/rust-lang/rust/pull/87666.
The 3DS does not support user space process spawning, so these APIs are unimplemented (similar to ESP-IDF).
[`libctru`]: https://github.com/devkitPro/libctru
[`libc`]: https://github.com/rust-lang/libc
Updated the HashMap's documentation to include two references to
add_modify.
The first is when the `Entry` API is mentioned at the beginning. I was
hesitant to change the "attack" example (although I believe that it is
perfect example of where `add_modify` should be used) because both uses
work equally, but one is more idiomatic (`add_modify`).
The second is with the `entry` function that is used for the `Entry`
API. The code example was a perfect use for `add_modify`, which is why
it was changed to reflect that.
This stabilizes the `Path::try_exists()` method which returns
`Result<bool, io::Error>` instead of `bool` allowing handling of errors
unrelated to the file not existing. (e.g permission errors)
Along with the stabilization it also:
* Warns that the `exists()` method is error-prone and suggests to use
the newly stabilized one.
* Suggests it instead of `metadata()` to handle errors.
* Mentions TOCTOU bugs to avoid false assumption that `try_exists()` is
completely safe fixed version of `exists()`.
* Renames the feature of still-unstable `std::fs::try_exists()` to
`fs_try_exists` to avoid name conflict.
The tracking issue #83186 remains open to track `fs_try_exists`.
Integrate measureme's hardware performance counter support.
*Note: this is a companion to https://github.com/rust-lang/measureme/pull/143, and duplicates some information with it for convenience*
**(much later) EDIT**: take any numbers with a grain of salt, they may have changed since initial PR open.
## Credits
I'd like to start by thanking `@alyssais,` `@cuviper,` `@edef1c,` `@glandium,` `@jix,` `@Mark-Simulacrum,` `@m-ou-se,` `@mystor,` `@nagisa,` `@puckipedia,` and `@yorickvP,` for all of their help with testing, and valuable insight and suggestions.
Getting here wouldn't have been possible without you!
(If I've forgotten anyone please let me know, I'm going off memory here, plus some discussion logs)
## Summary
This PR adds support to `-Z self-profile` for counting hardware events such as "instructions retired" (as opposed to being limited to time measurements), using the `rdpmc` instruction on `x86_64` Linux.
While other OSes may eventually be supported, preliminary research suggests some kind of kernel extension/driver is required to enable this, whereas on Linux any user can profile (at least) their own threads.
Supporting Linux on architectures other than x86_64 should be much easier (provided the hardware supports such performance counters), and was mostly not done due to a lack of readily available test hardware.
That said, 32-bit `x86` (aka `i686`) would be almost trivial to add and test once we land the initial `x86_64` version (as all the CPU detection code can be reused).
A new flag `-Z self-profile-counter` was added, to control which of the named `measureme` counters is used, and which defaults to `wall-time`, in order to keep `-Z self-profile`'s current functionality unchanged (at least for now).
The named counters so far are:
* `wall-time`: the existing time measurement
* name chosen for consistency with `perf.rust-lang.org`
* continues to use `std::time::Instant` for a nanosecond-precision "monotonic clock"
* `instructions:u`: the hardware performance counter usually referred to as "Instructions retired"
* here "retired" (roughly) means "fully executed"
* the `:u` suffix is from the Linux `perf` tool and indicates the counter only runs while userspace code is executing, and therefore counts no kernel instructions
* *see [Caveats/Subtracting IRQs](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs) for why this isn't entirely true and why `instructions-minus-irqs:u` should be preferred instead*
* `instructions-minus-irqs:u`: same as `instructions:u`, except the count of hardware interrupts ("IRQs" here for brevity) is subtracted
* *see [Caveats/Subtracting IRQs](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs) for why this should be preferred over `instructions:u`*
* `instructions-minus-r0420:u`: experimental counter, same as `instructions-minus-irqs:u` but subtracting an undocumented counter (`r0420:u`) instead of IRQs
* the `rXXXX` notation is again from Linux `perf`, and indicates a "raw" counter, with a hex representation of the low-level counter configuration - this was picked because we still don't *really* know what it is
* this only exists for (future) testing and isn't included/used in any comparisons/data we've put together so far
* *see [Challenges/Zen's undocumented 420 counter](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Epilogue-Zen’s-undocumented-420-counter) for details on how this counter was found and what it does*
---
There are also some additional commits:
* ~~see [Challenges/Rebasing *shouldn't* affect the results, right?](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Rebasing-*shouldn’t*-affect-the-results,-right) for details on the changes to `rustc_parse` and `rustc_trait_section` (the latter far more dubious, and probably shouldn't be merged, or not as-is)~~
* **EDIT**: the effects of these are no long quantifiable, the PR includes reverts for them
* ~~see [Challenges/`jemalloc`: purging will commence in ten seconds](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#jemalloc-purging-will-commence-in-ten-seconds) for details on the `jemalloc` change~~
* this is also separately found in #77162, and we probably want to avoid doing it by default, ideally we'd use the runtime control API `jemalloc` offers (assuming that can stop the timer that's already running, which I'm not sure about)
* **EDIT**: until we can do this based on `-Z` flags, this commit has also been reverted
* the `proc_macro` change was to avoid randomized hashing and therefore ASLR-like effects
---
**(much later) EDIT**: take any numbers with a grain of salt, they may have changed since initial PR open.
#### Write-up / report
Because of how extensive the full report ended up being, I've kept most of it [on `hackmd.io`](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view), but for convenient access, here are all the sections (with individual links):
<sup>(someone suggested I'd make a backup, so [here it is on the wayback machine](http://web.archive.org/web/20201127164748/https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view) - I'll need to remember to update that if I have to edit the write-up)</sup>
* [**Motivation**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Motivation)
* [**Results**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Results)
* [**Overhead**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Overhead)
*Preview (see the report itself for more details):*
|Counter|Total<br>`instructions-minus-irqs:u`|Overhead from "Baseline"<br>(for all 1903881<br>counter reads)|Overhead from "Baseline"<br>(per each counter read)|
|-|-|-|-|
|Baseline|63637621286 ±6||
|`instructions:u`|63658815885 ±2| +21194599 ±8| +11|
|`instructions-minus-irqs:u`|63680307361 ±13| +42686075 ±19| +22|
|`wall-time`|63951958376 ±10275|+314337090 ±10281|+165|
* [**"Macro" noise (self time)**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#“Macro”-noise-(self-time))
*Preview (see the report itself for more details):*
|| `wall-time` (ns) | `instructions:u` | `instructions-minus-irqs:u`
-: | -: | -: | -:
`typeck` | 5478261360 ±283933373 (±~5.2%) | 17350144522 ±6392 (±~0.00004%) | 17351035832.5 ±4.5 (±~0.00000003%)
`expand_crate` | 2342096719 ±110465856 (±~4.7%) | 8263777916 ±2937 (±~0.00004%) | 8263708389 ±0 (±~0%)
`mir_borrowck` | 2216149671 ±119458444 (±~5.4%) | 8340920100 ±2794 (±~0.00003%) | 8341613983.5 ±2.5 (±~0.00000003%)
`mir_built` | 1269059734 ±91514604 (±~7.2%) | 4454959122 ±1618 (±~0.00004%) | 4455303811 ±1 (±~0.00000002%)
`resolve_crate` | 942154987.5 ±53068423.5 (±~5.6%) | 3951197709 ±39 (±~0.000001%) | 3951196865 ±0 (±~0%)
* [**"Micro" noise (individual sampling intervals)**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#“Micro”-noise-(individual-sampling-intervals))
* [**Caveats**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Caveats)
* [**Disabling ASLR**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Disabling-ASLR)
* [**Non-deterministic proc macros**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Non-deterministic-proc-macros)
* [**Subtracting IRQs**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Subtracting-IRQs)
* [**Lack of support for multiple threads**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Lack-of-support-for-multiple-threads)
* [**Challenges**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Challenges)
* [**How do we even read hardware performance counters?**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#How-do-we-even-read-hardware-performance-counters)
* [**ASLR: it's free entropy**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#ASLR-it’s-free-entropy)
* [**The serializing instruction**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#The-serializing-instruction)
* [**Getting constantly interrupted**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Getting-constantly-interrupted)
* [**AMD patented time-travel and dubbed it `SpecLockMap`<br><sup> or: "how we accidentally unlocked `rr` on AMD Zen"</sup>**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#AMD-patented-time-travel-and-dubbed-it-SpecLockMapnbspnbspnbspnbspnbspnbspnbspnbspor-“how-we-accidentally-unlocked-rr-on-AMD-Zen”)
* [**`jemalloc`: purging will commence in ten seconds**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#jemalloc-purging-will-commence-in-ten-seconds)
* [**Rebasing *shouldn't* affect the results, right?**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Rebasing-*shouldn’t*-affect-the-results,-right)
* [**Epilogue: Zen's undocumented 420 counter**](https://hackmd.io/sH315lO2RuicY-SEt7ynGA?view#Epilogue-Zen’s-undocumented-420-counter)
Our condvar doesn't support setting attributes, like
pthread_condattr_setclock, which the current wait_timeout expects to
have configured.
Switch to a different implementation, following espidf.
Use `fcntl(fd, F_GETFD)` to detect if standard streams are open
In the previous implementation, if the standard streams were open,
but the RLIMIT_NOFILE value was below three, the poll would fail
with EINVAL:
> ERRORS: EINVAL The nfds value exceeds the RLIMIT_NOFILE value.
Switch to the existing fcntl based implementation to avoid the issue.
Fixes#96621.
std::io: Modify some ReadBuf method signatures to return `&mut Self`
This allows using `ReadBuf` in a builder-like style and to setup a `ReadBuf` and
pass it to `read_buf` in a single expression, e.g.,
```
// With this PR:
reader.read_buf(ReadBuf::uninit(buf).assume_init(init_len))?;
// Previously:
let mut buf = ReadBuf::uninit(buf);
buf.assume_init(init_len);
reader.read_buf(&mut buf)?;
```
r? `@sfackler`
cc https://github.com/rust-lang/rust/issues/78485, https://github.com/rust-lang/rust/issues/94741
`Mutex::lock()` and `RwLock::write()` are poison-guarded against panics,
in that they set the poison flag if a panic occurs while they're locked.
But if we're already in a panic (`thread::panicking()`), they leave the
poison flag alone.
That check is a bit of a waste for methods that never set the poison
flag though, namely `get_mut()`, `into_inner()`, and `RwLock::read()`.
These use-cases are now split to avoid that unnecessary call.
This allows to format into an `OsString` without unnecessary
allocations. E.g.
```
let mut temp_filename = path.into_os_string();
write!(&mut temp_filename, ".tmp.{}", process::id());
```
impl Read and Write for VecDeque<u8>
Implementing `Read` and `Write` for `VecDeque<u8>` fills in the VecDeque api surface where `Vec<u8>` and `Cursor<Vec<u8>>` already impl Read and Write. Not only for completeness, but VecDeque in particular is a very handy mock interface for a TCP echo service, if only it supported Read/Write.
Since this PR is just an impl trait, I don't think there is a way to limit it behind a feature flag, so it's "insta-stable". Please correct me if I'm wrong here, not trying to rush stability.
This commit adds a new unstable attribute, `#[doc(tuple_varadic)]`, that
shows a 1-tuple as `(T, ...)` instead of just `(T,)`, and links to a section
in the tuple primitive docs that talks about these.
In some situations it is possible for required functions to be called before they've had a chance to be loaded. Therefore, we make it possible to recover from this situation simply by looking at error codes.
Add documentation to `std::os::unix::io` describing Rust's stance on
`/proc/self/mem`, treating it as an external entity which is outside
the scope of Rust's safety guarantees.
Remove confusing sentence from `Mutex` docs
The docs were saying something about "statically initializing" the
mutex, and it's not clear what this means. Remove that part to avoid
confusion.
Remove migrate borrowck mode
Closes#58781Closes#43234
# Stabilization proposal
This PR proposes the stabilization of `#![feature(nll)]` and the removal of `-Z borrowck`. Current borrow checking behavior of item bodies is currently done by first infering regions *lexically* and reporting any errors during HIR type checking. If there *are* any errors, then MIR borrowck (NLL) never occurs. If there *aren't* any errors, then MIR borrowck happens and any errors there would be reported. This PR removes the lexical region check of item bodies entirely and only uses MIR borrowck. Because MIR borrowck could never *not* be run for a compiled program, this should not break any programs. It does, however, change diagnostics significantly and allows a slightly larger set of programs to compile.
Tracking issue: #43234
RFC: https://github.com/rust-lang/rfcs/blob/master/text/2094-nll.md
Version: 1.63 (2022-06-30 => beta, 2022-08-11 => stable).
## Motivation
Over time, the Rust borrow checker has become "smarter" and thus allowed more programs to compile. There have been three different implementations: AST borrowck, MIR borrowck, and polonius (well, in progress). Additionally, there is the "lexical region resolver", which (roughly) solves the constraints generated through HIR typeck. It is not a full borrow checker, but does emit some errors.
The AST borrowck was the original implementation of the borrow checker and was part of the initially stabilized Rust 1.0. In mid 2017, work began to implement the current MIR borrow checker and that effort ompleted by the end of 2017, for the most part. During 2018, efforts were made to migrate away from the AST borrow checker to the MIR borrow checker - eventually culminating into "migrate" mode - where HIR typeck with lexical region resolving following by MIR borrow checking - being active by default in the 2018 edition.
In early 2019, migrate mode was turned on by default in the 2015 edition as well, but with MIR borrowck errors emitted as warnings. By late 2019, these warnings were upgraded to full errors. This was followed by the complete removal of the AST borrow checker.
In the period since, various errors emitted by the MIR borrow checker have been improved to the point that they are mostly the same or better than those emitted by the lexical region resolver.
While there do remain some degradations in errors (tracked under the [NLL-diagnostics tag](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-diagnostics), those are sufficiently small and rare enough that increased flexibility of MIR borrow check-only is now a worthwhile tradeoff.
## What is stabilized
As said previously, this does not fundamentally change the landscape of accepted programs. However, there are a [few](https://github.com/rust-lang/rust/issues?q=is%3Aopen+is%3Aissue+label%3ANLL-fixed-by-NLL) cases where programs can compile under `feature(nll)`, but not otherwise.
There are two notable patterns that are "fixed" by this stabilization. First, the `scoped_threads` feature, which is a continutation of a pre-1.0 API, can sometimes emit a [weird lifetime error](https://github.com/rust-lang/rust/issues/95527) without NLL. Second, actually seen in the standard library. In the `Extend` impl for `HashMap`, there is an implied bound of `K: 'a` that is available with NLL on but not without - this is utilized in the impl.
As mentioned before, there are a large number of diagnostic differences. Most of them are better, but some are worse. None are serious or happen often enough to need to block this PR. The biggest change is the loss of error code for a number of lifetime errors in favor of more general "lifetime may not live long enough" error. While this may *seem* bad, the former error codes were just attempts to somewhat-arbitrarily bin together lifetime errors of the same type; however, on paper, they end up being roughly the same with roughly the same kinds of solutions.
## What isn't stabilized
This PR does not completely remove the lexical region resolver. In the future, it may be possible to remove that (while still keeping HIR typeck) or to remove it together with HIR typeck.
## Tests
Many test outputs get updated by this PR. However, there are number of tests specifically geared towards NLL under `src/test/ui/nll`
## History
* On 2017-07-14, [tracking issue opened](https://github.com/rust-lang/rust/issues/43234)
* On 2017-07-20, [initial empty MIR pass added](https://github.com/rust-lang/rust/pull/43271)
* On 2017-08-29, [RFC opened](https://github.com/rust-lang/rfcs/pull/2094)
* On 2017-11-16, [Integrate MIR type-checker with NLL](https://github.com/rust-lang/rust/pull/45825)
* On 2017-12-20, [NLL feature complete](https://github.com/rust-lang/rust/pull/46862)
* On 2018-07-07, [Don't run AST borrowck on mir mode](https://github.com/rust-lang/rust/pull/52083)
* On 2018-07-27, [Add migrate mode](https://github.com/rust-lang/rust/pull/52681)
* On 2019-04-22, [Enable migrate mode on 2015 edition](https://github.com/rust-lang/rust/pull/59114)
* On 2019-08-26, [Don't downgrade errors on 2015 edition](https://github.com/rust-lang/rust/pull/64221)
* On 2019-08-27, [Remove AST borrowck](https://github.com/rust-lang/rust/pull/64790)
Add note to documentation of HashSet::intersection
The functionality of the `std::collections::HashSet::intersection(...)` method was slightly surprising to me so I wanted to take a sec to contribute to the documentation for this method.
I've added a `Note:` section if that is appropriate.
Call the OS function to set the main thread's name on program init
Normally, `Thread::spawn` takes care of setting the thread's name, if
one was provided, but since the main thread wasn't created by calling
`Thread::spawn`, we need to call that function in `std::rt::init`.
This is mainly useful for system tools like debuggers and profilers
which might show the thread name to a user. Prior to these changes, gdb
and WinDbg would show all thread names except the main thread's name to
a user. I've validated that this patch resolves the issue for both
debuggers.
Lazily allocate and initialize pthread locks.
Lazily allocate and initialize pthread locks.
This allows {Mutex, Condvar, RwLock}::new() to be const, while still using the platform's native locks for features like priority inheritance and debug tooling. E.g. on macOS, we cannot directly use the (private) APIs that pthread's locks are implemented with, making it impossible for us to use anything other than pthread while still preserving priority inheritance, etc.
This PR doesn't yet make the public APIs const. That's for a separate PR with an FCP.
Tracking issue: https://github.com/rust-lang/rust/issues/93740
Tweak insert docs
For `{Hash, BTree}Map::insert`, I always have to take a few extra seconds to think about the slight weirdness about the fact that if we "did not" insert (which "sounds" false), we return true, and if we "did" insert, (which "sounds" true), we return false.
This tweaks the doc comments for the `insert` methods of those types (as well as what looks like a rustc internal data structure that I found just by searching the codebase for "If the set did") to first use the "Returns whether _something_" pattern used in e.g. `remove`, where we say that `remove` "returns whether the value was present".
Expose `get_many_mut` and `get_many_unchecked_mut` to HashMap
This pull-request expose the function [`get_many_mut`](https://docs.rs/hashbrown/0.12.0/hashbrown/struct.HashMap.html#method.get_many_mut) and [`get_many_unchecked_mut`](https://docs.rs/hashbrown/0.12.0/hashbrown/struct.HashMap.html#method.get_many_unchecked_mut) from `hashbrown` to the standard library `HashMap` type. They obviously keep the same API and are added under the (new) `map_many_mut` feature.
- `get_many_mut`: Attempts to get mutable references to `N` values in the map at once.
- `get_many_unchecked_mut`: Attempts to get mutable references to `N` values in the map at once, without validating that the values are unique.
library/std: Bump compiler_builtins
Some neat changes include faster float conversions & fixes for AVR 🙂
(note that's it's my first time upgrading `compiler_builtins`, so I'm not 100% sure if bumping `library/std/Cargo.toml` is enough; certainly seems to be so, though.)
Put a bound on collection misbehavior
As currently written, when a logic error occurs in a collection's trait parameters, this allows *completely arbitrary* misbehavior, so long as it does not cause undefined behavior in std. However, because the extent of misbehavior is not specified, it is allowed for *any* code in std to start misbehaving in arbitrary ways which are not formally UB; consider the theoretical example of a global which gets set on an observed logic error. Because the misbehavior is only bound by not resulting in UB from safe APIs and the crate-level encapsulation boundary of all of std, this makes writing user unsafe code that utilizes std theoretically impossible, as it now relies on undocumented QOI (quality of implementation) that unrelated parts of std cannot be caused to misbehave by a misuse of std::collections APIs.
In practice, this is a nonconcern, because std has reasonable QOI and an implementation that takes advantage of this freedom is essentially a malicious implementation and only compliant by the most langauage-lawyer reading of the documentation.
To close this hole, we just add a small clause to the existing logic error paragraph that ensures that any misbehavior is limited to the collection which observed the logic error, making it more plausible to prove the soundness of user unsafe code.
This is not meant to be formal; a formal refinement would likely need to mention that values derived from the collection can also misbehave after a logic error is observed, as well as define what it means to "observe" a logic error in the first place. This fix errs on the side of informality in order to close the hole without complicating a normal reading which can assume a reasonable nonmalicious QOI.
See also [discussion on IRLO][1].
[1]: https://internals.rust-lang.org/t/using-std-collections-and-unsafe-anything-can-happen/16640
r? rust-lang/libs-api ```@rustbot``` label +T-libs-api -T-libs
This technically adds a new guarantee to the documentation, though I argue as written it's one already implicitly provided.
Rollup of 6 pull requests
Successful merges:
- #97089 (Improve settings theme display)
- #97229 (Document the current aliasing rules for `Box<T>`.)
- #97371 (Suggest adding a semicolon to a closure without block)
- #97455 (Stabilize `toowned_clone_into`)
- #97565 (Add doc alias `memset` to `write_bytes`)
- #97569 (Remove `memset` alias from `fill_with`.)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Implement [OsStr]::join
Implements join for `OsStr` and `OsString` slices:
```Rust
let strings = [OsStr::new("hello"), OsStr::new("dear"), OsStr::new("world")];
assert_eq!("hello dear world", strings.join(OsStr::new(" ")));
````
This saves one from converting to strings and back, or from implementing it manually.
This PR has been re-filed after #96744 was first accidentally merged and then reverted.
The change is instantly stable and thus:
r? rust-lang/libs-api `@rustbot` label +T-libs-api -T-libs
cc `@thomcc` `@m-ou-se` `@faptc`
Remove "sys isn't exported yet" phrase
The oldest occurence is from 9e224c2bf1,
which is from the pre-1.0 days. In the years since then, std::sys still
hasn't been exported, and the last attempt was met with strong criticism:
https://github.com/rust-lang/rust/pull/97151
Thus, removing the "yet" part makes a lot of sense.
Use Box::new() instead of box syntax in library tests
The tests inside `library/*` have no reason to use `box` syntax as they have 0 performance relevance. Therefore, we can safely remove them (instead of having to use alternatives like the one in #97293).
The oldest occurence is from 9e224c2bf1,
which is from the pre-1.0 days. In the years since then, std::sys still
hasn't been exported, and the last attempt was met with strong criticism:
https://github.com/rust-lang/rust/pull/97151
Thus, removing the "yet" part makes a lot of sense.
Finish bumping stage0
It looks like the last time had left some remaining cfg's -- which made me think
that the stage0 bump was actually successful. This brings us to a released 1.62
beta though.
This now brings us to cfg-clean, with the exception of check-cfg-features in bootstrap;
I'd prefer to leave that for a separate PR at this time since it's likely to be more tricky.
cc https://github.com/rust-lang/rust/pull/97147#issuecomment-1132845061
r? `@pietroalbini`
Normally, `Thread::spawn` takes care of setting the thread's name, if
one was provided, but since the main thread wasn't created by calling
`Thread::spawn`, we need to call that function in `std::rt::init`.
This is mainly useful for system tools like debuggers and profilers
which might show the thread name to a user. Prior to these changes, gdb
and WinDbg would show all thread names except the main thread's name to
a user. I've validated that this patch resolves the issue for both
debuggers.
It looks like the last time had left some remaining cfg's -- which made me think
that the stage0 bump was actually successful. This brings us to a released 1.62
beta though.
Add section on common message styles for Result::expect
Based on a question from https://github.com/rust-lang/project-error-handling/issues/50#issuecomment-1092339937
~~One thing I haven't decided on yet, should I duplicate this section on `Option::expect`, link to this section, or move it somewhere else and link to that location from both docs?~~: I ended up moving the section to `std::error` and referencing it from both `Result::expect` and `Option::expect`'s docs.
I think this section, when combined with the similar update I made on [`std::panic!`](https://doc.rust-lang.org/nightly/std/macro.panic.html#when-to-use-panic-vs-result) implies that we should possibly more aggressively encourage and support the "expect as precondition" style described in this section. The consensus among the libs team seems to be that panic should be used for bugs, not expected potential failure modes. The "expect as error message" style seems to align better with the panic for unrecoverable errors style where they're seen as normal errors where the only difference is a desire to kill the current execution unit (aka erlang style error handling). I'm wondering if we should be providing a panic hook similar to `human-panic` or more strongly recommending the "expect as precondition" style of expect message.
explain how to turn integers into fn ptrs
(with an intermediate raw ptr, not a direct transmute)
Direct int2ptr transmute, under the semantics I am imagining, will produce a ptr with "invalid" provenance that is invalid to deref or call. We cannot give it the same semantics as int2ptr casts since those do [something complicated](https://www.ralfj.de/blog/2022/04/11/provenance-exposed.html).
To my great surprise, that is already what the example in the `transmute` docs does. :) I still added a comment to say that that part is important, and I added a section explicitly talking about this to the `fn()` type docs.
With https://github.com/rust-lang/miri/pull/2151, Miri will start complaining about direct int-to-fnptr transmutes (in the sense that it is UB to call the resulting pointer).
As currently written, when a logic error occurs in a collection's trait
parameters, this allows *completely arbitrary* misbehavior, so long as
it does not cause undefined behavior in std. However, because the extent
of misbehavior is not specified, it is allowed for *any* code in std to
start misbehaving in arbitrary ways which are not formally UB; consider
the theoretical example of a global which gets set on an observed logic
error. Because the misbehavior is only bound by not resulting in UB from
safe APIs and the crate-level encapsulation boundary of all of std, this
makes writing user unsafe code that utilizes std theoretically
impossible, as it now relies on undocumented QOI that unrelated parts of
std cannot be caused to misbehave by a misuse of std::collections APIs.
In practice, this is a nonconcern, because std has reasonable QOI and an
implementation that takes advantage of this freedom is essentially a
malicious implementation and only compliant by the most langauage-lawyer
reading of the documentation.
To close this hole, we just add a small clause to the existing logic
error paragraph that ensures that any misbehavior is limited to the
collection which observed the logic error, making it more plausible to
prove the soundness of user unsafe code.
This is not meant to be formal; a formal refinement would likely need to
mention that values derived from the collection can also misbehave after a
logic error is observed, as well as define what it means to "observe" a
logic error in the first place. This fix errs on the side of informality
in order to close the hole without complicating a normal reading which
can assume a reasonable nonmalicious QOI.
See also [discussion on IRLO][1].
[1]: https://internals.rust-lang.org/t/using-std-collections-and-unsafe-anything-can-happen/16640
Document rounding for floating-point primitive operations and string parsing
The docs for floating point don't have much to say at present about either the precision of their results or rounding behaviour.
As I understand it[^1][^2], Rust doesn't support operating with non-default rounding directions, so we need only describe roundTiesToEven.
[^1]: https://github.com/rust-lang/rust/issues/41753#issuecomment-299322887
[^2]: https://github.com/llvm/llvm-project/issues/8472#issuecomment-980888781
This PR makes a start by documenting that for primitive operations and `from_str()`.
Use const initializer for LOCAL_PANIC_COUNT
This reduces the size of the __getit function for LOCAL_PANIC_COUNT and should speed up accesses of LOCAL_PANIC_COUNT a bit.
Make write/print macros eagerly drop temporaries
This PR fixes the 2 regressions in #96434 (`println` and `eprintln`) and changes all the other similar macros (`write`, `writeln`, `print`, `eprint`) to match the old pre-#94868 behavior of `println` and `eprintln`.
argument position | before #94868 | after #94868 | after this PR
--- |:---:|:---:|:---:
`write!($tmp, "…", …)` | 😡 | 😡 | 😺
`write!(…, "…", $tmp)` | 😡 | 😡 | 😺
`writeln!($tmp, "…", …)` | 😡 | 😡 | 😺
`writeln!(…, "…", $tmp)` | 😡 | 😡 | 😺
`print!("…", $tmp)` | 😡 | 😡 | 😺
`println!("…", $tmp)` | 😺 | 😡 | 😺
`eprint!("…", $tmp)` | 😡 | 😡 | 😺
`eprintln!("…", $tmp)` | 😺 | 😡 | 😺
`panic!("…", $tmp)` | 😺 | 😺 | 😺
Example of code that is affected by this change:
```rust
use std::sync::Mutex;
fn main() {
let mutex = Mutex::new(0);
print!("{}", mutex.lock().unwrap()) /* no semicolon */
}
```
You can see several real-world examples like this in the Crater links at the top of #96434. This code failed to compile prior to this PR as follows, but works after this PR.
```console
error[E0597]: `mutex` does not live long enough
--> src/main.rs:5:18
|
5 | print!("{}", mutex.lock().unwrap()) /* no semicolon */
| ^^^^^^^^^^^^---------
| |
| borrowed value does not live long enough
| a temporary with access to the borrow is created here ...
6 | }
| -
| |
| `mutex` dropped here while still borrowed
| ... and the borrow might be used here, when that temporary is dropped and runs the `Drop` code for type `MutexGuard`
```
Stabilize `Ipv6Addr::to_ipv4_mapped`
CC https://github.com/rust-lang/rust/issues/27709 (tracking issue for the `ip` feature which contains more
functions)
The function `Ipv6Addr::to_ipv4` is bad because it also returns an IPv4
address for the IPv6 loopback address `::1`. Stabilize
`Ipv6Addr::to_ipv4_mapped` so we can recommend that function instead.
Fix typo in futex RwLock::write_contended.
I wrote `state` where I should've used `s`.
This was spotted by `@Warrenren.`
This change removes the unnecessary `s` variable to prevent that mistake.
Fortunately, this typo didn't affect the correctness of the lock, as the
second half of the condition (!has_writers_waiting) is enough for
correctness, which explains why this mistake didn't show up during
testing.
Fixes https://github.com/rust-lang/rust/issues/97162
Fix rusty grammar in `std::error::Reporter` docs
### Commit
I initially saw "print's" instead of "prints" at the start of the doc comment for `std::error::Reporter`, while reading the docs for that type. Then I figured 'probably more where that came from', so, as well as correcting the foregoing to "prints", I've patched up these three minor solecisms (well, two [types](https://en.wikipedia.org/wiki/Type%E2%80%93token_distinction), three [tokens](https://en.wikipedia.org/wiki/Type%E2%80%93token_distinction)):
- One use of the indicative which should be subjunctive - indeed the sentence immediately following it, which mirrors its structure, _does_ use the subjunctive ([L871](https://github.com/rust-lang/rust/blob/master/library/std/src/error.rs?plain=1#L871)). Replaced with the subjunctive.
- Two separate clauses joined with commas ([L975](https://github.com/rust-lang/rust/blob/master/library/std/src/error.rs?plain=1#L975), [L1023](https://github.com/rust-lang/rust/blob/master/library/std/src/error.rs?plain=1#L1023)). Replaced the first with a semicolon and the second with a period. Admittedly those judgements are pretty much 100% subjective, based on my sense of how the sentences flowed into each other (though ofc the _replacement of the comma itself_ is not subjective or opinion-based).
I know this is silly and finicky, but I hope it helps tidy up the docs a bit for future readers!
### PR notes
**This is very much non-urgent (and, honestly, non-important).** I just figured it might be a nice quality-of-life improvement and bit of tidying up for the core contributors themselves not to have to do. 🙂
I'm tagging Steve, per the [contributing guidelines](https://rustc-dev-guide.rust-lang.org/contributing.html#r) ("Steve usually reviews documentation changes. So if you were to make a documentation change, add `r? `@steveklabnik`"):`
r? `@steveklabnik`
I wrote `state` where I should've used `s`.
This removes the unnecessary `s` variable to prevent that mistake.
Fortunately, this typo didn't affect the correctness of the lock, as the
second half of the condition (!has_writers_waiting) is enough for
correctness, which explains why this mistake didn't show up during
testing.
From reading the source code, it appears like the desired semantic of
std::unix::rand is to always provide some bytes and never block. For
that reason GRND_NONBLOCK is checked before calling getrandom(0), so
that getrandom(0) won't block. If it would block, then the function
falls back to using /dev/urandom, which for the time being doesn't
block. There are some drawbacks to using /dev/urandom, however, and so
getrandom(GRND_INSECURE) was created as a replacement for this exact
circumstance.
getrandom(GRND_INSECURE) is the same as /dev/urandom, except:
- It won't leave a warning in dmesg if used at early boot time, which is
a common occurance (and the reason why I found this issue);
- It won't introduce a tiny delay at early boot on newer kernels when
/dev/urandom tries to opportunistically create jitter entropy;
- It only requires 1 syscall, rather than 3.
Other than that, it returns the same "quality" of randomness as
/dev/urandom, and never blocks.
It's only available on kernels ≥5.6, so we try to use it, cache the
result of that attempt, and fall back to to the previous code if it
didn't work.
It is not obvious (at least for me) that complexity of iteration over hash tables depends on capacity and not length. Especially comparing with other containers like Vec or String. I think, this behaviour is worth mentioning.
I run benchmark which tests iteration time for maps with length 50 and different capacities and get this results:
```
capacity - time
64 - 203.87 ns
256 - 351.78 ns
1024 - 607.87 ns
4096 - 965.82 ns
16384 - 3.1188 us
```
If you want to dig why it behaves such way, you can look current implementation in [hashbrown code](f3a9f211d0/src/raw/mod.rs (L1933)).
Benchmarks code would be presented in PR related to this commit.
* For read and read_buf, only the front slice of a discontiguous
VecDeque is copied. The VecDeque is advanced after reading, making any
back slice available for reading with a second call to Read::read(_buf).
* For write, the VecDeque always appends the entire slice to the end,
growing its allocation when necessary.
Remove libstd's calls to `C-unwind` foreign functions
Remove all libstd and its dependencies' usage of `extern "C-unwind"`.
This is a prerequiste of a WIP PR which will forbid libraries calling `extern "C-unwind"` functions to be compiled in `-Cpanic=unwind` and linked against `panic_abort` (this restriction is necessary to address soundness bug #96926).
Cargo will ensure all crates are compiled with the same `-Cpanic` but the std is only compiled `-Cpanic=unwind` but needs the ability to be linked into `-Cpanic=abort`.
Currently there are two places where `C-unwind` is used in libstd:
* `__rust_start_panic` is used for interfacing to the panic runtime. This could be `extern "Rust"`
* `_{rdl,rg}_oom`: a shim `__rust_alloc_error_handler` will be generated by codegen to call into one of these; they can also be `extern "Rust"` (in fact, the generated shim is used as `extern "Rust"`, so I am not even sure why these are not, probably because they used to `extern "C"` and was changed to `extern "C-unwind"` when we allow alloc error hooks to unwind, but they really should just be using Rust ABI).
For dependencies, there is only one `extern "C-unwind"` function call, in `unwind` crate. This can be expressed as a re-export.
More dicussions can be seen in the Zulip thread: https://rust-lang.zulipchat.com/#narrow/stream/210922-project-ffi-unwind/topic/soundness.20in.20mixed.20panic.20mode
`@rustbot` label: T-libs F-c_unwind
Make HashMap fall back to RtlGenRandom if BCryptGenRandom fails
With PR #84096, Rust `std::collections::hash_map::RandomState` changed from using `RtlGenRandom()` ([msdn](https://docs.microsoft.com/en-us/windows/win32/api/ntsecapi/nf-ntsecapi-rtlgenrandom)) to `BCryptGenRandom()` ([msdn](https://docs.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom)) as its source of secure randomness after much discussion ([here](https://github.com/rust-random/getrandom/issues/65#issuecomment-753634074), among other places).
Unfortunately, after that PR landed, Mozilla Firefox started experiencing fairly-rare crashes during startup while attempting to initialize the `env_logger` crate. ([docs for env_logger](https://docs.rs/env_logger/latest/env_logger/)) The root issue is that on some machines, `BCryptGenRandom()` will fail with an `Access is denied. (os error 5)` error message. ([Bugzilla issue 1754490](https://bugzilla.mozilla.org/show_bug.cgi?id=1754490)) (Discussion in issue #94098)
Note that this is happening upon startup of Firefox's unsandboxed Main Process, so this behavior is different and separate from previous issues ([like this](https://bugzilla.mozilla.org/show_bug.cgi?id=1746254)) where BCrypt DLLs were blocked by process sandboxing. In the case of sandboxing, we knew we were doing something abnormal and expected that we'd have to resort to abnormal measures to make it work.
However, in this case we are in a regular unsandboxed process just trying to initialize `env_logger` and getting a panic. We suspect that this may be caused by a virus scanner or some other security software blocking the loading of the BCrypt DLLs, but we're not completely sure as we haven't been able to replicate locally.
It is also possible that Firefox is not the only software affected by this; we just may be one of the pieces of Rust software that has the telemetry and crash reporting necessary to catch it.
I have read some of the historical discussion around using `BCryptGenRandom()` in Rust code, and I respect the decision that was made and agree that it was a good course of action, so I'm not trying to open a discussion about a return to `RtlGenRandom()`. Instead, I'd like to suggest that perhaps we use `RtlGenRandom()` as a "fallback RNG" in the case that BCrypt doesn't work.
This pull request implements this fallback behavior. I believe this would improve the robustness of this essential data structure within the standard library, and I see only 2 potential drawbacks:
1. Slight added overhead: It should be quite minimal though. The first call to `sys::rand::hashmap_random_keys()` will incur a bit of initialization overhead, and every call after will incur roughly 2 non-atomic global reads and 2 easily predictable branches. Both should be negligible compared to the actual cost of generating secure random numbers
2. `RtlGenRandom()` is deprecated by Microsoft: Technically true, but as mentioned in [this comment on GoLang](https://github.com/golang/go/issues/33542#issuecomment-626124873), this API is ubiquitous in Windows software and actually removing it would break lots of things. Also, Firefox uses it already in [our C++ code](https://searchfox.org/mozilla-central/rev/5f88c1d6977e03e22d3420d0cdf8ad0113c2eb31/mfbt/RandomNum.cpp#25), and [Chromium uses it in their code as well](https://source.chromium.org/chromium/chromium/src/+/main:base/rand_util_win.cc) (which transitively means that Microsoft uses it in their own web browser, Edge). If there did come a time when Microsoft truly removes this API, it should be easy enough for Rust to simply remove the fallback in the code I've added here
Fix use of SetHandleInformation on UWP
The use of `SetHandleInformation` (introduced in #96441 to make `HANDLE` inheritable) breaks UWP builds because it is not available for UWP targets.
Proposed workaround: duplicate the `HANDLE` with `inherit = true` and immediately close the old one. Traditional Windows Desktop programs are not affected.
cc `@ChrisDenton`
Add rustc_nonnull_optimization_guaranteed to Owned/Borrowed Fd/Socket
PR #94586 added support for using
`rustc_nonnull_optimization_guaranteed` on values where the "null" value
is the all-ones bitpattern.
Now that #94586 has made it to the stage0 compiler, add
`rustc_nonnull_optimization_guaranteed` to `OwnedFd`, `BorrowedFd`,
`OwnedSocket`, and `BorrowedSocket`, since these types all exclude
all-ones bitpatterns.
This allows `Option<OwnedFd>`, `Option<BorrowedFd>`, `Option<OwnedSocket>`,
and `Option<BorrowedSocket>` to be used in FFI declarations, as described
in the [I/O safety RFC].
[I/O safety RFC]: https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md#ownedfd-and-borrowedfdfd-1
ExitCode::exit_process() method
cc `@yaahc` / #93840
(eeek, hit ctrl-enter before I meant to and right after realizing the branch name was wrong. oh, well)
I feel like it makes sense to have the `exit(ExitCode)` function as a method or at least associated function on ExitCode, but maybe that would hurt discoverability? Probably not as much if it's at the top of the `process::exit()` documentation or something, but idk. Also very unsure about the name, I'd like something that communicates that you are exiting with *this* ExitCode, but with a method name being postfix it doesn't seem to flow. `code.exit_process_with()` ? `.exit_process_with_self()` ? Blech. Maybe it doesn't matter, since ideally just `code.exit()` or something would be clear simply by the name and single parameter but 🤷
Also I'd like to touch up the `ExitCode` docs (which I did a bit here), but that would probably be good in a separate PR, right? Since I think the beta deadline is coming up.
Clarify what values `BorrowedHandle`, `OwnedHandle` etc. can hold.
Reword the documentation to clarify that when `BorrowedHandle`, `OwnedHandle`, or `HandleOrNull` hold the value `-1`, it always means the current process handle, and not `INVALID_HANDLE_VALUE`.
`-1` should only mean `INVALID_HANDLE_VALUE` after a call to a function documented to return that to report errors, which should lead I/O functions to produce errors rather than succeeding and producing `OwnedHandle` or `BorrowedHandle` values. So if a consumer of an `OwnedHandle` or `BorrowedHandle` ever sees them holding a `-1`, it should always mean the current process handle.
PR #94586 added support for using
`rustc_nonnull_optimization_guaranteed` on values where the "null" value
is the all-ones bitpattern.
Now that #94586 has made it to the stage0 compiler, add
`rustc_nonnull_optimization_guaranteed` to `OwnedFd`, `BorrowedFd`,
`OwnedSocket`, and `BorrowedSocket`, since these types all exclude
all-ones bitpatterns.
This allows `Option<OwnedFd>`, `Option<BorrowedFd>`, `Option<OwnedSocket>`,
and `Option<BorrowedSocket>` to be used in FFI declarations, as described
in the [I/O safety RFC].
[I/O safety RFC]: https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md#ownedfd-and-borrowedfdfd-1
In the previous implementation, if the standard streams were open,
but the RLIMIT_NOFILE value was below three, the poll would fail
with EINVAL:
> ERRORS: EINVAL The nfds value exceeds the RLIMIT_NOFILE value.
Switch to the existing fcntl based implementation to avoid the issue.
Clarify that when `BorrowedHandle`, `OwnedHandle`, or `HandleOrNull`
hold the value `-1`, it always means the current process handle, and not
`INVALID_HANDLE_VALUE`.
Make `BorrowedFd::borrow_raw` a const fn.
Making `BorrowedFd::borrow_raw` a const fn allows it to be used to
create a constant `BorrowedFd<'static>` holding constants such as
`AT_FDCWD`. This will allow [`rustix::fs::cwd`] to become a const fn.
For consistency, make similar changes to `BorrowedHandle::borrow_raw`
and `BorrowedSocket::borrow_raw`.
[`rustix::fs::cwd`]: https://docs.rs/rustix/latest/rustix/fs/fn.cwd.html
r? `@joshtriplett`
CC #27709 (tracking issue for the `ip` feature which contains more
functions)
The function `Ipv6Addr::to_ipv4` is bad because it also returns an IPv4
address for the IPv6 loopback address `::1`. Stabilize
`Ipv6Addr::to_ipv4_mapped` so we can recommend that function instead.
Issue #84096 changed the hashmap RNG to use BCryptGenRandom instead of
RtlGenRandom on Windows.
Mozilla Firefox started experiencing random failures in
env_logger::Builder::new() (Issue #94098) during initialization of their
unsandboxed main process with an "Access Denied" error message from
BCryptGenRandom(), which is used by the HashMap contained in
env_logger::Builder
The root cause appears to be a virus scanner or other software interfering
with BCrypt DLLs loading.
This change adds a fallback option if BCryptGenRandom is unusable for
whatever reason. It will fallback to RtlGenRandom in this case.
Fixes#94098
Revert "Implement [OsStr]::join", which was merged without FCP.
This reverts commit 4fcbc53820, see https://github.com/rust-lang/rust/pull/96744. (I'm terribly sorry, and truly don't remember r+ing it, or even having seen it before yesterday, which is... genuinely very worrisome for me).
r? `@m-ou-se`
Improve floating point documentation
This is my attempt to improve/solve https://github.com/rust-lang/rust/issues/95468 and https://github.com/rust-lang/rust/issues/73328 .
Added/refined explanations:
- Refine the "NaN as a special value" top level explanation of f32
- Refine `const NAN` docstring: add an explanation about there being multitude of NaN bitpatterns and disclaimer about the portability/stability guarantees.
- Refine `fn is_sign_positive` and `fn is_sign_negative` docstrings: add disclaimer about the sign bit of NaNs.
- Refine `fn min` and `fn max` docstrings: explain the semantics and their relationship to the standard and libm better.
- Refine `fn trunc` docstrings: explain the semantics slightly more.
- Refine `fn powi` docstrings: add disclaimer that the rounding behaviour might be different from `powf`.
- Refine `fn copysign` docstrings: add disclaimer about payloads of NaNs.
- Refine `minimum` and `maximum`: add disclaimer that "propagating NaN" doesn't mean that propagating the NaN bit patterns is guaranteed.
- Refine `max` and `min` docstrings: add "ignoring NaN" to bring the one-row explanation to parity with `minimum` and `maximum`.
Cosmetic changes:
- Reword `NaN` and `NAN` as plain "NaN", unless they refer to the specific `const NAN`.
- Reword "a number" to `self` in function docstrings to clarify.
- Remove "Returns NAN if the number is NAN" from `abs`, as this is told to be the default behavior in the top explanation.
Remove `#[rustc_deprecated]`
This removes `#[rustc_deprecated]` and introduces diagnostics to help users to the right direction (that being `#[deprecated]`). All uses of `#[rustc_deprecated]` have been converted. CI is expected to fail initially; this requires #95958, which includes converting `stdarch`.
I plan on following up in a short while (maybe a bootstrap cycle?) removing the diagnostics, as they're only intended to be short-term.
Add more diagnostic items
This just adds a handful diagnostic items I noticed were missing.
Would it be worth doing this for all of the remaining types? I'm willing to do it if it'd be helpful.
Create clippy lint against unexpectedly late drop for temporaries in match scrutinee expressions
A new clippy lint for issue 93883 (https://github.com/rust-lang/rust/issues/93883). Relies on a new trait in `marker` (called `SignificantDrop` to enable linting), which is why this PR is for the rust-lang repo and not the clippy repo.
changelog: new lint [`significant_drop_in_scrutinee`]
Remove hard links from `env::current_exe` security example
The security example shows that `env::current_exe` will return the path used when the program was started. This is not really surprising considering how hard links work: after `ln foo bar`, the two files are _equivalent_. It is _not_ the case that `bar` is a “link” to `foo`, nor is `foo` a link to `bar`. They are simply two names for the same underlying data.
The security vulnerability linked to seems to be different: there an attacker would start a SUID binary from a directory under the control of the attacker. The binary would respawn itself by executing the program found at `/proc/self/exe` (which the attacker can control). This is a real problem. In my opinion, the example given here doesn’t really show the same problem, it just shows a misunderstanding of what hard links are.
I looked through the history a bit and found that the example was introduced in https://github.com/rust-lang/rust/pull/33526. That PR actually has two commits, and the first (8478d48dad) explains the race condition at the root of the linked security vulnerability. The second commit proceeds to replace the explanation with the example we have today.
This commit reverts most of the second commit from https://github.com/rust-lang/rust/pull/33526.
Add aliases for std::fs::canonicalize
The aliases are `realpath` and `GetFinalPathNameByHandle` which are explicitly mentioned in `canonicalize`'s documentation.
Use 64-bit time on 32-bit linux-gnu
The standard library suffered the [Year 2038 problem][Y2038] in two main places on targets with 32-bit `time_t`:
- In `std::time::SystemTime`, we stored a `timespec` that has `time_t` seconds. This is now changed to directly store 64-bit seconds and nanoseconds, and on 32-bit linux-gnu we try to use `__clock_gettime64` (glibc 2.34+) to get the larger timestamp.
- In `std::fs::Metadata`, we store a `stat64`, which has 64-bit `off_t` but still 32-bit `time_t`, and unfortunately that is baked in the API by the (deprecated) `MetadataExt::as_raw_stat()`. However, we can use `statx` for 64-bit `statx_timestamp` to store in addition to the `stat64`, as we already do to support creation time, and the rest of the `MetadataExt` methods can return those full values. Note that some filesystems may still be limited in their actual timestamp support, but that's not something Rust can change.
There remain a few places that need `timespec` for system call timeouts -- I leave that to future work.
[Y2038]: https://en.wikipedia.org/wiki/Year_2038_problem
Add a dedicated length-prefixing method to `Hasher`
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths
This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
Fixes#94026
r? rust-lang/libs
---
The core of this change is the following two new methods on `Hasher`:
```rust
pub trait Hasher {
/// Writes a length prefix into this hasher, as part of being prefix-free.
///
/// If you're implementing [`Hash`] for a custom collection, call this before
/// writing its contents to this `Hasher`. That way
/// `(collection![1, 2, 3], collection![4, 5])` and
/// `(collection![1, 2], collection![3, 4, 5])` will provide different
/// sequences of values to the `Hasher`
///
/// The `impl<T> Hash for [T]` includes a call to this method, so if you're
/// hashing a slice (or array or vector) via its `Hash::hash` method,
/// you should **not** call this yourself.
///
/// This method is only for providing domain separation. If you want to
/// hash a `usize` that represents part of the *data*, then it's important
/// that you pass it to [`Hasher::write_usize`] instead of to this method.
///
/// # Examples
///
/// ```
/// #![feature(hasher_prefixfree_extras)]
/// # // Stubs to make the `impl` below pass the compiler
/// # struct MyCollection<T>(Option<T>);
/// # impl<T> MyCollection<T> {
/// # fn len(&self) -> usize { todo!() }
/// # }
/// # impl<'a, T> IntoIterator for &'a MyCollection<T> {
/// # type Item = T;
/// # type IntoIter = std::iter::Empty<T>;
/// # fn into_iter(self) -> Self::IntoIter { todo!() }
/// # }
///
/// use std:#️⃣:{Hash, Hasher};
/// impl<T: Hash> Hash for MyCollection<T> {
/// fn hash<H: Hasher>(&self, state: &mut H) {
/// state.write_length_prefix(self.len());
/// for elt in self {
/// elt.hash(state);
/// }
/// }
/// }
/// ```
///
/// # Note to Implementers
///
/// If you've decided that your `Hasher` is willing to be susceptible to
/// Hash-DoS attacks, then you might consider skipping hashing some or all
/// of the `len` provided in the name of increased performance.
#[inline]
#[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
fn write_length_prefix(&mut self, len: usize) {
self.write_usize(len);
}
/// Writes a single `str` into this hasher.
///
/// If you're implementing [`Hash`], you generally do not need to call this,
/// as the `impl Hash for str` does, so you can just use that.
///
/// This includes the domain separator for prefix-freedom, so you should
/// **not** call `Self::write_length_prefix` before calling this.
///
/// # Note to Implementers
///
/// The default implementation of this method includes a call to
/// [`Self::write_length_prefix`], so if your implementation of `Hasher`
/// doesn't care about prefix-freedom and you've thus overridden
/// that method to do nothing, there's no need to override this one.
///
/// This method is available to be overridden separately from the others
/// as `str` being UTF-8 means that it never contains `0xFF` bytes, which
/// can be used to provide prefix-freedom cheaper than hashing a length.
///
/// For example, if your `Hasher` works byte-by-byte (perhaps by accumulating
/// them into a buffer), then you can hash the bytes of the `str` followed
/// by a single `0xFF` byte.
///
/// If your `Hasher` works in chunks, you can also do this by being careful
/// about how you pad partial chunks. If the chunks are padded with `0x00`
/// bytes then just hashing an extra `0xFF` byte doesn't necessarily
/// provide prefix-freedom, as `"ab"` and `"ab\u{0}"` would likely hash
/// the same sequence of chunks. But if you pad with `0xFF` bytes instead,
/// ensuring at least one padding byte, then it can often provide
/// prefix-freedom cheaper than hashing the length would.
#[inline]
#[unstable(feature = "hasher_prefixfree_extras", issue = "88888888")]
fn write_str(&mut self, s: &str) {
self.write_length_prefix(s.len());
self.write(s.as_bytes());
}
}
```
With updates to the `Hash` implementations for slices and containers to call `write_length_prefix` instead of `write_usize`.
`write_str` defaults to using `write_length_prefix` since, as was pointed out in the issue, the `write_u8(0xFF)` approach is insufficient for hashers that work in chunks, as those would hash `"a\u{0}"` and `"a"` to the same thing. But since `SipHash` works byte-wise (there's an internal buffer to accumulate bytes until a full chunk is available) it overrides `write_str` to continue to use the add-non-UTF-8-byte approach.
---
Compatibility:
Because the default implementation of `write_length_prefix` calls `write_usize`, the changed hash implementation for slices will do the same thing the old one did on existing `Hasher`s.
Use futex-based locks and thread parker on {Free, Open, DragonFly}BSD.
This switches *BSD to our futex-based locks and thread parker.
Tracking issue: https://github.com/rust-lang/rust/issues/93740
This is a draft, because this still needs a new version of the `libc` crate to be published that includes https://github.com/rust-lang/libc/pull/2770.
r? `@Amanieu`
This accomplishes two main goals:
- Make it clear who is responsible for prefix-freedom, including how they should do it
- Make it feasible for a `Hasher` that *doesn't* care about Hash-DoS resistance to get better performance by not hashing lengths
This does not change rustc-hash, since that's in an external crate, but that could potentially use it in future.
Implement [OsStr]::join
Implements join for `OsStr` and `OsString` slices:
```Rust
let strings = [OsStr::new("hello"), OsStr::new("dear"), OsStr::new("world")];
assert_eq!("hello dear world", strings.join(OsStr::new(" ")));
````
This saves one from converting to strings and back, or from implementing it manually.
Relax memory ordering used in SameMutexCheck
`SameMutexCheck` only requires atomicity for `self.addr`, but does not need ordering of other memory accesses in either the success or failure case. Using `Relaxed`, the code still correctly handles the case when two threads race to store an address.
Relax memory ordering used in `min_stack`
`min_stack` does not provide any synchronization guarantees to its callers, and only requires atomicity for `MIN` itself, so relaxed memory ordering is sufficient.
This allows using `ReadBuf` in a builder-like style and to setup a `ReadBuf` and
pass it to `read_buf` in a single expression, e.g.,
```
// With this PR:
reader.read_buf(ReadBuf::uninit(buf).assume_init(init_len))?;
// Previously:
let mut buf = ReadBuf::uninit(buf);
buf.assume_init(init_len);
reader.read_buf(&mut buf)?;
```
Signed-off-by: Nick Cameron <nrc@ncameron.org>
The security example shows that `env::current_exe` will return the
path used when the program was started. This is not really surprising
considering how hard links work: after `ln foo bar`, the two files are
_equivalent_. It is _not_ the case that `bar` is a “link” to `foo`,
nor is `foo` a link to `bar`. They are simply two names for the same
underlying data.
The security vulnerability linked to seems to be different: there an
attacker would start a SUID binary from a directory under the control
of the attacker. The binary would respawn itself by executing the
program found at `/proc/self/exe` (which the attacker can control).
This is a real problem. In my opinion, the example given here doesn’t
really show the same problem, it just shows a misunderstanding of what
hard links are.
I looked through the history a bit and found that the example was
introduced in #33526. That PR actually has two commits, and the
first (8478d48dad) explains the race
condition at the root of the linked security vulnerability. The second
commit proceeds to replace the explanation with the example we have
today.
This commit reverts most of the second commit from #33526.
`SameMutexCheck` only requires atomicity for `self.addr`, but does not need ordering of other memory accesses in either the success or failure case. Using `Relaxed`, the code still correctly handles the case when two threads race to store an address.
`min_stack` does not provide any synchronization guarantees to its callers, and only requires atomicity for `MIN` itself, so relaxed memory ordering is sufficient.
rustdoc: Resolve doc links referring to `macro_rules` items
cc https://github.com/rust-lang/rust/issues/81633
UPD: the fallback to considering *all* `macro_rules` in the crate for unresolved names is not removed in this PR, it will be removed separately and will be run through crater.
Make [e]println macros eagerly drop temporaries (for backport)
This PR extracts the subset of #96455 which is only the parts necessary for fixing the 1.61-beta regressions in #96434.
My larger PR #96455 contains a few other changes relative to the pre-#94868 behavior; those are not necessary to backport into 1.61.
argument position | before #94868 | after #94868 | after this PR
--- |:---:|:---:|:---:
`write!($tmp, "…", …)` | 😡 | 😡 | 😡
`write!(…, "…", $tmp)` | 😡 | 😡 | 😡
`writeln!($tmp, "…", …)` | 😡 | 😡 | 😡
`writeln!(…, "…", $tmp)` | 😡 | 😡 | 😡
`print!("…", $tmp)` | 😡 | 😡 | 😡
`println!("…", $tmp)` | 😺 | 😡 | 😺
`eprint!("…", $tmp)` | 😡 | 😡 | 😡
`eprintln!("…", $tmp)` | 😺 | 😡 | 😺
`panic!("…", $tmp)` | 😺 | 😺 | 😺
Revert "Re-export core::ffi types from std::ffi"
This reverts commit 9aed829fe6.
Fixes https://github.com/rust-lang/rust/issues/96435 , a regression
in crates doing `use std::ffi::*;` and `use std::os::raw::*;`.
We can re-add this re-export once the `core::ffi` types
are stable, and thus the `std::os::raw` types can become re-exports as
well, which will avoid the conflict. (Type aliases to the same type
still conflict, but re-exports of the same type don't.)
Windows: Make stdin pipes synchronous
Stdin pipes do not need to be used asynchronously within the standard library. This is a first step in making pipes mostly synchronous.
r? `@m-ou-se`
std: directly use pthread in UNIX parker implementation
`Mutex` and `Condvar` are being replaced by more efficient implementations, which need thread parking themselves (see #93740). Therefore we should use the `pthread` synchronization primitives directly. Also, we can avoid allocating the mutex and condition variable because the `Parker` struct is being placed in an `Arc` anyways.
This basically is just a copy of the current `Mutex` and `Condvar` code, which will however be removed (again, see #93740). An alternative implementation could be to use dedicated private `OsMutex` and `OsCondvar` types, but all the other platforms supported by std actually have their own thread parking primitives.
I used `Pin` to guarantee a stable address for the `Parker` struct, while the current implementation does not, rather using extra unsafe declaration. Since the thread struct is shared anyways, I assumed this would not add too much clutter while being clearer.
Make EncodeWide implement FusedIterator
[`EncodeUtf16`](https://doc.rust-lang.org/std/str/struct.EncodeUtf16.html) and [`EncodeWide`](https://doc.rust-lang.org/std/os/windows/ffi/struct.EncodeWide.html) currently serve similar purposes: They convert from UTF-8 to UTF-16 and WTF-8 to WTF-16, respectively. `EncodeUtf16` wraps a &str, whereas `EncodeWide` wraps an &OsStr.
When Iteration has concluded, these iterators wrap an empty slice, which will forever yield `None` values. Hence, `EncodeUtf16` rightfully implements `FusedIterator`. However, `EncodeWide` in contrast does not, even though it serves an almost identical purpose.
This PR attempts to fix that issue. I consider this change minor and non-controversial, hence why I have not added a RFC/FCP. Please let me know if the stability attribute is wrong or contains a wrong version number. Thanks in advance.
Fixes https://github.com/rust-lang/rust/issues/96368
This reverts commit 9aed829fe6.
Fixes https://github.com/rust-lang/rust/issues/96435 , a regression
in crates doing `use std::ffi::*;` and `use std::os::raw::*;`.
We can re-add this re-export once the `core::ffi` types
are stable, and thus the `std::os::raw` types can become re-exports as
well, which will avoid the conflict. (Type aliases to the same type
still conflict, but re-exports of the same type don't.)
Define a dedicated error type for `HandleOrNull` and `HandleOrInvalid`.
Define `NullHandleError` and `InvalidHandleError` types, that implement std::error::Error, and use them as the error types in `HandleOrNull` and `HandleOrInvalid`,
This addresses [this concern](https://github.com/rust-lang/rust/issues/87074#issuecomment-1080031167).
This is the same as #95387.
r? `@joshtriplett`
Mutex and Condvar are being replaced by more efficient implementations, which need thread parking themselves (see #93740). Therefore use the pthread synchronization primitives directly. Also, avoid allocating because the Parker struct is being placed in an Arc anyways.
Windows Command: Don't run batch files using verbatim paths
Fixes#95178
Note that the first commit does some minor refactoring (moving command line argument building to args.rs). The actual changes are in the second.
Reduce allocations for path conversions on Windows
Previously, UTF-8 to UTF-16 Path conversions on Windows unnecessarily allocate twice, as described in #96297. This commit fixes that issue.
Improve Windows path prefix parsing
This PR fixes improves parsing of Windows path prefixes. `parse_prefix` now supports both types of separators on Windows (`/` and `\`).
[fuchsia] Add implementation for `current_exe`
This implementation returns a best attempt at the current exe path. On
fuchsia, fdio will always use `argv[0]` as the process name and if it is
not set then an error will be returned. Because this is not guaranteed
to be the case, this implementation returns an error if `argv` does not
contain any elements.
remove_dir_all_recursive: treat ELOOP the same as ENOTDIR
On older Linux kernels (I tested on 4.4, corresponding to Ubuntu 16.04), opening a symlink using `O_DIRECTORY | O_NOFOLLOW` returns `ELOOP` instead of `ENOTDIR`. We should handle it the same, since a symlink is still not a directory and needs to be `unlink`ed.
Use sys::unix::locks::futex* on wasm+atomics.
This removes the wasm-specific lock implementations and instead re-uses the implementations from sys::unix.
Tracking issue: https://github.com/rust-lang/rust/issues/93740
cc ``@alexcrichton``
Improve AddrParseError description
The existing description was incorrect for socket addresses, and misleading: users would see “invalid IP address syntax” and suppose they were supposed to provide an IP address rather than a socket address.
I contemplated making it two variants (IP, socket), but realised we can do still better for the IPv4 and IPv6 types, so here it is as six.
I contemplated more precise error descriptions (e.g. “invalid IPv6 socket address syntax: expected a decimal scope ID after %”), but that’s a more invasive change, and probably not worthwhile anyway.
Making `BorrowedFd::borrow_raw` a const fn allows it to be used to
create a constant `BorrowedFd<'static>` holding constants such as
`AT_FDCWD`. This will allow [`rustix::fs::cwd`] to become a const fn.
For consistency, make similar changes to `BorrowedHandle::borrow_raw`
and `BorrowedSocket::borrow_raw`.
[`rustix::fs::cwd`]: https://docs.rs/rustix/latest/rustix/fs/fn.cwd.html
This implementation returns a best attempt at the current exe path. On
fuchsia, fdio will always use `argv[0]` as the process name and if it is
not set then an error will be returned. Because this is not guaranteed
to be the case, this implementation returns an error if `argv` does not
contain any elements.
The existing description was incorrect for socket addresses, and
misleading: users would see “invalid IP address syntax” and suppose they
were supposed to provide an IP address rather than a socket address.
I contemplated making it two variants (IP, socket), but realised we can
do still better for the IPv4 and IPv6 types, so here it is as six.
I contemplated more precise error descriptions (e.g. “invalid IPv6
socket address syntax: expected a decimal scope ID after %”), but that’s
a more invasive change, and probably not worthwhile anyway.
