Hashing does not have to use the whole Components parsing machinery because we only need it to match the
normalizations that Components does.
* stripping redundant separators -> skipping separators
* stripping redundant '.' directories -> skipping '.' following after a separator
That's all it takes.
And instead of hashing individual slices for each component we feed the bytes directly into the hasher which avoids
hashing the length of each component in addition to its contents.
Make `std:🧵:available_concurrency` support process-limited number of CPUs
Use `libc::sched_getaffinity` and count the number of CPUs in the returned mask. This handles cases where the process doesn't have access to all CPUs, such as when limited via `taskset` or similar.
This also covers cgroup cpusets.
This stabilizes dereferencing immutable raw pointers in const contexts.
It does not stabilize `*mut T` dereferencing. This is placed behind the
`const_raw_mut_ptr_deref` feature gate.
In #89522 we learned that `clone3` is interacting poorly with Gentoo's
`sandbox` tool. We only need that for the unstable pidfd extensions, so
otherwise avoid that and use a normal `fork`.
Replace `std::os::raw::c_ssize_t` with `std::os::raw::c_ptrdiff_t`
The discussions in #88345 brought up that `ssize_t` is not actually the signed index type defined in stddef.h, but instead it's `ptrdiff_t`. It seems pretty clear that the use of `ssize_t` here was a mistake on my part, and that if we're going to bother having a isize-alike for FFI in `std::os::raw`, it should be `ptrdiff_t` and not `ssize_t`.
Anyway, both this and `c_size_t` are dubious in the face of the discussion in https://internals.rust-lang.org/t/pre-rfc-usize-is-not-size-t/15369, and any RFC/project-group/etc that handles those issues there should contend with these types in some manner, but that doesn't mean we shouldn't fix something wrong like this, even if it is unstable.
All that said, `size_t` is *vastly* more common in function signatures than either `ssize_t` or `ptrdiff_t`, so I'm going to update the tracking issue's list of unresolved questions to note that perhaps we only want `c_size_t` — I mostly added the signed version for symmetry, rather than to meet a need. (Given this, I'm also fine with modifying this patch to instead remove `c_ssize_t` without a replacement)
CC `@magicant` (who brought the issue up)
CC `@chorman0773` (who has a significantly firmer grasp on the minutae of the C standard than I do)
r? `@joshtriplett` (original reviewer, active in the discussions around this)
Windows thread-local keyless drop
`#[thread_local]` allows us to maintain a per-thread list of destructors. This also avoids the need to synchronize global data (which is particularly tricky within the TLS callback function).
r? `@alexcrichton`
Add JoinHandle::is_running.
This adds:
```rust
impl<T> JoinHandle<T> {
/// Checks if the the associated thread is still running its main function.
///
/// This might return `false` for a brief moment after the thread's main
/// function has returned, but before the thread itself has stopped running.
pub fn is_running(&self) -> bool;
}
```
The usual way to check if a background thread is still running is to set some atomic flag at the end of its main function. We already do that, in the form of dropping an Arc which will reduce the reference counter. So we might as well expose that information.
This is useful in applications with a main loop (e.g. a game, gui, control system, ..) where you spawn some background task, and check every frame/iteration whether the background task is finished to .join() it in that frame/iteration while keeping the program responsive.
`#[thread_local]` allows us to maintain a per-thread list of destructors. This also avoids the need to synchronize global data (which is particularly tricky within the TLS callback function).
Add #[must_use] to remaining std functions (O-Z)
I've run out of compelling reasons to group functions together across crates so I'm just going to go module-by-module. This is half of the remaining items from the `std` crate, from O-Z.
`panicking::take_hook` has a side effect: it unregisters the current panic hook, returning it. I almost ignored it, but the documentation example shows `let _ = panic::take_hook();`, so following suit I went ahead and added a `#[must_use]`.
```rust
std::panicking fn take_hook() -> Box<dyn Fn(&PanicInfo<'_>) + 'static + Sync + Send>;
```
I added these functions that clippy did not flag:
```rust
std::path::Path fn starts_with<P: AsRef<Path>>(&self, base: P) -> bool;
std::path::Path fn ends_with<P: AsRef<Path>>(&self, child: P) -> bool;
std::path::Path fn with_file_name<S: AsRef<OsStr>>(&self, file_name: S) -> PathBuf;
std::path::Path fn with_extension<S: AsRef<OsStr>>(&self, extension: S) -> PathBuf;
```
Parent issue: #89692
r? `@joshtriplett`
Add #[must_use] to expensive computations
The unifying theme for this commit is weak, admittedly. I put together a list of "expensive" functions when I originally proposed this whole effort, but nobody's cared about that criterion. Still, it's a decent way to bite off a not-too-big chunk of work.
Given the grab bag nature of this commit, the messages I used vary quite a bit. I'm open to wording changes.
For some reason clippy flagged four `BTreeSet` methods but didn't say boo about equivalent ones on `HashSet`. I stared at them for a while but I can't figure out the difference so I added the `HashSet` ones in.
```rust
// Flagged by clippy.
alloc::collections::btree_set::BTreeSet<T> fn difference<'a>(&'a self, other: &'a BTreeSet<T>) -> Difference<'a, T>;
alloc::collections::btree_set::BTreeSet<T> fn symmetric_difference<'a>(&'a self, other: &'a BTreeSet<T>) -> SymmetricDifference<'a, T>
alloc::collections::btree_set::BTreeSet<T> fn intersection<'a>(&'a self, other: &'a BTreeSet<T>) -> Intersection<'a, T>;
alloc::collections::btree_set::BTreeSet<T> fn union<'a>(&'a self, other: &'a BTreeSet<T>) -> Union<'a, T>;
// Ignored by clippy, but not by me.
std::collections::HashSet<T, S> fn difference<'a>(&'a self, other: &'a HashSet<T, S>) -> Difference<'a, T, S>;
std::collections::HashSet<T, S> fn symmetric_difference<'a>(&'a self, other: &'a HashSet<T, S>) -> SymmetricDifference<'a, T, S>
std::collections::HashSet<T, S> fn intersection<'a>(&'a self, other: &'a HashSet<T, S>) -> Intersection<'a, T, S>;
std::collections::HashSet<T, S> fn union<'a>(&'a self, other: &'a HashSet<T, S>) -> Union<'a, T, S>;
```
Parent issue: #89692
r? ```@joshtriplett```
Stabilize `is_symlink()` for `Metadata` and `Path`
I'm not fully sure about `since` version, correct me if I'm wrong
Needs update after stabilization: [cargo-test-support](8063672238/crates/cargo-test-support/src/paths.rs (L202))
Linked issue: #85748
Use libc::sched_getaffinity and count the number of CPUs in the returned
mask. This handles cases where the process doesn't have access to all
CPUs, such as when limited via taskset or similar.
Automatically convert paths to verbatim for filesystem operations that support it
This allows using longer paths without the user needing to `canonicalize` or manually prefix paths. If the path is already verbatim then this has no effect.
Fixes: #32689
Rollup of 5 pull requests
Successful merges:
- #90239 (Consistent big O notation in map.rs)
- #90267 (fix: inner attribute followed by outer attribute causing ICE)
- #90288 (Add hint for people missing `TryFrom`, `TryInto`, `FromIterator` import pre-2021)
- #90304 (Add regression test for #75961)
- #90344 (Add tracking issue number to const_cstr_unchecked)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Clean up special function const checks
Mark them as const and `#[rustc_do_not_const_check]` instead of hard-coding them in const-eval checks.
r? `@oli-obk`
`@rustbot` label A-const-eval T-compiler
Remove fNN::lerp
Lerp is [surprisingly complex with multiple tradeoffs depending on what guarantees you want to provide](https://github.com/rust-lang/rust/issues/86269#issuecomment-869108301) (and what you're willing to drop for raw speed), so we don't have consensus on what implementation to use, let alone what signature - `t.lerp(a, b)` nicely puts `a, b` together, but makes dispatch to lerp custom types with the same signature basically impossible, and major ecosystem crates (e.g. nalgebra, glium) use `a.lerp(b, t)`, which is easily confusable. It was suggested to maybe provide a `Lerp<T>` trait and `t.lerp([a, b])`, which _could_ be implemented by downstream math libraries for their types, but also significantly raises the bar from a simple fNN method to a full trait, and does nothing to solve the implementation question. (It also raises the question of whether we'd support higher-order bezier interpolation.)
The only consensus we have is the lack of consensus, and the [general temperature](https://github.com/rust-lang/rust/issues/86269#issuecomment-951347135) is that we should just remove this method (giving the method space back to 3rd party libs) and revisit this if (and likely only if) IEEE adds lerp to their specification.
If people want a lerp, they're _probably_ already using (or writing) a math support library, which provides a lerp function for its custom math types and can provide the same lerp implementation for the primitive types via an extension trait.
See also [previous Zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/lerp.20API.20design)
cc ``@clarfonthey`` (original PR author), ``@m-ou-se`` (original r+), ``@scottmcm`` (last voice in tracking issue, prompted me to post this)
Closes#86269 (removed)
Stabilise unix_process_wait_more, extra ExitStatusExt methods
This stabilises the feature `unix_process_wait_more`. Tracking issue #80695, FCP needed.
This was implemented in #79982 and merged in January.
My change to use `Type::def_id()` (formerly `Type::def_id_full()`) in
more places caused some docs to show up that used to be missed by
rustdoc. Those docs contained unescaped square brackets, which triggered
linkcheck errors. This commit escapes the square brackets and adds this
particular instance to the linkcheck exception list.
Inline CStr::from_ptr
Inlining this function is valuable, as it allows LLVM to apply `strlen`-specific optimizations without having to enable LTO.
For instance, the following function:
```rust
pub fn f(p: *const c_char) -> Option<u8> {
unsafe { CStr::from_ptr(p) }.to_bytes().get(0).copied()
}
```
Looks like this if `CStr::from_ptr` is allowed to be inlined.
```asm
before:
push rax
call qword ptr [rip + std::ffi::c_str::CStr::from_ptr@GOTPCREL]
mov rcx, rax
cmp rdx, 1
sete dl
test rax, rax
sete al
or al, dl
jne .LBB1_2
mov dl, byte ptr [rcx]
.LBB1_2:
xor al, 1
pop rcx
ret
after:
mov dl, byte ptr [rdi]
test dl, dl
setne al
ret
```
Note that optimization turned this from O(N) to O(1) in terms of performance, as LLVM knows that it doesn't really need to call `strlen` to determine whether a string is empty or not.
Stabilize feature `saturating_div` for rust 1.58.0
The tracking issue is #89381
This seems like a reasonable simple change(?). The feature `saturating_div` was added as part of the ongoing effort to implement a `Saturating` integer type (see #87921). The implementation has been discussed [here](https://github.com/rust-lang/rust/pull/87921#issuecomment-899357720) and [here](https://github.com/rust-lang/rust/pull/87921#discussion_r691888556). It extends the list of saturating operations on integer types (like `saturating_add`, `saturating_sub`, `saturating_mul`, ...) by the function `fn saturating_div(self, rhs: Self) -> Self`.
The stabilization of the feature `saturating_int_impl` (for the `Saturating` type) needs to have this stabilized first.
Closes#89381
Previously, it wasn't clear whether "This could include" was referring
to logic errors, or undefined behaviour. Tweak wording to clarify this
sentence does not relate to UB.
Stabilize CString::from_vec_with_nul[_unchecked]
Closes the tracking issue #73179. I am keeping this in _draft_ mode until the FCP has ended.
This is my first time stabilizing a feature, so I would appreciate any guidance on things I should do differently.
Closes#73179
Remove unnecessary condition in Barrier::wait()
This is my first pull request for Rust, so feel free to call me out if anything is amiss.
After some examination, I realized that the second condition of the "spurious-wakeup-handler" loop in ``std::sync::Barrier::wait()`` should always evaluate to ``true``, making it redundant in the ``&&`` expression.
Here is the affected function before the fix:
```rust
#[stable(feature = "rust1", since = "1.0.0")]
pub fn wait(&self) -> BarrierWaitResult {
let mut lock = self.lock.lock().unwrap();
let local_gen = lock.generation_id;
lock.count += 1;
if lock.count < self.num_threads {
// We need a while loop to guard against spurious wakeups.
// https://en.wikipedia.org/wiki/Spurious_wakeup
while local_gen == lock.generation_id && lock.count < self.num_threads { // fixme
lock = self.cvar.wait(lock).unwrap();
}
BarrierWaitResult(false)
} else {
lock.count = 0;
lock.generation_id = lock.generation_id.wrapping_add(1);
self.cvar.notify_all();
BarrierWaitResult(true)
}
}
```
At first glance, it seems that the check that ``lock.count < self.num_threads`` would be necessary in order for a thread A to detect when another thread B has caused the barrier to reach its thread count, making thread B the "leader".
However, the control flow implicitly results in an invariant that makes observing ``!(lock.count < self.num_threads)``, i.e. ``lock.count >= self.num_threads`` impossible from thread A.
When thread B, which will be the leader, calls ``.wait()`` on this shared instance of the ``Barrier``, it locks the mutex in the first line and saves the ``MutexGuard`` in the ``lock`` variable. It then increments the value of ``lock.count``. However, it then proceeds to check if ``lock.count < self.num_threads``. Since it is the leader, it is the case that (after the increment of ``lock.count``), the lock count is *equal* to the number of threads. Thus, the second branch is immediately taken and ``lock.count`` is zeroed. Additionally, the generation ID is incremented (with wrap). Then, the condition variable is signalled. But, the other threads are waiting at the line ``lock = self.cvar.wait(lock).unwrap();``, so they cannot resume until thread B's call to ``Barrier::wait()`` returns, which drops the ``MutexGuard`` acquired in the first ``let`` statement and unlocks the mutex.
The order of events is thus:
1. A thread A calls `.wait()`
2. `.wait()` acquires the mutex, increments `lock.count`, and takes the first branch
3. Thread A enters the ``while`` loop since the generation ID has not changed and the count is less than the number of threads for the ``Barrier``
3. Spurious wakeups occur, but both conditions hold, so the thread A waits on the condition variable
4. This process repeats for N - 2 additional times for non-leader threads A'
5. *Meanwhile*, Thread B calls ``Barrier::wait()`` on the same barrier that threads A, A', A'', etc. are waiting on. The thread count reaches the number of threads for the ``Barrier``, so all threads should now proceed, with B being the leader. B acquires the mutex and increments the value ``lock.count`` only to find that it is not less than ``self.num_threads``. Thus, it immediately clamps ``self.num_threads`` back down to 0 and increments the generation. Then, it signals the condvar to tell the A (prime) threads that they may continue.
6. The A, A', A''... threads wake up and attempt to re-acquire the ``lock`` as per the internal operation of a condition variable. When each A has exclusive access to the mutex, it finds that ``lock.generation_id`` no longer matches ``local_generation`` **and the ``&&`` expression short-circuits -- and even if it were to evaluate it, ``self.count`` is definitely less than ``self.num_threads`` because it has been reset to ``0`` by thread B *before* B dropped its ``MutexGuard``**.
Therefore, it my understanding that it would be impossible for the non-leader threads to ever see the second boolean expression evaluate to anything other than ``true``. This PR simply removes that condition.
Any input would be appreciated. Sorry if this is terribly verbose. I'm new to the Rust community and concurrency can be hard to explain in words. Thanks!
Reject octal zeros in IPv4 addresses
This fixes#86964 by rejecting octal zeros in IP addresses, such that `192.168.00.00000000` is rejected with a parse error, since having leading zeros in front of another zero indicates it is a zero written in octal notation, which is not allowed in the strict mode specified by RFC 6943 3.1.1. Octal rejection was implemented in #83652, but due to the way it was implemented octal zeros were still allowed.
removing TLS support in x86_64-unknown-none-hermitkernel
HermitCore's kernel itself doesn't support TLS. Consequently, the entries in x86_64-unknown-none-hermitkernel should be removed. This commit should help to finalize #89062.
linux/aarch64 Now() should be actually_monotonic()
While issues have been seen on arm64 platforms the Arm architecture requires
that the counter monotonically increases and that it must provide a uniform
view of system time (e.g. it must not be possible for a core to receive a
message from another core with a time stamp and observe time going backwards
(ARM DDI 0487G.b D11.1.2). While there have been a few 64bit SoCs that have
bugs (#49281, #56940) which cause time to not monotonically increase, these have
been fixed in the Linux kernel and we shouldn't penalize all Arm SoCs for those
who refuse to update their kernels:
SUN50I_ERRATUM_UNKNOWN1 - Allwinner A64 / Pine A64 - fixed in 5.1
FSL_ERRATUM_A008585 - Freescale LS2080A/LS1043A - fixed in 4.10
HISILICON_ERRATUM_161010101 - Hisilicon 1610 - fixed in 4.11
ARM64_ERRATUM_858921 - Cortex A73 - fixed in 4.12
255a3f3e18 std: Force `Instant::now()` to be monotonic added a Mutex to work around
this problem and a small test program using glommio shows the majority of time spent
acquiring and releasing this Mutex. 3914a7b0da tries to improve this, but actually
makes it worse on big systems as for 128b atomics a ldxp/stxp pair (and successful loop)
for v8.4 systems that don't support FEAT_LSE2 is required which is expensive as a lock
and because of how the load/store-exclusives scale on large Arm systems is both unfair
to threads and tends to go backwards in performance.
A small sample program using glommio improves by 70x on a 32 core Graviton2
system with this change.
[fuchsia] Update process info struct
The fuchsia platform is in the process of softly transitioning over to
using a new value for ZX_INFO_PROCESS with a new corresponding struct.
This change migrates libstd.
See [fxrev.dev/510478](https://fxrev.dev/510478) and [fxbug.dev/30751](https://fxbug.dev/30751) for more detail.
Add abstract namespace support for Unix domain sockets
Hello! The other day I wanted to mess around with UDS in Rust and found that abstract namespaces ([unix(7)](https://man7.org/linux/man-pages/man7/unix.7.html)) on Linux still needed development. I took the approach of adding `_addr` specific public functions to reduce conflicts.
Feature name: `unix_socket_abstract`
Tracking issue: #85410
Further context: #42048
## Non-platform specific additions
`UnixListener::bind_addr(&SocketAddr) -> Result<UnixListener>`
`UnixStream::connect_addr(&SocketAddr) -> Result<()>`
`UnixDatagram::bind_addr(&SocketAddr) -> Result<UnixDatagram>`
`UnixDatagram::connect_addr(&SocketAddr) -> Result<()>`
`UnixDatagram::send_to_addr(&self, &[u8], &SocketAddr) -> Result<usize>`
## Platform-specific (Linux) additions
`SocketAddr::from_abstract_namespace(&[u8]) -> SocketAddr`
`SockerAddr::as_abstract_namespace() -> Option<&[u8]>`
## Example
```rust
#![feature(unix_socket_abstract)]
use std::os::unix::net::{UnixListener, SocketAddr};
fn main() -> std::io::Result<()> {
let addr = SocketAddr::from_abstract_namespace(b"namespace")?; // Linux only
let listener = match UnixListener::bind_addr(&addr) {
Ok(sock) => sock,
Err(err) => {
println!("Couldn't bind: {:?}", err);
return Err(err);
}
};
Ok(())
}
```
## Further Details
The main inspiration for the implementation came from the [nix-rust](https://github.com/nix-rust/nix/blob/master/src/sys/socket/addr.rs#L558) crate but there are also other [historical](c4db0685b1) [attempts](https://github.com/tormol/uds/blob/master/src/addr.rs#L324) with similar approaches.
A comment I did have was with this change, we now allow a `SocketAddr` to be constructed explicitly rather than just used almost as a handle for the return of `peer_addr` and `local_addr`. We could consider adding other explicit constructors (e.g. `SocketAddr::from_pathname`, `SockerAddr::from_unnamed`).
Cheers!
Use BCryptGenRandom instead of RtlGenRandom on Windows.
This removes usage of RtlGenRandom on Windows, in favour of BCryptGenRandom.
BCryptGenRandom isn't available on XP, but we dropped XP support a while ago.
The fuchsia platform is in the process of softly transitioning over to
using a new value for ZX_INFO_PROCESS with a new corresponding struct.
This change migrates libstd.
See fxrev.dev/510478 and fxbug.dev/30751 for more detail.
Fix ctrl-c causing reads of stdin to return empty on Windows.
Pressing ctrl+c (or ctrl+break) on Windows caused a blocking read of stdin to unblock and return empty, unlike other platforms which continue to block.
On ctrl-c, `ReadConsoleW` will return success, but also set `LastError` to `ERROR_OPERATION_ABORTED`.
This change detects this case, and re-tries the call to `ReadConsoleW`.
Fixes#89177. See issue for further details.
Tested on Windows 7 and Windows 10 with both MSVC and GNU toolchains
Improve `std:🧵:available_parallelism` docs
_Tracking issue: https://github.com/rust-lang/rust/issues/74479_
This PR reworks the documentation of `std:🧵:available_parallelism`, as requested [here](https://github.com/rust-lang/rust/pull/89324#issuecomment-934343254).
## Changes
The following changes are made:
- We've removed prior mentions of "hardware threads" and instead centers the docs around "parallelism" as a resource available to a program.
- We now provide examples of when `available_parallelism` may return numbers that differ from the number of CPU cores in the host machine.
- We now mention that the amount of available parallelism may change over time.
- We make note of which platform components we don't take into account which more advanced users may want to take note of.
- The example has been updated, which should be a bit easier to use.
- We've added a docs alias to `num-cpus` which provides similar functionality to `available_parallelism`, and is one of the most popular crates on crates.io.
---
Thanks!
r? `@BurntSushi`
The unifying theme for this commit is weak, admittedly. I put together a
list of "expensive" functions when I originally proposed this whole
effort, but nobody's cared about that criterion. Still, it's a decent
way to bite off a not-too-big chunk of work.
Given the grab bag nature of this commit, the messages I used vary quite
a bit.
Add #[must_use] to is_condition tests
I threw in `std::path::Path::has_root` for funsies.
A continuation of #89718.
Parent issue: #89692
r? ```@joshtriplett```
Add #[must_use] to non-mutating verb methods
These are methods that could be misconstrued to mutate their input, similar to #89694. I gave each one a different custom message.
I wrote that `upgrade` and `downgrade` don't modify the input pointers. Logically they don't, but technically they do...
Parent issue: #89692
r? ```@joshtriplett```
