Fix is_absolute on WASI
WASI does not match `cfg(unix)`, but its paths are Unix-like (`/some/path`) and don't have Windows-like prefixes.
Without this change, `is_absolute` for any paths, including `/some/path`, was returning `false`on a WASI target, which is obviously not true and undesirable.
Split sys_common::Mutex in StaticMutex and MovableMutex.
The (unsafe) `Mutex` from `sys_common` had a rather complicated interface. You were supposed to call `init()` manually, unless you could guarantee it was neither moved nor used reentrantly.
Calling `destroy()` was also optional, although it was unclear if 1) resources might be leaked or not, and 2) if `destroy()` should only be called when `init()` was called.
This allowed for a number of interesting (confusing?) different ways to use this `Mutex`, all captured in a single type.
In practice, this type was only ever used in two ways:
1. As a static variable. In this case, neither `init()` nor `destroy()` are called. The variable is never moved, and it is never used reentrantly. It is only ever locked using the `LockGuard`, never with `raw_lock`.
2. As a `Box`ed variable. In this case, both `init()` and `destroy()` are called, it will be moved and possibly used reentrantly.
No other combinations are used anywhere in `std`.
This change simplifies things by splitting this `Mutex` type into two types matching the two use cases: `StaticMutex` and `MovableMutex`.
The interface of both new types is now both safer and simpler. The first one does not call nor expose `init`/`destroy`, and the second one calls those automatically in its `new()` and `Drop` functions. Also, the locking functions of `MovableMutex` are no longer unsafe.
---
This will also make it easier to conditionally box mutexes later, by moving that decision into sys/sys_common. Some of the mutex implementations (at least those of Wasm and 'sys/unsupported') are safe to move, so wouldn't need a box. ~~(But that's blocked on #76932 for now.)~~ (See #77380.)
Improve std::sys::windows::compat
Improves the compat_fn macro in sys::windows, which is used for conditionally loading APIs that might not be available.
- The module (dll) name can now be any string, not just an ident. (Not all Windows api modules are valid Rust identifiers. E.g. `WaitOnAddress` comes from `API-MS-Win-Core-Synch-l1-2-0.dll`.)
- Adds `FuncName::is_available()` for checking if a function is really available without having to do a duplicate lookup.
- Add comment explaining the lack of locking.
- Use `$_:block` to simplify the macro_rules.
- Apply `allow(unused_variables)` only to the fallback instead of everything.
---
The second point (`is_available()`) simplifies code that needs to pick an implementation depening on what is available, like `sys/windows/mutex.rs`. Before this change, it'd do its own lookup and keep its own `AtomicUsize` to track the result. Now it can just use `c::AcquireSRWLockExclusive::is_available()` directly.
This will also be useful when park/unpark/CondVar/etc. get improved implementations (e.g. from parking_lot or something else), as the best APIs for those are not available before Windows 8.
Make RawFd implement the RawFd traits
This PR makes `RawFd` implement `AsRawFd`, `IntoRawFd` and `FromRawFd`, so it can be passed to interfaces that use one of those traits as a bound.
- Module name can now be any string, not just an ident.
(Not all Windows api modules are valid Rust identifiers.)
- Adds c::FuncName::is_available() for checking if a function is really
available without having to do a duplicate lookup.
- Add comment explaining the lack of locking.
- Use `$_:block` to simplify the macro_rules.
- Apply allow(unused_variables) only to the fallback instead of
everything.
Use futex-based thread::park/unpark on Linux.
This moves the parking/unparking logic out of `thread/mod.rs` into a module named `thread_parker` in `sys_common`. The current implementation is moved to `sys_common/thread_parker/generic.rs` and the new implementation using futexes is added in `sys_common/thread_parker/futex.rs`.
The posix_spawn had been available in a form suitable for use in a
Command implementation since musl 0.9.12. Use it in a preference to a
fork when possible, to benefit from CLONE_VM|CLONE_VFORK used there.
Use `rtassert!` instead of `assert!` from the child process after fork() in std::sys::unix::process::Command::spawn()
As discussed in #73894, `assert!` panics on failure, which is not signal-safe, and `rtassert!` is a suitable replacement.
Fixes#73894.
r? @Amanieu @cuviper @joshtriplett
WASI does not match `cfg(unix)`, but its paths are Unix-like (`/some/path`) and don't have Windows-like prefixes.
Without this change, `is_absolute` for paths like `/some/path` was returning `false`on a WASI target, which is obviously not true and undesirable.
Remove `#[rustc_allow_const_fn_ptr]` and add `#![feature(const_fn_fn_ptr_basics)]`
`rustc_allow_const_fn_ptr` was a hack to work around the lack of an escape hatch for the "min `const fn`" checks in const-stable functions. Now that we have co-opted `allow_internal_unstable` for this purpose, we no longer need a bespoke attribute.
Now this functionality is gated under `const_fn_fn_ptr_basics` (how concise!), and `#[allow_internal_unstable(const_fn_fn_ptr_basics)]` replaces `#[rustc_allow_const_fn_ptr]`. `const_fn_fn_ptr_basics` allows function pointer types to appear in the arguments and locals of a `const fn` as well as function pointer casts to be performed inside a `const fn`. Both of these were allowed in constants and statics already. Notably, this does **not** allow users to invoke function pointers in a const context. Presumably, we will use a nicer name for that (`const_fn_ptr`?).
r? @oli-obk
UI to unit test for those using Cell/RefCell/UnsafeCell
Helps with #76268.
I'm working on all files using `Cell` and moving them to unit tests when possible.
r? @matklad
Add missing definitions required by the sparc-unknown-linux-gnu target
This PR adds a few missing definitions required by sparc-unknown-linux-target which were discovered during build tests.
The syscalls returning a new file descriptors generally use
lowest-numbered file descriptor not currently opened, without any
exceptions for those corresponding to the standard streams.
Previously when any of standard streams has been closed before starting
the application, operations on std::io::{stderr,stdin,stdout} objects
were likely to operate on other logically unrelated file resources
opened afterwards.
Avoid the issue by reopening the standard streams when they are closed.
The thread local LOCAL_STDOUT and LOCAL_STDERR are only used by the test
crate to capture output from tests when running them in the same process
in differen threads. However, every program will check these variables
on every print, even outside of testing.
This involves allocating a thread local key, and registering a thread
local destructor. This can be somewhat expensive.
This change keeps a global flag (LOCAL_STREAMS) which will be set to
true when either of these local streams is used. (So, effectively only
in test and benchmark runs.) When this flag is off, these thread locals
are not even looked at and therefore will not be initialized on the
first output on every thread, which also means no thread local
destructors will be registered.
The (unsafe) Mutex from sys_common had a rather complicated interface.
You were supposed to call init() manually, unless you could guarantee it
was neither moved nor used reentrantly.
Calling `destroy()` was also optional, although it was unclear if 1)
resources might be leaked or not, and 2) if destroy() should only be
called when `init()` was called.
This allowed for a number of interesting (confusing?) different ways to
use this Mutex, all captured in a single type.
In practice, this type was only ever used in two ways:
1. As a static variable. In this case, neither init() nor destroy() are
called. The variable is never moved, and it is never used
reentrantly. It is only ever locked using the LockGuard, never with
raw_lock.
2. As a Boxed variable. In this case, both init() and destroy() are
called, it will be moved and possibly used reentrantly.
No other combinations are used anywhere in `std`.
This change simplifies things by splitting this Mutex type into
two types matching the two use cases: StaticMutex and MovableMutex.
The interface of both new types is now both safer and simpler. The first
one does not call nor expose init/destroy, and the second one calls
those automatically in its new() and Drop functions. Also, the locking
functions of MovableMutex are no longer unsafe.
Remove std::io::lazy::Lazy in favour of SyncOnceCell
The (internal) std::io::lazy::Lazy was used to lazily initialize the stdout and stdin buffers (and mutexes). It uses atexit() to register a destructor to flush the streams on exit, and mark the streams as 'closed'. Using the stream afterwards would result in a panic.
Stdout uses a LineWriter which contains a BufWriter that will flush the buffer on drop. This one is important to be executed during shutdown, to make sure no buffered output is lost. It also forbids access to stdout afterwards, since the buffer is already flushed and gone.
Stdin uses a BufReader, which does not implement Drop. It simply forgets any previously read data that was not read from the buffer yet. This means that in the case of stdin, the atexit() function's only effect is making stdin inaccessible to the program, such that later accesses result in a panic. This is uncessary, as it'd have been safe to access stdin during shutdown of the program.
---
This change removes the entire io::lazy module in favour of SyncOnceCell. SyncOnceCell's fast path is much faster (a single atomic operation) than locking a sys_common::Mutex on every access like Lazy did.
However, SyncOnceCell does not use atexit() to drop the contained object during shutdown.
As noted above, this is not a problem for stdin. It simply means stdin is now usable during shutdown.
The atexit() call for stdout is moved to the stdio module. Unlike the now-removed Lazy struct, SyncOnceCell does not have a 'gone and unusable' state that panics. Instead of adding this again, this simply replaces the buffer with one with zero capacity. This effectively flushes the old buffer *and* makes any writes afterwards pass through directly without touching a buffer, making print!() available during shutdown without panicking.
---
In addition, because the contents of the SyncOnceCell are no longer dropped, we can now use `&'static` instead of `Arc` in `Stdout` and `Stdin`. This also saves two levels of indirection in `stdin()` and `stdout()`, since Lazy effectively stored a `Box<Arc<T>>`, and SyncOnceCell stores the `T` directly.
Add `#![feature(const_fn_floating_point_arithmetic)]`
cc #76618
This is a template for splitting up `const_fn` into granular feature gates. I think this will make it easier, both for us and for users, to track stabilization of each individual feature. We don't *have* to do this, however. We could also keep stabilizing things out from under `const_fn`.
cc @rust-lang/wg-const-eval
r? @oli-obk
Std/thread: deny unsafe op in unsafe fn
Partial fix of #73904.
This encloses `unsafe` operations in `unsafe fn` in `libstd/thread`.
`@rustbot` modify labels: F-unsafe-block-in-unsafe-fn
Relax promises about condition variable.
For quite a while now, there have been plans to at some point use parking_lot or some other more efficient implementation of mutexes and condition variables. Right now, Mutex and CondVar both Box the 'real' mutex/condvar inside, to give it a stable address. This was done because implementations like pthread and Windows critical sections may not be moved. More efficient implementations based on futexes, WaitOnAddress, Windows SRW locks, parking_lot, etc. may be moved (while not borrowed), so wouldn't need boxing.
However, not boxing them (which would be great goal to achieve), breaks a promise std currently makes about CondVar. CondVar promises to panic when used with different mutexes, to ensure consistent behaviour on all platforms. To this check, a mutex is considered 'the same' if the address of the 'real mutex' in the Box is the same. This address doesn't change when moving a `std::mutex::Mutex` object, effectively giving it an identity that survives moves of the Mutex object. If we ever switch to a non-boxed version, they no longer carry such an identity, and this check can no longer be made.
Four options:
1. Always box mutexes.
2. Add a `MutexId` similar to `ThreadId`. Making mutexes bigger, and making it hard to ever have a `const fn new` for them.
3. Making the requirement of CondVar stricter: panic if the Mutex object itself moved.
4. Making the promise of CondVar weaker: don't promise to panic.
1, 2, and 3 seem like bad options. This PR updates the documentation for 4.
Make delegation methods of `std::net::IpAddr` unstably const
Make the following methods of `std::net::IpAddr` unstable const under the `const_ip` feature:
- `is_unspecified`
- `is_loopback`
- `is_global`
- `is_multicast`
Also adds a test for these methods in a const context.
Possible because these methods delegate to the inner `Ipv4Addr` or `Ipv6Addr`, which were made const ([PR#76205](https://github.com/rust-lang/rust/pull/76142) and [PR#76206](https://github.com/rust-lang/rust/pull/76206)), and the recent stabilization of const control flow.
Part of #76205
r? @ecstatic-morse
The (internal) std::io::lazy::Lazy was used to lazily initialize the
stdout and stdin buffers (and mutexes). It uses atexit() to register a
destructor to flush the streams on exit, and mark the streams as
'closed'. Using the stream afterwards would result in a panic.
Stdout uses a LineWriter which contains a BufWriter that will flush the
buffer on drop. This one is important to be executed during shutdown,
to make sure no buffered output is lost. It also forbids access to
stdout afterwards, since the buffer is already flushed and gone.
Stdin uses a BufReader, which does not implement Drop. It simply forgets
any previously read data that was not read from the buffer yet. This
means that in the case of stdin, the atexit() function's only effect is
making stdin inaccessible to the program, such that later accesses
result in a panic. This is uncessary, as it'd have been safe to access
stdin during shutdown of the program.
---
This change removes the entire io::lazy module in favour of
SyncOnceCell. SyncOnceCell's fast path is much faster (a single atomic
operation) than locking a sys_common::Mutex on every access like Lazy
did.
However, SyncOnceCell does not use atexit() to drop the contained object
during shutdown.
As noted above, this is not a problem for stdin. It simply means stdin
is now usable during shutdown.
The atexit() call for stdout is moved to the stdio module. Unlike the
now-removed Lazy struct, SyncOnceCell does not have a 'gone and
unusable' state that panics. Instead of adding this again, this simply
replaces the buffer with one with zero capacity. This effectively
flushes the old buffer *and* makes any writes afterwards pass through
directly without touching a buffer, making print!() available during
shutdown without panicking.
Rollup of 9 pull requests
Successful merges:
- #76898 (Record `tcx.def_span` instead of `item.span` in crate metadata)
- #76939 (emit errors during AbstractConst building)
- #76965 (Add cfg(target_has_atomic_equal_alignment) and use it for Atomic::from_mut.)
- #76993 (Changing the alloc() to accept &self instead of &mut self)
- #76994 (fix small typo in docs and comments)
- #77017 (Add missing examples on Vec iter types)
- #77042 (Improve documentation for ToSocketAddrs)
- #77047 (Miri: more informative deallocation error messages)
- #77055 (Add #[track_caller] to more panicking Cell functions)
Failed merges:
r? `@ghost`
Make the following methods of `std::net::IpAddr` unstable const under the `const_ip` feature:
- `is_unspecified`
- `is_loopback`
- `is_global`
- `is_multicast`
Also adds a test for these methods in a const context.
Possible because these methods delegate to the inner `Ipv4Addr` or `Ipv6Addr`, which were made const, and the recent stabilization of const control flow.
Part of #76205
Function to convert OpenOptions to c_int
Fixes: #74943
The creation_mode and access_mode function were already available in the OpenOptions struct, but currently private. I've added a new free functions to unix/fs.rs which takes the OpenOptions, and returns the c_int to be used as parameter for the `open` call.
Add non-`unsafe` `.get_mut()` for `Unsafecell`
- Tracking issue: https://github.com/rust-lang/rust/issues/76943
As discussed in: https://internals.rust-lang.org/t/add-non-unsafe-get-mut-for-unsafecell/12407
- ### [Rendered documentation](https://modest-dubinsky-1f9f47.netlify.app/core/cell/struct.unsafecell)
This PR tries to move the sound `&mut UnsafeCell<T> -> &mut T` projection that all the "downstream" constructions were already relying on, up to the root abstraction, where it rightfully belongs, and officially blessing it.
- this **helps reduce the amount of `unsafe` snippets out there** (_c.f._, the second commit of this PR: 09503fd1b3)
The fact that this getter is now expose for `UnsafeCell<T>` itself, will also help convey the idea that **`UnsafeCell` is not magical _w.r.t._ `&mut` accesses**, contrary to what some people incorrectly think.
- Even the standard library itself at some point had such a confusion, _c.f._ this comment where there is a mention of multi-threaded (and thus _shared_) access despite dealing with exclusive references over unique ownership: 59fb88d061/library/core/src/cell.rs (L498-L499)
r? @RalfJung
Remove unused feature gates from library/ crates
Removes some unused feature gates from library crates. It's likely not a complete list as I only tested a subset for which it's more likely that it is unused.
deny(unsafe_op_in_unsafe_fn) in libstd/path.rs
The libstd/path.rs part of #73904 . Wraps the two calls to an unsafe fn Initializer::nop() in an unsafe block.
Small cleanups in Windows Mutex.
- Move `held` into the boxed part, since the SRW lock implementation does not use this. This makes the Mutex 50% smaller.
- Use `Cell` instead of `UnsafeCell` for `held`, such that `.replace()` can be used.
- Add some comments.
- Avoid creating multiple `&mut`s to the critical section object in `ReentrantMutex`.
[fuchsia] Propagate the userspace UTC clock
On Fuchsia, spawning a subprocess does not automatically
clone all of the parent process' capabilities. UTC time on
Fuchsia is managed by a top-level userspace clock capability
that is cloned and passed to subprocesses.
This change ensures that any Rust subprocess gets access to the
UTC clock, if the parent had access to it. This is critical for
tests, which on Fuchsia, use panic=abort and spawn subprocesses
per test.
Consolidate some duplicate code in the sys modules.
This consolidates some modules which were duplicated throughout the sys module. The intent is to make it easier to update and maintain this code. This mainly affects the wasi, sgx, and "unsupported" targets.
I explicitly skipped hermit, cloudabi, and vxworks. These tier-3 targets have copied large sections of the sys tree. I don't think they should have, but I don't want to put effort into changing them. It also doesn't help that there aren't any scripts or instructions for building them.
There are still sections of duplicate code here and there, but this PR covers the easy parts where entire modules are the same.
deny(unsafe_op_in_unsafe_fn) in libstd/process.rs
The libstd/process.rs part of #73904 . Wraps the two calls to an unsafe fn Initializer::nop() in an unsafe block.
Will have to wait for #73909 to be merged, because of the feature in the libstd/lib.rs
On Fuchsia, spawning a subprocess does not automatically
clone all of the parent process' capabilities. UTC time on
Fuchsia is managed by a top-level userspace clock capability
that is cloned and passed to subprocesses.
This change ensures that any Rust subprocess gets access to the
UTC clock, if the parent had access to it. This is critical for
tests, which on Fuchsia, use panic=abort and spawn subprocesses
per test.
Implementation of peer credentials for Unix sockets
The code in `ucred.rs` is based on the work done in [PR 13](https://github.com/tokio-rs/tokio-uds/pull/13) in the tokio-uds repository on GitHub.
This commit is effectively a port to the stdlib, so credit to Martin Habovštiak (`@Kixunil)` and contributors for the meat of this work. 🥇
Happy to make changes as needed. 🙂
The code in `ucred.rs` is based on the work done in PR 13 in the
tokio-uds repository on GitHub. Link below for reference:
https://github.com/tokio-rs/tokio-uds/pull/13
Credit to Martin Habovštiak (GitHub username Kixunil) and contributors
for this work!
