Fix the stability attributes for `std::os::fd`.
As `@bjorn3` pointed out [here], I used the wrong stability attribute in #98368 when making `std::os::fd` public. I set it to Rust 1.63, which was when io-safety was stabilized, but it should be Rust 1.66, which was when `std::os::fd` was stabilized.
[here]: https://github.com/rust-lang/rust/pull/98368#discussion_r1063721420
As @bjorn3 pointed out [here], I used the wrong stability attribute in #98368
when making `std::os::fd` public. I set it to Rust 1.63, which was when
io-safety was stabilized, but it should be Rust 1.66, which was when
`std::os::fd` was stabilized.
[here]: https://github.com/rust-lang/rust/pull/98368#discussion_r1063721420
Remove various double spaces in the libraries.
I was just pretty bothered by this when reading the source for a function, and was suggested to check if this happened elsewhere.
Stop probing for statx unless necessary
As is the current toy program:
fn main() -> std::io::Result<()> {
use std::fs;
let metadata = fs::metadata("foo.txt")?;
assert!(!metadata.is_dir());
Ok(())
}
... observed under strace will issue:
[snip]
statx(0, NULL, AT_STATX_SYNC_AS_STAT, STATX_ALL, NULL) = -1 EFAULT (Bad address) statx(AT_FDCWD, "foo.txt", AT_STATX_SYNC_AS_STAT, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
While statx is not necessarily always present, checking for it can be delayed to the first error condition. Said condition may very well never happen, in which case the check got avoided altogether.
Note this is still suboptimal as there still will be programs issuing it, but bulk of the problem is removed.
Tested by forbidding the syscall for the binary and observing it correctly falls back to newfstatat.
While here tidy up the commentary, in particular by denoting some problems with the current approach.
[LSDA] Take ttype_index into account when taking unwind action
If `cs_action != 0`, we should check the `ttype_index` field in action record. If `ttype_index == 0`, a clean up action is taken; otherwise catch action is taken.
This can fix unwind failure on AIX which uses LLVM's libunwind by default. IIUC, rust's LSDA is borrowed from c++ and I'm assuming itanium-cxx-abi https://itanium-cxx-abi.github.io/cxx-abi/exceptions.pdf should be followed, so the fix follows what libcxxabi does. See ec48682ce9/libcxxabi/src/cxa_personality.cpp (L152) for use of `ttype_index`.
- Fixes https://github.com/rust-lang/rust/issues/91628.
- Fixes https://github.com/emscripten-core/emscripten/issues/15722.
See discussion in both issues.
The TL;DR is that weak linkage causes LLVM to produce broken Wasm, presumably due to pointer mismatch. The code is casting a void pointer to a function pointer with specific signature, but Wasm is very strict about function pointer compatibility, so the resulting code is invalid.
Ideally LLVM should catch this earlier in the process rather than emit invalid Wasm, but it currently doesn't and this is an easy and valid fix, given that Emcripten doesn't have `__cxa_thread_atexit_impl` these days anyway.
Unfortunately, I can't add a regression test as even after looking into this issue for a long time, I couldn't reproduce it with any minimal Rust example, only with extracted LLVM IR or on a large project involving Rust + C++.
r? @alexcrichton
std tests: use __OsLocalKeyInner from realstd
This is basically the same as https://github.com/rust-lang/rust/pull/100201, but for __OsLocalKeyInner:
Some std tests are failing in Miri on Windows because [this static](a377893da2/library/std/src/sys/windows/thread_local_key.rs (L234-L239)) is getting duplicated, and Miri does not handle that properly -- Miri does not support this magic `.CRT$XLB` linker section, but instead just looks up this particular hard-coded static in the standard library. This PR lets the test suite use the std static instead of having its own copy.
Fixes https://github.com/rust-lang/miri/issues/2754
r? `@thomcc`
As is the current toy program:
fn main() -> std::io::Result<()> {
use std::fs;
let metadata = fs::metadata("foo.txt")?;
assert!(!metadata.is_dir());
Ok(())
}
... observed under strace will issue:
[snip]
statx(0, NULL, AT_STATX_SYNC_AS_STAT, STATX_ALL, NULL) = -1 EFAULT (Bad address)
statx(AT_FDCWD, "foo.txt", AT_STATX_SYNC_AS_STAT, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
While statx is not necessarily always present, checking for it can be
delayed to the first error condition. Said condition may very well never
happen, in which case the check got avoided altogether.
Note this is still suboptimal as there still will be programs issuing
it, but bulk of the problem is removed.
Tested by forbidding the syscall for the binary and observing it
correctly falls back to newfstatat.
While here tidy up the commentary, in particular by denoting some
problems with the current approach.
The `std::os::unix` module is stubbed out when building docs for these
target platforms. The introduction of Linux-specific extension traits
caused `std::os::net` to depend on sub-modules of `std::os::unix`,
which broke rustdoc for the `wasm32-unknown-unknown` target.
Adding an additional `#[cfg]` guard solves that rustdoc failure by
not declaring `linux_ext` on targets with a stubbed `std::os::unix`.
Update `rand` in the stdlib tests, and remove the `getrandom` feature from it.
The main goal is actually removing `getrandom`, so that eventually we can allow running the stdlib test suite on tier3 targets which don't have `getrandom` support. Currently those targets can only run the subset of stdlib tests that exist in uitests, and (generally speaking), we prefer not to test libstd functionality in uitests, which came up recently in https://github.com/rust-lang/rust/pull/104095 and https://github.com/rust-lang/rust/pull/104185. Additionally, the fact that we can't update `rand`/`getrandom` means we're stuck with the old set of tier3 targets, so can't test new ones.
~~Anyway, I haven't checked that this actually does allow use on tier3 targets (I think it does not, as some work is needed in stdlib submodules) but it moves us slightly closer to this, and seems to allow at least finally updating our `rand` dep, which definitely improves the status quo.~~ Checked and works now.
For the most part, our tests and benchmarks are fine using hard-coded seeds. A couple tests seem to fail with this (stuff manipulating the environment expecting no collisions, for example), or become pointless (all inputs to a function become equivalent). In these cases I've done a (gross) dance (ab)using `RandomState` and `Location::caller()` for some extra "entropy".
Trying to share that code seems *way* more painful than it's worth given that the duplication is a 7-line function, even if the lines are quite gross. (Keeping in mind that sharing it would require adding `rand` as a non-dev dep to std, and exposing a type from it publicly, all of which sounds truly awful, even if done behind a perma-unstable feature).
See also some previous attempts:
- https://github.com/rust-lang/rust/pull/86963 (in particular https://github.com/rust-lang/rust/pull/86963#issuecomment-885438936 which explains why this is non-trivial)
- https://github.com/rust-lang/rust/pull/89131
- https://github.com/rust-lang/rust/pull/96626#issuecomment-1114562857 (I tried in that PR at the same time, but settled for just removing the usage of `thread_rng()` from the benchmarks, since that was the main goal).
- https://github.com/rust-lang/rust/pull/104185
- Probably more. It's very tempting of a thing to "just update".
r? `@Mark-Simulacrum`
default OOM handler: use non-unwinding panic, to match std handler
The OOM handler in std will by default abort. This adjusts the default in liballoc to do the same, using the `can_unwind` flag on the panic info to indicate a non-unwinding panic.
In practice this probably makes little difference since the liballoc default will only come into play in no-std situations where people write a custom panic handler, which most likely will not implement unwinding. But still, this seems more consistent.
Cc `@rust-lang/wg-allocators,` https://github.com/rust-lang/rust/issues/66741
Add notes and examples about non-intuitive `PathBuf::set_extension` behavior
Basically, passing the empty string will actually remove the extension instead of setting it to the empty string. This might change what is considered to be an extension. Additionally, passing an extension that contains dots will make the path only consider the last part of it to be the new extension.
Unify id-based thread parking implementations
Multiple platforms currently use thread-id-based parking implementations (NetBSD and SGX[^1]). Even though the strategy does not differ, these are duplicated for each platform, as the id is encoded into an atomic thread variable in different ways for each platform.
Since `park` is only called by one thread, it is possible to move the thread id into a separate field. By ensuring that the field is only written to once, before any other threads access it, these accesses can be unsynchronized, removing any restrictions on the size and niches of the thread id.
This PR also renames the internal `thread_parker` modules to `thread_parking`, as that name now better reflects their contents. I hope this does not add too much reviewing noise.
r? `@m-ou-se`
`@rustbot` label +T-libs
[^1]: SOLID supports this as well, I will switch it over in a follow-up PR.
Basically, passing the empty string will actually remove the extension
instead of setting it to the empty string. This might change what is
considered to be an extension. Additionally, passing an extension that
contains dots will make the path only consider the last part of it to be
the new extension.
Add #[inline] markers to once_cell methods
Added inline markers to all simple methods under the `once_cell` feature. Relates to #74465 and #105587
This should not block #105587
`IN6ADDR_ANY_INIT` and `IN6ADDR_LOOPBACK_INIT` documentation.
Added documentation for IPv6 Addresses `IN6ADDR_ANY_INIT` also known as `in6addr_any` and `IN6ADDR_LOOPBACK_INIT` also known as `in6addr_loopback` similar to `INADDR_ANY` for IPv4 Addresses.
Catch panics/unwinding in destruction of TLS values
`destroy_value` is/can be called from C code (libc). Unwinding from Rust to C code is undefined behavior, which is why unwinding is caught here.
This problem caused an infinite loop inside the unwinding code when running `src/test/ui/threads-sendsync/issue-24313.rs` on a tier 3 target (QNX/Neutrino) on aarch64.
See also https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/Infinite.20unwinding.20bug.
kmc-solid: Fix memory ordering in thread operations
Fixes two memory ordering issues in the thread state machine (`ThreadInner::lifecycle`) of the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets.
1. When detaching a thread that is still running (i.e., the owner updates `lifecycle` first, and the child updates it next), the first update did not synchronize-with the second update, resulting in a data race between the first update and the deallocation of `ThreadInner` by the child thread.
2. When joining on a thread, the joiner has to pass its own task ID to the joinee in order to be woken up later, but in doing so, it did not synchronize-with the read operation, creating possible sequences of execution where the joinee wakes up an incorrect or non-existent task.
Both issue are theoretical and most likely have never manifested in practice because of the stronger guarantees provided by the Arm memory model (particularly due to its barrier-based definition). Compiler optimizations could have subverted this, but the inspection of compiled code did not reveal such optimizations taking place.
Bump master bootstrap compiler
This PR bumps the bootstrap compiler to the beta created earlier this week, cherry-picks the stabilization version number updates, and updates the `cfg(bootstrap)`s.
r? `@Mark-Simulacrum`
adjust message on non-unwinding panic
"thread panicked while panicking" is just plain wrong in case this is a non-unwinding panic, such as
- a panic out of a `nounwind` function
- the sanity checks we have in `mem::uninitialized` and `mem::zeroed`
- the optional debug assertion in various unsafe std library functions
Clarify `catch_unwind` docs about panic hooks
Makes it clear from `catch_unwind` docs that the panic hook will be called before the panic is caught.
Fixes#105432
Make sentinel value configurable in `library/std/src/sys_common/thread_local_key.rs`
This is an excerpt of a changeset for the QNX/Neutrino OS. To make the patch for QNX smaller and easier to review, I've extracted this change (which is OS independent). I would be surprised if no other OS is also affected.
All this patch does is to define a `const` for a sentinel value instead of using it directly at several places.
There are OSs that always return the lowest free value. The algorithm in `lazy_init` always avoids keys with the sentinel value.
In affected OSs, this means that each call to `lazy_init` will always request two keys from the OS and returns/frees the first one (with sentinel value) immediately afterwards.
By making the sentinel value configurable, affected OSs can use a different value than zero to prevent this performance issue.
On QNX/Neutrino, it is planned to use a different sentinel value:
```rust
// Define a sentinel value that is unlikely to be returned
// as a TLS key (but it may be returned).
#[cfg(not(target_os = "nto"))]
const KEY_SENTVAL: usize = 0;
// On QNX/Neutrino, 0 is always returned when currently not in use.
// Using 0 would mean to always create two keys and remote the first
// one (with value of 0) immediately afterwards.
#[cfg(target_os = "nto")]
const KEY_SENTVAL: usize = libc::PTHREAD_KEYS_MAX + 1;
```
It seems like no other OS defines `PTHREAD_KEYS_MAX` in Rusts libc, but `limits.h` on unix systems does.
available_parallelism: Gracefully handle zero value cfs_period_us
There seem to be some scenarios where the cgroup cpu quota field `cpu.cfs_period_us` can contain `0`. This field is used to determine the "amount" of parallelism suggested by the function `std:🧵:available_parallelism`
A zero value of this field cause a panic when `available_parallelism()` is invoked. This issue was detected by the call from binaries built by `cargo test`. I really don't feel like `0` is a good value for `cpu.cfs_period_us`, but I also don't think applications should panic if this value is seen.
This panic started happening with rust 1.64.0.
This case is gracefully handled by other projects which read this information: [num_cpus](e437b9d908/src/linux.rs (L207-L210)), [ninja](https://github.com/ninja-build/ninja/pull/2174/files), [dotnet](c4341d45ac/src/coreclr/pal/src/misc/cgroup.cpp (L481-L483))
Before this change, running `cargo test` in environments configured as described above would trigger this panic:
```
$ RUST_BACKTRACE=1 cargo test
Finished test [unoptimized + debuginfo] target(s) in 3.55s
Running unittests src/main.rs (target/debug/deps/x-9a42e145aca2934d)
thread 'main' panicked at 'attempt to divide by zero', library/std/src/sys/unix/thread.rs:546:70
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::panicking::panic
3: std::sys::unix:🧵:cgroups::quota
4: std::sys::unix:🧵:available_parallelism
5: std:🧵:available_parallelism
6: test::helpers::concurrency::get_concurrency
7: test::console::run_tests_console
8: test::test_main
9: test::test_main_static
10: x::main
at ./src/main.rs:1:1
11: core::ops::function::FnOnce::call_once
at /tmp/rust-1.64-1.64.0-1/library/core/src/ops/function.rs:248:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: test failed, to rerun pass '--bin x'
```
I've tested this change in an environment which has the bad (questionable?) setup and rebuilding the test executable against a fixed std library fixes the panic.
Move `ReentrantMutex` to `std::sync`
If I understand #84187 correctly, `sys_common` should not contain platform-independent code, even if it is private.
Fix backoff doc to match implementation
The commit 8dddb22943 in the crossbeam-channel PR (#93563) changed the backoff strategy to be quadratic instead of exponential. This updates the doc to prevent confusion.
Use correct clock in `park_timeout` on Horizon
Horizon does not support using `CLOCK_MONOTONIC` with condition variables, so use the system time instead.
More inference-friendly API for lazy
The signature for new was
```
fn new<F>(f: F) -> Lazy<T, F>
```
Notably, with `F` unconstrained, `T` can be literally anything, and just `let _ = Lazy::new(|| 92)` would not typecheck.
This historiacally was a necessity -- `new` is a `const` function, it couldn't have any bounds. Today though, we can move `new` under the `F: FnOnce() -> T` bound, which gives the compiler enough data to infer the type of T from closure.
Some C commandline parsers (e.g. GLib and Qt) are replacing already
handled arguments in `argv` with `NULL` and move them to the end. That
means that `argc` might be bigger than the actual number of non-`NULL`
pointers in `argv` at this point.
To handle this we simply stop iterating at the first `NULL` argument.
`argv` is also guaranteed to be `NULL`-terminated so any non-`NULL`
arguments after the first `NULL` can safely be ignored.
Fixes https://github.com/rust-lang/rust/issues/105999
Realistic `Path::as_mut_os_str` doctest
With "Implement DerefMut for PathBuf" (#105018) now merged, it's
possible to exercise `Path::as_mut_os_str` (#105002) without going
through `into_boxed_path`.
Use a more efficient `Once` on platforms without threads
The current implementation uses an atomic queue and spins rather than panicking when calling `call_once` recursively. Since concurrency is not supported on platforms like WASM, `Once` can be implemented much more efficiently using just a single non-atomic state variable.
fs: Fix#50619 (again) and add a regression test
Bug #50619 was fixed by adding an end_of_stream flag in #50630.
Unfortunately, that fix only applied to the readdir_r() path. When I
switched Linux to use readdir() in #92778, I inadvertently reintroduced
the bug on that platform. Other platforms that had always used
readdir() were presumably never fixed.
This patch enables end_of_stream for all platforms, and adds a
Linux-specific regression test that should hopefully prevent the bug
from being reintroduced again.
Allow blocking `Command::output`
### Problem
Currently, `Command::output` is internally implemented using `Command::spawn`. This is problematic because some targets (like UEFI) do not actually support multitasking and thus block while the program is executing. This coupling does not make much sense as `Command::output` is supposed to block until the execution is complete anyway and thus does not need to rely on a non-blocking `Child` or any other intermediate.
### Solution
This PR moves the implementation of `Command::output` to `std::sys`. This means targets can choose to implement only `Command::output` without having to implement `Command::spawn`.
### Additional Information
This was originally conceived when working on https://github.com/rust-lang/rust/pull/100316. Currently, the only target I know about that will benefit from this change is UEFI.
This PR can also be used to implement more efficient `Command::output` since the intermediate `Process` is not actually needed anymore, but that is outside the scope of this PR.
Since this is not a public API change, I'm not sure if an RFC is needed or not.
Implement DerefMut for PathBuf
Without this, there's no way to get a `&mut Path` from `PathBuf` without
going through `into_boxed_path`. This is relevant now that #105002 adds
`PathBuf::as_mut_os_string` and `Path::as_mut_os_str`.
Use more LFS functions.
On Linux, use mmap64, open64, openat64, and sendfile64 in place of their non-LFS counterparts.
This is relevant to #94173.
With these changes (together with rust-lang/backtrace-rs#501), the simple binaries I produce with rustc seem to have no non-LFS functions, so maybe #94173 is fixed. But I can't be sure if I've missed something and maybe some non-LFS functions could sneak in somehow.
Bug #50619 was fixed by adding an end_of_stream flag in #50630.
Unfortunately, that fix only applied to the readdir_r() path. When I
switched Linux to use readdir() in #92778, I inadvertently reintroduced
the bug on that platform. Other platforms that had always used
readdir() were presumably never fixed.
This patch enables end_of_stream for all platforms, and adds a
Linux-specific regression test that should hopefully prevent the bug
from being reintroduced again.
Better documentation for env::home_dir()'s broken behaviour
This improves the documentation to say *why* it was deprecated. The reason was because it reads `HOME` on Windows which is meaningless there. Note that the PR that deprecated it stated that returning an empty string if `HOME` is set to an empty string was a problem, however I can find no evidence that this is the case. `cd` handles it fine whereas if `HOME` is unset it gives an explicit `HOME not set` error.
* Original deprecation reason: https://internals.rust-lang.org/t/deprecate-or-break-fix-std-env-home-dir/7315
* Original deprecation PR: https://github.com/rust-lang/rust/pull/51656
See #71684
This allows decoupling `Command::spawn` and `Command::output`. This is
useful for targets which do support launching programs in blocking mode
but do not support multitasking (Eg: UEFI).
This was originally conceived when working on https://github.com/rust-lang/rust/pull/100316
Signed-off-by: Ayush Singh <ayushsingh1325@gmail.com>
