Reduce allocations for path conversions on Windows
Previously, UTF-8 to UTF-16 Path conversions on Windows unnecessarily allocate twice, as described in #96297. This commit fixes that issue.
Improve Windows path prefix parsing
This PR fixes improves parsing of Windows path prefixes. `parse_prefix` now supports both types of separators on Windows (`/` and `\`).
[fuchsia] Add implementation for `current_exe`
This implementation returns a best attempt at the current exe path. On
fuchsia, fdio will always use `argv[0]` as the process name and if it is
not set then an error will be returned. Because this is not guaranteed
to be the case, this implementation returns an error if `argv` does not
contain any elements.
remove_dir_all_recursive: treat ELOOP the same as ENOTDIR
On older Linux kernels (I tested on 4.4, corresponding to Ubuntu 16.04), opening a symlink using `O_DIRECTORY | O_NOFOLLOW` returns `ELOOP` instead of `ENOTDIR`. We should handle it the same, since a symlink is still not a directory and needs to be `unlink`ed.
Use sys::unix::locks::futex* on wasm+atomics.
This removes the wasm-specific lock implementations and instead re-uses the implementations from sys::unix.
Tracking issue: https://github.com/rust-lang/rust/issues/93740
cc ``@alexcrichton``
Improve AddrParseError description
The existing description was incorrect for socket addresses, and misleading: users would see “invalid IP address syntax” and suppose they were supposed to provide an IP address rather than a socket address.
I contemplated making it two variants (IP, socket), but realised we can do still better for the IPv4 and IPv6 types, so here it is as six.
I contemplated more precise error descriptions (e.g. “invalid IPv6 socket address syntax: expected a decimal scope ID after %”), but that’s a more invasive change, and probably not worthwhile anyway.
Making `BorrowedFd::borrow_raw` a const fn allows it to be used to
create a constant `BorrowedFd<'static>` holding constants such as
`AT_FDCWD`. This will allow [`rustix::fs::cwd`] to become a const fn.
For consistency, make similar changes to `BorrowedHandle::borrow_raw`
and `BorrowedSocket::borrow_raw`.
[`rustix::fs::cwd`]: https://docs.rs/rustix/latest/rustix/fs/fn.cwd.html
This implementation returns a best attempt at the current exe path. On
fuchsia, fdio will always use `argv[0]` as the process name and if it is
not set then an error will be returned. Because this is not guaranteed
to be the case, this implementation returns an error if `argv` does not
contain any elements.
The existing description was incorrect for socket addresses, and
misleading: users would see “invalid IP address syntax” and suppose they
were supposed to provide an IP address rather than a socket address.
I contemplated making it two variants (IP, socket), but realised we can
do still better for the IPv4 and IPv6 types, so here it is as six.
I contemplated more precise error descriptions (e.g. “invalid IPv6
socket address syntax: expected a decimal scope ID after %”), but that’s
a more invasive change, and probably not worthwhile anyway.
Use a single ReentrantMutex implementation on all platforms.
This replaces all platform specific ReentrantMutex implementations by the one I added in #95727 for Linux, since that one does not depend on any platform specific details.
r? `@Amanieu`
fix error handling for pthread_sigmask(3)
Errors from `pthread_sigmask(3)` were handled using `cvt()`, which expects a return value of `-1` on error and uses `errno`.
However, `pthread_sigmask(3)` returns `0` on success and an error number otherwise.
Fix it by replacing `cvt()` with `cvt_nz()`.
Use u32 instead of i32 for futexes.
This changes futexes from i32 to u32. The [Linux man page](https://man7.org/linux/man-pages/man2/futex.2.html) uses `uint32_t` for them, so I'm not sure why I used i32 for them. Maybe because I first used them for thread parkers, where I used -1, 0, and 1 as the states.
(Wasm's `memory.atomic.wait32` does use `i32`, because wasm doesn't support `u32`.)
It doesn't matter much, but using the unsigned type probably results in fewer surprises when shifting bits around or using comparison operators.
r? ```@Amanieu```
Create (unstable) 2024 edition
[On Zulip](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Deprecating.20macro.20scoping.20shenanigans/near/272860652), there was a small aside regarding creating the 2024 edition now as opposed to later. There was a reasonable amount of support and no stated opposition.
This change creates the 2024 edition in the compiler and creates a prelude for the 2024 edition. There is no current difference between the 2021 and 2024 editions. Cargo and other tools will need to be updated separately, as it's not in the same repository. This change permits the vast majority of work towards the next edition to proceed _now_ instead of waiting until 2024.
For sanity purposes, I've merged the "hello" UI tests into a single file with multiple revisions. Otherwise we'd end up with a file per edition, despite them being essentially identical.
````@rustbot```` label +T-lang +S-waiting-on-review
Not sure on the relevant team, to be honest.
Windows: Use a pipe relay for chaining pipes
Fixes#95759
This fixes the issue by chaining pipes synchronously and manually pumping messages between them. It's not ideal but it has the advantage of not costing anything if pipes are not chained ("don't pay for what you don't use") and it also avoids breaking existing code that rely on our end of the pipe being asynchronous (which includes rustc's own testing framework).
Libraries can avoid needing this by using their own pipes to chain commands.
Document that DirEntry holds the directory open
I had a bug where holding onto DirEntry structs caused file descriptor exhaustion, and thought it would be good to document this.
Replace RwLock by a futex based one on Linux
This replaces the pthread-based RwLock on Linux by a futex based one.
This implementation is similar to [the algorithm](https://gist.github.com/kprotty/3042436aa55620d8ebcddf2bf25668bc) suggested by `@kprotty,` but modified to prefer writers and spin before sleeping. It uses two futexes: One for the readers to wait on, and one for the writers to wait on. The readers futex contains the state of the RwLock: The number of readers, a bit indicating whether writers are waiting, and a bit indicating whether readers are waiting. The writers futex is used as a simple condition variable and its contents are meaningless; it just needs to be changed on every notification.
Using two futexes rather than one has the obvious advantage of allowing a separate queue for readers and writers, but it also means we avoid the problem a single-futex RwLock would have of making it hard for a writer to go to sleep while the number of readers is rapidly changing up and down, as the writers futex is only changed when we actually want to wake up a writer.
It always prefers writers, as we decided [here](https://github.com/rust-lang/rust/issues/93740#issuecomment-1070696128).
To be able to prefer writers, it relies on futex_wake to return the number of awoken threads to be able to handle write-unlocking while both the readers-waiting and writers-waiting bits are set. Instead of waking both and letting them race, it first wakes writers and only continues to wake the readers too if futex_wake reported there were no writers to wake up.
r? `@Amanieu`
Remove ptr-int transmute in std::sync::mpsc
Since https://github.com/rust-lang/rust/pull/95340 landed, Miri with `-Zmiri-check-number-validity` produces an error on the test suites of some crates which implement concurrency tools<sup>*</sup>, because it seems like such crates tend to use `std::sync::mpsc` in their tests. This fixes the problem by storing pointer bytes in a pointer.
<sup>*</sup> I have so far seen errors in the test suites of `once_cell`, `parking_lot`, and `crossbeam-utils`.
(just updating the list for fun, idk)
Also `threadpool`, `async-lock`, `futures-timer`, `fragile`, `scoped_threadpool`, `procfs`, `slog-async`, `scheduled-thread-pool`, `tokio-threadpool`, `mac`, `futures-cpupool`, `ntest`, `actix`, `zbus`, `jsonrpc-client-transports`, `fail`, `libp2p-gossipsub`, `parity-send-wrapper`, `async-broadcast,` `libp2p-relay`, `http-client`, `mockito`, `simple-mutex`, `surf`, `pollster`, and `pulse`. Then I turned the bot off.
Since https://github.com/rust-lang/rust/pull/95340 landed, Miri with
-Zmiri-check-number-validity produces an error on the test suites of
some crates which implement concurrency tools, because it seems like
such crates tend to use std::sync::mpsc in their tests. This fixes the
problem by storing pointer bytes in a pointer.
Relevant commit messages from squashed history in order:
Add initial version of ThinBox
update test to actually capture failure
swap to middle ptr impl based on matthieu-m's design
Fix stack overflow in debug impl
The previous version would take a `&ThinBox<T>` and deref it once, which
resulted in a no-op and the same type, which it would then print causing
an endless recursion. I've switched to calling `deref` by name to let
method resolution handle deref the correct number of times.
I've also updated the Drop impl for good measure since it seemed like it
could be falling prey to the same bug, and I'll be adding some tests to
verify that the drop is happening correctly.
add test to verify drop is behaving
add doc examples and remove unnecessary Pointee bounds
ThinBox: use NonNull
ThinBox: tests for size
Apply suggestions from code review
Co-authored-by: Alphyr <47725341+a1phyr@users.noreply.github.com>
use handle_alloc_error and fix drop signature
update niche and size tests
add cfg for allocating APIs
check null before calculating offset
add test for zst and trial usage
prevent optimizer induced ub in drop and cleanup metadata gathering
account for arbitrary size and alignment metadata
Thank you nika and thomcc!
Update library/alloc/src/boxed/thin.rs
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
Update library/alloc/src/boxed/thin.rs
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
make windows compat_fn (crudely) work on Miri
With https://github.com/rust-lang/rust/pull/95469, Windows `compat_fn!` now has to be supported by Miri to even make stdout work. Unfortunately, it relies on some outside-of-Rust linker hacks (`#[link_section = ".CRT$XCU"]`) that are rather hard to make work in Miri. So I came up with this crude hack to make this stuff work in Miri regardless. It should come at no cost for regular executions, so I hope this is okay.
Cc https://github.com/rust-lang/rust/issues/95627 `@ChrisDenton`
Bump bootstrap compiler to 1.61.0 beta
This PR bumps the bootstrap compiler to the 1.61.0 beta. The first commit changes the stage0 compiler, the second commit applies the "mechanical" changes and the third and fourth commits apply changes explained in the relevant comments.
r? `@Mark-Simulacrum`
Update libc to 0.2.121
With the updated libc, UNIX stack overflow handling in libstd can now
use the common `si_addr` accessor function, rather than attempting to
use a field from that name in `siginfo_t`. This simplifies the
collection of the fault address, particularly on platforms where that
data resides within a union in `siginfo_t`.
Don't cast thread name to an integer for prctl
`libc::prctl` and the `prctl` definitions in glibc, musl, and the kernel headers are C variadic functions. Therefore, all the arguments (except for the first) are untyped. It is only the Linux man page which says that `prctl` takes 4 `unsigned long` arguments. I have no idea why it says this.
In any case, the upshot is that we don't need to cast the pointer to an integer and confuse Miri.
But in light of this... what are we doing with those three `0`s? We're passing 3 `i32`s to `prctl`, which doesn't fill me with confidence. The man page says `unsigned long` and all the constants in the linux kernel are macros for expressions of the form `1UL << N`. I'm mostly commenting on this because looks a whole lot like some UB that was found in SQLite a few years ago: <https://youtu.be/LbzbHWdLAI0?t=1925> that was related to accidentally passing a 32-bit value from a literal `0` instead of a pointer-sized value. This happens to work on x86 due to the size of pointers and happens to work on x86_64 due to the calling convention. But also, there is no good reason for an implementation to be looking at those arguments. Some other calls to `prctl` require that other arguments be zeroed, but not `PR_SET_NAME`... so why are we even passing them?
I would prefer to end such questions by either passing 3 `libc::c_ulong`, or not passing those at all, but I'm not sure which is better.
Fix unsound `File` methods
This is a draft attempt to fix#81357. *EDIT*: this PR now tackles `read()`, `write()`, `read_at()`, `write_at()` and `read_buf`. Still needs more testing though.
cc `@jstarks,` can you confirm the the Windows team is ok with the Rust stdlib using `NtReadFile` and `NtWriteFile`?
~Also, I'm provisionally using `CancelIo` in a last ditch attempt to recover but I'm not sure that this is actually a good idea. Especially as getting into this state would be a programmer error so aborting the process is justified in any case.~ *EDIT*: removed, see comments.
With the updated libc, UNIX stack overflow handling in libstd can now
use the common `si_addr` accessor function, rather than attempting to
use a field from that name in `siginfo_t`. This simplifies the
collection of the fault address, particularly on platforms where that
data resides within a union in `siginfo_t`.
Windows: Synchronize asynchronous pipe reads and writes
On Windows, the pipes used for spawned processes are opened for asynchronous access but `read` and `write` are done using the standard methods that assume synchronous access. This means that the buffer (and variables on the stack) may be read/written to after the function returns.
This PR ensures reads/writes complete before returning. Note that this only applies to pipes we create and does not affect the standard file read/write methods.
Fixes#95411
libc::prctl and the prctl definitions in glibc, musl, and the kernel
headers are C variadic functions. Therefore, all the arguments (except
for the first) are untyped. It is only the Linux man page which says
that prctl takes 4 unsigned long arguments. I have no idea why it says
this.
In any case, the upshot is that we don't need to cast the pointer to an
integer and confuse Miri.
Handle rustc_const_stable attribute in library feature collector
The library feature collector in [compiler/rustc_passes/src/lib_features.rs](551b4fa395/compiler/rustc_passes/src/lib_features.rs) has only been looking at `#[stable(…)]`, `#[unstable(…)]`, and `#[rustc_const_unstable(…)]` attributes, while ignoring `#[rustc_const_stable(…)]`. The consequences of this were:
- When any const feature got stabilized (changing one or more `rustc_const_unstable` to `rustc_const_stable`), users who had previously enabled that unstable feature using `#![feature(…)]` would get told "unknown feature", rather than rustc's nicer "the feature … has been stable since … and no longer requires an attribute to enable".
This can be seen in the way that https://github.com/rust-lang/rust/pull/93957#issuecomment-1079794660 failed after rebase:
```console
error[E0635]: unknown feature `const_ptr_offset`
--> $DIR/offset_from_ub.rs:1:35
|
LL | #![feature(const_ptr_offset_from, const_ptr_offset)]
| ^^^^^^^^^^^^^^^^
```
- We weren't enforcing that a particular feature is either stable everywhere or unstable everywhere, and that a feature that has been stabilized has the same stabilization version everywhere, both of which we enforce for the other stability attributes.
This PR updates the library feature collector to handle `rustc_const_stable`, and fixes places in the standard library and test suite where `rustc_const_stable` was being used in a way that does not meet the rules for a stability attribute.
- Refine the "NaN as a special value" top level explanation of f32
- Refine `const NAN` docstring.
- Refine `fn is_sign_positive` and `fn is_sign_negative` docstrings.
- Refine `fn min` and `fn max` docstrings.
- Refine `fn trunc` docstrings.
- Refine `fn powi` docstrings.
- Refine `fn copysign` docstrings.
- Reword `NaN` and `NAN` as plain "NaN", unless they refer to the specific `const NAN`.
- Reword "a number" to `self` in function docstrings to clarify.
- Remove "Returns NAN if the number is NAN" as this is told to be the default behavior in the top explanation.
- Remove "propagating NaNs", as full propagation (preservation of payloads) is not guaranteed.
allow arbitrary inherent impls for builtin types in core
Part of https://github.com/rust-lang/compiler-team/issues/487. Slightly adjusted after some talks with `@m-ou-se` about the requirements of `t-libs-api`.
This adds a crate attribute `#![rustc_coherence_is_core]` which allows arbitrary impls for builtin types in core.
For other library crates impls for builtin types should be avoided if possible. We do have to allow the existing stable impls however. To prevent us from accidentally adding more of these in the future, there is a second attribute `#[rustc_allow_incoherent_impl]` which has to be added to **all impl items**. This only supports impls for builtin types but can easily be extended to additional types in a future PR.
This implementation does not check for overlaps in these impls. Perfectly checking that requires us to check the coherence of these incoherent impls in every crate, as two distinct dependencies may add overlapping methods. It should be easy enough to detect if it goes wrong and the attribute is only intended for use inside of std.
The first two commits are mostly unrelated cleanups.
Strict Provenance MVP
This patch series examines the question: how bad would it be if we adopted
an extremely strict pointer provenance model that completely banished all
int<->ptr casts.
The key insight to making this approach even *vaguely* pallatable is the
ptr.with_addr(addr) -> ptr
function, which takes a pointer and an address and creates a new pointer
with that address and the provenance of the input pointer. In this way
the "chain of custody" is completely and dynamically restored, making the
model suitable even for dynamic checkers like CHERI and Miri.
This is not a formal model, but lots of the docs discussing the model
have been updated to try to the *concept* of this design in the hopes
that it can be iterated on.
See #95228
Ensure io::Error's bitpacked repr doesn't accidentally impl UnwindSafe
Sadly, I'm not sure how to easily test that we don't impl a trait, though (or can libstd use `where io::Error: !UnwindSafe` or something).
Fixes#95203
Stabilize Termination and ExitCode
From https://github.com/rust-lang/rust/issues/43301
This PR stabilizes the Termination trait and associated ExitCode type. It also adjusts the ExitCode feature flag to replace the placeholder flag with a more permanent name, as well as splitting off the `to_i32` method behind its own permanently unstable feature flag.
This PR stabilizes the termination trait with the following signature:
```rust
pub trait Termination {
fn report(self) -> ExitCode;
}
```
The existing impls of `Termination` are effectively already stable due to the prior stabilization of `?` in main.
This PR also stabilizes the following APIs on exit code
```rust
#[derive(Clone, Copy, Debug)]
pub struct ExitCode(_);
impl ExitCode {
pub const SUCCESS: ExitCode;
pub const FAILURE: ExitCode;
}
impl From<u8> for ExitCode { /* ... */ }
```
---
All of the previous blockers have been resolved. The main ones that were resolved recently are:
* The trait's name: We decided against changing this since none of the alternatives seemed particularly compelling. Instead we decided to end the bikeshedding and stick with the current name. ([link to the discussion](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/Termination.2FExit.20Status.20Stabilization/near/269793887))
* Issues around platform specific representations: We resolved this issue by changing the return type of `report` from `i32` to the opaque type `ExitCode`. That way we can change the underlying representation without affecting the API, letting us offer full support for platform specific exit code APIs in the future.
* Custom exit codes: We resolved this by adding `From<u8> for ExitCode`. We choose to only support u8 initially because it is the least common denominator between the sets of exit codes supported by our current platforms. In the future we anticipate adding platform specific extension traits to ExitCode for constructors from larger or negative numbers, as needed.
Fix build on i686-apple-darwin systems
Replace `target_arch = "x86_64"` with `not(target_arch = "aarch64")` so that i686-apple-darwin systems dynamically choose implementation.
Move std::sys::{mutex, condvar, rwlock} to std::sys::locks.
This cleans up the the std::sys modules a bit by putting the locks in a single module called `locks` rather than spread over the three modules `mutex`, `condvar`, and `rwlock`. This makes it easier to organise lock implementations, which helps with https://github.com/rust-lang/rust/issues/93740.
Fold aarch64 feature +fp into +neon
Arm's FEAT_FP and Feat_AdvSIMD describe the same thing on AArch64:
The Neon unit, which handles both floating point and SIMD instructions.
Moreover, a configuration for AArch64 must include both or neither.
Arm says "entirely proprietary" toolchains may omit floating point:
https://developer.arm.com/documentation/102374/0101/Data-processing---floating-point
In the Programmer's Guide for Armv8-A, Arm says AArch64 can have
both FP and Neon or neither in custom implementations:
https://developer.arm.com/documentation/den0024/a/AArch64-Floating-point-and-NEON
In "Bare metal boot code for Armv8-A", enabling Neon and FP
is just disabling the same trap flag:
https://developer.arm.com/documentation/dai0527/a
In an unlikely future where "Neon and FP" become unrelated,
we can add "[+-]fp" as its own feature flag.
Until then, we can simplify programming with Rust on AArch64 by
folding both into "[+-]neon", which is valid as it supersets both.
"[+-]neon" is retained for niche uses such as firmware, kernels,
"I just hate floats", and so on.
I am... pretty sure no one is relying on this.
An argument could be made that, as we are not an "entirely proprietary" toolchain, we should not support AArch64 without floats at all. I think that's a bit excessive. However, I want to recognize the intent: programming for AArch64 should be simplified where possible. For x86-64, programmers regularly set up illegal feature configurations because it's hard to understand them, see https://github.com/rust-lang/rust/issues/89586. And per the above notes, plus the discussion in https://github.com/rust-lang/rust/issues/86941, there should be no real use cases for leaving these features split: the two should in fact always go together.
- Fixesrust-lang/rust#95002.
- Fixesrust-lang/rust#95064.
- Fixesrust-lang/rust#95122.
Arm's FEAT_FP and Feat_AdvSIMD describe the same thing on AArch64:
The Neon unit, which handles both floating point and SIMD instructions.
Moreover, a configuration for AArch64 must include both or neither.
Arm says "entirely proprietary" toolchains may omit floating point:
https://developer.arm.com/documentation/102374/0101/Data-processing---floating-point
In the Programmer's Guide for Armv8-A, Arm says AArch64 can have
both FP and Neon or neither in custom implementations:
https://developer.arm.com/documentation/den0024/a/AArch64-Floating-point-and-NEON
In "Bare metal boot code for Armv8-A", enabling Neon and FP
is just disabling the same trap flag:
https://developer.arm.com/documentation/dai0527/a
In an unlikely future where "Neon and FP" become unrelated,
we can add "[+-]fp" as its own feature flag.
Until then, we can simplify programming with Rust on AArch64 by
folding both into "[+-]neon", which is valid as it supersets both.
"[+-]neon" is retained for niche uses such as firmware, kernels,
"I just hate floats", and so on.
Preserve the Windows `GetLastError` error in `HandleOrInvalid`.
In the `TryFrom<HandleOrInvalid> for OwnedHandle` and
`TryFrom<HandleOrNull> for OwnedHandle` implemenations, `forget` the
owned handle on the error path, to avoid calling `CloseHandle` on an
invalid handle. It's harmless, except that it may overwrite the
thread's `GetLastError` error.
r? `@joshtriplett`
In the `TryFrom<HandleOrInvalid> for OwnedHandle` and
`TryFrom<HandleOrNull> for OwnedHandle` implemenations, `forget` the
owned handle on the error path, to avoid calling `CloseHandle` on an
invalid handle. It's harmless, except that it may overwrite the
thread's `GetLastError` error.
Skip a test if symlink creation is not possible
If someone running tests on Windows does not have Developer Mode enabled then creating symlinks will fail which in turn would cause this test to fail. This can be a stumbling block for contributors.
remove_dir_all: use fallback implementation on Miri
Fixes https://github.com/rust-lang/miri/issues/1966
The new implementation requires `openat`, `unlinkat`, and `fdopendir`. These cannot easily be shimmed in Miri since libstd does not expose APIs corresponding to them. So for now it is probably easiest to just use the fallback code in Miri. Nobody should run Miri as root anyway...
Relax tests for Windows dos device names
Windows 11 no longer turn paths ending with dos device names into device paths.
E.g. `C:\path\to\COM1.txt` used to get turned into `\\.\COM1`. Whereas now this path is left as is.
Note though that if the given path is an exact (case-insensitive) match for the string `COM1` then it'll still be converted to `\\.\COM1`.
Add a `process_group` method to UNIX `CommandExt`
- Tracking issue: #93857
- RFC: https://github.com/rust-lang/rfcs/pull/3228
Add a `process_group` method to `std::os::unix::process::CommandExt` that
allows setting the process group id (i.e. calling `setpgid`) in the child, thus
enabling users to set process groups while leveraging the `posix_spawn` fast
path.
add `CStr` method that accepts any slice containing a nul-terminated string
I haven't created an issue (tracking or otherwise) for this yet; apologies if my approach isn't correct. This is my first code contribution.
This change adds a member fn that converts a slice into a `CStr`; it is intended to be safer than `from_ptr` (which is unsafe and may read out of bounds), and more useful than `from_bytes_with_nul` (which requires that the caller already know where the nul byte is).
The reason I find this useful is for situations like this:
```rust
let mut buffer = [0u8; 32];
unsafe {
some_c_function(buffer.as_mut_ptr(), buffer.len());
}
let result = CStr::from_bytes_with_nul(&buffer).unwrap();
```
This code above returns an error with `kind = InteriorNul`, because `from_bytes_with_nul` expects that the caller has passed in a slice with the NUL byte at the end of the slice. But if I just got back a nul-terminated string from some FFI function, I probably don't know where the NUL byte is.
I would wish for a `CStr` constructor with the following properties:
- Accept `&[u8]` as input
- Scan for the first NUL byte and return the `CStr` that spans the correct sub-slice (see [future note below](https://github.com/rust-lang/rust/pull/94984#issuecomment-1070754281)).
- Return an error if no NUL byte is found within the input slice
I asked on [Zulip](https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/CStr.20from.20.26.5Bu8.5D.20without.20knowing.20the.20NUL.20location.3F) whether this sounded like a good idea, and got a couple of positive-sounding responses from ``@joshtriplett`` and ``@AzureMarker.``
This is my first draft, so feedback is welcome.
A few issues that definitely need feedback:
1. Naming. ``@joshtriplett`` called this `from_bytes_with_internal_nul` on Zulip, but after staring at all of the available methods, I believe that this function is probably what end users want (rather than the existing fn `from_bytes_with_nul`). Giving it a simpler name (**`from_bytes`**) implies that this should be their first choice.
2. Should I add a similar method on `CString` that accepts `Vec<u8>`? I'd assume the answer is probably yes, but I figured I'd try to get early feedback before making this change bigger.
3. What should the error type look like? I made a unit struct since `CStr::from_bytes` can only fail in one obvious way, but if I need to do this for `CString` as well then that one may want to return `FromVecWithNulError`. And maybe that should dictate the shape of the `CStr` error type also?
Also, cc ``@poliorcetics`` who wrote #73139 containing similar fns.
Consistently present absent stdio handles on Windows as NULL handles.
This addresses #90964 by making the std API consistent about presenting
absent stdio handles on Windows as NULL handles. Stdio handles may be
absent due to `#![windows_subsystem = "windows"]`, due to the console
being detached, or due to a child process having been launched from a
parent where stdio handles are absent.
Specifically, this fixes the case of child processes of parents with absent
stdio, which previously ended up with `stdin().as_raw_handle()` returning
`INVALID_HANDLE_VALUE`, which was surprising, and which overlapped with an
unrelated valid handle value. With this patch, `stdin().as_raw_handle()`
now returns null in these situation, which is consistent with what it
does in the parent process.
And, document this in the "Windows Portability Considerations" sections of
the relevant documentation.
Implement `Write for Cursor<[u8; N]>`, plus `A: Allocator` cursor support
This implements `Write for Cursor<[u8; N]>`, and also adds support for generic `A: Allocator` in `Box` and `Vec` cursors.
This was inspired by a user questioning why they couldn't write a `Cursor<[u8; N]>`:
https://users.rust-lang.org/t/why-vec-and-not-u8-makes-cursor-have-write/68210
Related history:
- #27197 switched `AsRef<[u8]>` for reading and seeking
- #67415 tried to use `AsMut<[u8]>` for writing, but did not specialize `Vec`.
Update stdlib for the l4re target
This PR contains the work by ``@humenda`` and myself to update standard library support for the x86_64-unknown-l4re-uclibc tier 3 target, split out from humenda/rust as requested in #85967. The changes have been rebased on current master and updated in follow up commits by myself. The publishing of the changes is authorized and preferred by the original author. To preserve attribution, when standard library changes were introduced as part of other changes to the compiler, I have kept the changes concerning the standard library and altered the commit messages as indicated. Any incompatibilities have been remedied in follow up commits, so that the PR as a whole should result in a clean update of the target.
Use verbatim paths for `process::Command` if necessary
In #89174, the standard library started using verbatim paths so longer paths are usable by default. However, `Command` was originally left out because of the way `CreateProcessW` was being called. This was changed as a side effect of #87704 so now `Command` paths can be converted to verbatim too (if necessary).
This adds a member fn that converts a slice into a CStr; it is intended
to be safer than from_ptr (which is unsafe and may read out of bounds),
and more useful than from_bytes_with_nul (which requires that the caller
already know where the nul byte is).
feature gate: cstr_from_bytes_until_nul
Also add an error type FromBytesUntilNulError for this fn.
Implement -Z oom=panic
This PR removes the `#[rustc_allocator_nounwind]` attribute on `alloc_error_handler` which allows it to unwind with a panic instead of always aborting. This is then used to implement `-Z oom=panic` as per RFC 2116 (tracking issue #43596).
Perf and binary size tests show negligible impact.
Fix for localized windows editions in testcase fn read_link() Issue#93211
This PR aims to fix the issue with localized windows versions that do not necessarily have the folder "Documents and settings" in English.
The idea was provided by `@the8472.` We check if the "CI" environment variable is set, then we always check for the "Documents and Settings"-folder, otherwise we check if the folder exists on the local machine, and if not we skip this assert.
Resoles #93211.
Improve doc wording for retain on some collections
I found the documentation wording on the various retain methods on many collections to be unusual.
I tried to invert the relation by switching `such that` with `for which` .
Use modern formatting for format! macros
This updates the standard library's documentation to use the new format_args syntax.
The documentation is worthwhile to update as it should be more idiomatic
(particularly for features like this, which are nice for users to get acquainted
with). The general codebase is likely more hassle than benefit to update: it'll
hurt git blame, and generally updates can be done by folks updating the code if
(and when) that makes things more readable with the new format.
A few places in the compiler and library code are updated (mostly just due to
already having been done when this commit was first authored).
`eprintln!("{}", e)` becomes `eprintln!("{e}")`, but `eprintln!("{}", e.kind())` remains untouched.
This updates the standard library's documentation to use the new syntax. The
documentation is worthwhile to update as it should be more idiomatic
(particularly for features like this, which are nice for users to get acquainted
with). The general codebase is likely more hassle than benefit to update: it'll
hurt git blame, and generally updates can be done by folks updating the code if
(and when) that makes things more readable with the new format.
A few places in the compiler and library code are updated (mostly just due to
already having been done when this commit was first authored).
Merge `#[deprecated]` and `#[rustc_deprecated]`
The first commit makes "reason" an alias for "note" in `#[rustc_deprecated]`, while still prohibiting it in `#[deprecated]`.
The second commit changes "suggestion" to not just be a feature of `#[rustc_deprecated]`. This is placed behind the new `deprecated_suggestion` feature. This needs a tracking issue; let me know if this PR will be approved and I can create one.
The third commit is what permits `#[deprecated]` to be used when `#![feature(staged_api)]` is enabled. This isn't yet used in stdlib (only tests), as it would require duplicating all deprecation attributes until a bootstrap occurs. I intend to submit a follow-up PR that replaces all uses and removes the remaining `#[rustc_deprecated]` code after the next bootstrap.
`@rustbot` label +T-libs-api +C-feature-request +A-attributes +S-waiting-on-review
Ignore `close_read_wakes_up` test on SGX platform
PR #94714 enabled the `close_read_wakes_up` test for all platforms. This is incorrect. This test should be ignored at least for the SGX platform.
cc: ``@mzohreva`` ``@jethrogb``
The initial stdlib modifications for L4Re just used the linux specifics
directly because they were reasonably close to L4Re's behavior.
However, this breaks when Linux-specific code relies on code that is
only available for the linux target, such as in #81825.
Put L4Re into its own platform to avoid such breakage in the future.
This uses the Linux-specific code as a starting point, which seems to be
in line with other OSes with a unix-y interface such as Fuchsia.
L4Re provides limited POSIX support which includes support for
standard I/O streams, and a limited implementation of the standard file
handling API. However, because as a capability based OS it strives to
only make a local view available to each application, there are
currently no standardized special files like /dev/null that could serve
to sanitize closed standard FDs.
For now, skip any attempts to sanitize standard streams until a more
complete POSIX runtime is available.
unix: reduce the size of DirEntry
On platforms where we call `readdir` instead of `readdir_r`, we store
the name as an allocated `CString` for variable length. There's no point
carrying around a full `dirent64` with its fixed-length `d_name` too.
Reverted atomic_mut_ptr feature removal causing compilation break
Fixes a regression introduced as part of https://github.com/rust-lang/rust/pull/94546
Std no longer compiles on nightly while using the following commnd:
export RUSTFLAGS='-C target-feature=+atomics,+bulk-memory'
cargo build --target wasm32-unknown-unknown -Z build-std=panic_abort,std
I can help add tests to avoid future breaks but i couldn't understand the test framework
unix: Avoid name conversions in `remove_dir_all_recursive`
Each recursive call was creating an `OsString` for a `&Path`, only for
it to be turned into a `CString` right away. Instead we can directly
pass `.name_cstr()`, saving two allocations each time.
Enable `close_read_wakes_up` test on Windows
I wonder if we could/should try enabling this again? It was closed by #38867 due to #31657. I've tried running this test (along with other tests) on my machine a number of times and haven't seen this fail yet,
Caveat: the worst that can happen is this succeeds initially but then causes random hangs in CI. This is not a great failure mode and would be a reason not to do this.
If this does work out, closes#39006
r? `@Mark-Simulacrum`
On platforms where we call `readdir` instead of `readdir_r`, we store
the name as an allocated `CString` for variable length. There's no point
carrying around a full `dirent64` with its fixed-length `d_name` too.
Remove argument from closure in thread::Scope::spawn.
This implements ```@danielhenrymantilla's``` [suggestion](https://github.com/rust-lang/rust/issues/93203#issuecomment-1040798286) for improving the scoped threads interface.
Summary:
The `Scope` type gets an extra lifetime argument, which represents basically its own lifetime that will be used in `&'scope Scope<'scope, 'env>`:
```diff
- pub struct Scope<'env> { .. };
+ pub struct Scope<'scope, 'env: 'scope> { .. }
pub fn scope<'env, F, T>(f: F) -> T
where
- F: FnOnce(&Scope<'env>) -> T;
+ F: for<'scope> FnOnce(&'scope Scope<'scope, 'env>) -> T;
```
This simplifies the `spawn` function, which now no longer passes an argument to the closure you give it, and now uses the `'scope` lifetime for everything:
```diff
- pub fn spawn<'scope, F, T>(&'scope self, f: F) -> ScopedJoinHandle<'scope, T>
+ pub fn spawn<F, T>(&'scope self, f: F) -> ScopedJoinHandle<'scope, T>
where
- F: FnOnce(&Scope<'env>) -> T + Send + 'env,
+ F: FnOnce() -> T + Send + 'scope,
- T: Send + 'env;
+ T: Send + 'scope;
```
The only difference the user will notice, is that their closure now takes no arguments anymore, even when spawning threads from spawned threads:
```diff
thread::scope(|s| {
- s.spawn(|_| {
+ s.spawn(|| {
...
});
- s.spawn(|s| {
+ s.spawn(|| {
...
- s.spawn(|_| ...);
+ s.spawn(|| ...);
});
});
```
<details><summary>And, as a bonus, errors get <em>slightly</em> better because now any lifetime issues point to the outermost <code>s</code> (since there is only one <code>s</code>), rather than the innermost <code>s</code>, making it clear that the lifetime lasts for the entire <code>thread::scope</code>.
</summary>
```diff
error[E0373]: closure may outlive the current function, but it borrows `a`, which is owned by the current function
--> src/main.rs:9:21
|
- 7 | s.spawn(|s| {
- | - has type `&Scope<'1>`
+ 6 | thread::scope(|s| {
+ | - lifetime `'1` appears in the type of `s`
9 | s.spawn(|| println!("{:?}", a)); // might run after `a` is dropped
| ^^ - `a` is borrowed here
| |
| may outlive borrowed value `a`
|
note: function requires argument type to outlive `'1`
--> src/main.rs:9:13
|
9 | s.spawn(|| println!("{:?}", a)); // might run after `a` is dropped
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
help: to force the closure to take ownership of `a` (and any other referenced variables), use the `move` keyword
|
9 | s.spawn(move || println!("{:?}", a)); // might run after `a` is dropped
| ++++
"
```
</details>
The downside is that the signature of `scope` and `Scope` gets slightly more complex, but in most cases the user wouldn't need to write those, as they just use the argument provided by `thread::scope` without having to name its type.
Another downside is that this does not work nicely in Rust 2015 and Rust 2018, since in those editions, `s` would be captured by reference and not by copy. In those editions, the user would need to use `move ||` to capture `s` by copy. (Which is what the compiler suggests in the error.)
Each recursive call was creating an `OsString` for a `&Path`, only for
it to be turned into a `CString` right away. Instead we can directly
pass `.name_cstr()`, saving two allocations each time.
Unix path::absolute: Fix leading "." component
Testing leading `.` and `..` components were missing from the unix tests.
This PR adds them and fixes the leading `.` case. It also fixes the test cases so that they do an exact comparison.
This problem reported by ``@axetroy``
Windows 11 no longer turn paths ending with dos device names into device paths.
E.g. `C:\path\to\COM1.txt` used to get turned into `\\.\COM1`. Whereas now the path is left as is.
UNIX `remove_dir_all()`: Try recursing first on the slow path
This only affects the _slow_ code path - if there is no `dirent.d_type` or if it is `DT_UNKNOWN`.
POSIX specifies that calling `unlink()` or `unlinkat(..., 0)` on a directory is allowed to succeed:
> The _path_ argument shall not name a directory unless the process has appropriate privileges and the implementation supports using _unlink()_ on directories.
This however can cause dangling inodes requiring an fsck e.g. on Illumos UFS, so we have to avoid that in the common case. We now just try to recurse into it first and unlink() if we can't open it as a directory.
The other two commits integrate the Macos x86-64 implementation reducing redundancy. Split into two commits for better reviewing.
Fixes#94335.
Rollup of 5 pull requests
Successful merges:
- #94362 (Add well known values to `--check-cfg` implementation)
- #94577 (only disable SIMD for doctests in Miri (not for the stdlib build itself))
- #94595 (Fix invalid `unresolved imports` errors for a single-segment import)
- #94596 (Delay bug in expr adjustment when check_expr is called multiple times)
- #94618 (Don't round stack size up for created threads in Windows)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Enable conditional compilation checking on the Rust codebase
This pull-request enable conditional compilation checking on every rust project build by the `bootstrap` tool.
To be more specific, this PR only enable well known names checking + extra names (bootstrap, parallel_compiler, ...).
r? `@Mark-Simulacrum`
Add #[track_caller] to track callers when initializing poisoned Once
This PR is for this Issue.
https://github.com/rust-lang/rust/issues/87707
With this fix, we expect to be able to track the caller when poisoned Once is initialized.
Before calling `CreateProcessW`, stdio handles are passed through
`stdio::get_handle`, which already converts NULL to
`INVALID_HANDLE_VALUE`, so we don't need extra checks for NULL after
that point.
This addresses #90964 by making the std API consistent about presenting
absent stdio handles on Windows as NULL handles. Stdio handles may be
absent due to `#![windows_subsystem = "windows"]`, due to the console
being detached, or due to a child process having been launched from a
parent where stdio handles are absent.
Specifically, this fixes the case of child processes of parents with absent
stdio, which previously ended up with `stdin().as_raw_handle()` returning
`INVALID_HANDLE_VALUE`, which was surprising, and which overlapped with an
unrelated valid handle value. With this patch, `stdin().as_raw_handle()`
now returns null in these situation, which is consistent with what it
does in the parent process.
And, document this in the "Windows Portability Considerations" sections of
the relevant documentation.
This only affects the `slow` code path, if there is no `dirent.d_type` or if
the type is `DT_UNKNOWN`.
POSIX specifies that calling `unlink()` or `unlinkat(..., 0)` on a directory can
succeed:
> "The _path_ argument shall not name a directory unless the process has
> appropriate privileges and the implementation supports using _unlink()_ on
> directories."
This however can cause orphaned directories requiring an fsck e.g. on Illumos
UFS, so we have to avoid that in the common case. We now just try to recurse
into it first and unlink() if we can't open it as a directory.
Use `HandleOrNull` and `HandleOrInvalid` in the Windows FFI bindings.
Use the new `HandleOrNull` and `HandleOrInvalid` types that were introduced
as part of [I/O safety] in a few functions in the Windows FFI bindings.
This factors out an `unsafe` block and two `unsafe` function calls in the
Windows implementation code.
And, it helps test `HandleOrNull` and `HandleOrInvalid`, and indeed, it
turned up a bug: `OwnedHandle` also needs to be `#[repr(transparent)]`,
as it's used inside of `HandleOrNull` and `HandleOrInvalid` which are also
`#[repr(transparent)]`.
r? ```@joshtriplett```
[I/O safety]: https://github.com/rust-lang/rust/issues/87074
Make regular stdio lock() return 'static handles
This also deletes the unstable API surface area previously added to expose this
functionality on new methods rather than built into the current set.
Closes#86845 (tracking issue for unstable API needed without this)
r? ``````@dtolnay`````` to kick off T-libs-api FCP
Clarification of default socket flags
This PR outlines the decision to disable inheritance of socket objects when possible to child processes in the documentation.
Use the new `HandleOrNull` and `HandleOrInvalid` types that were introduced
as part of [I/O safety] in a few functions in the Windows FFI bindings.
This factors out an `unsafe` block and two `unsafe` function calls in the
Windows implementation code.
And, it helps test `HandleOrNull` and `HandleOrInvalid`, which indeed turned
up a bug: `OwnedHandle` also needs to be `#[repr(transparent)]`, as it's
used inside of `HandleOrNull` and `HandleOrInvalid` which are also
`#[repr(transparent)]`.
[I/O safety]: https://github.com/rust-lang/rust/issues/87074
Update the documentation for `{As,Into,From}Raw{Fd,Handle,Socket}`.
This change weakens the descriptions of the
`{as,into,from}_raw_{fd,handle,socket}` descriptions from saying that
they *do* express ownership relations to say that they are *typically used*
in ways that express ownership relations. This is needed since, for
example, std's own [`RawFd`] implements `{As,From,Into}Fd` without any of
the ownership relationships.
This adds proper `# Safety` comments to `from_raw_{fd,handle,socket}`,
adds the requirement that raw handles be not opened with the
`FILE_FLAG_OVERLAPPED` flag, and merges the `OwnedHandle::from_raw_handle`
comment into the main `FromRawHandle::from_raw_handle` comment.
And, this changes `HandleOrNull` and `HandleOrInvalid` to not implement
`FromRawHandle`, since they are intended for limited use in FFI situations,
and not for generic use, and they have constraints that are stronger than
the those of `FromRawHandle`.
[`RawFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/type.RawFd.html
There may eventually be something to say about `FILE_FLAG_OVERLAPPED` here,
however this appears to be independent of the other changes in this PR,
so remove them from this PR so that it can be discussed separately.
Rename `BorrowedFd::borrow_raw_fd` to `BorrowedFd::borrow_raw`.
Also, rename `BorrowedHandle::borrow_raw_handle` and
`BorrowedSocket::borrow_raw_socket` to `BorrowedHandle::borrow_raw` and
`BorrowedSocket::borrow_raw`.
This is just a minor rename to reduce redundancy in the user code calling
these functions, and to eliminate an inessential difference between
`BorrowedFd` code and `BorrowedHandle`/`BorrowedSocket` code.
While here, add a simple test exercising `BorrowedFd::borrow_raw_fd`.
r? ``````@joshtriplett``````
Add documentation about `BorrowedFd::to_owned`.
Following up on #88564, this adds documentation explaining why
`BorrowedFd::to_owned` returns another `BorrowedFd` rather than an
`OwnedFd`. And similar for `BorrowedHandle` and `BorrowedSocket`.
r? `````@joshtriplett`````
this avoids parsing mountinfo which can be huge on some systems and
something might be emulating cgroup fs for sandboxing reasons which means
it wouldn't show up as mountpoint
additionally the new implementation operates on a single pathbuffer, reducing allocations
Manually tested via
```
// spawn a new cgroup scope for the current user
$ sudo systemd-run -p CPUQuota="300%" --uid=$(id -u) -tdS
// quota.rs
#![feature(available_parallelism)]
fn main() {
println!("{:?}", std:🧵:available_parallelism()); // prints Ok(3)
}
```
Caveats
* cgroup v1 is ignored
* funky mountpoints (containing spaces, newlines or control chars) for cgroupfs will not be handled correctly since that would require unescaping /proc/self/mountinfo
The escaping behavior of procfs seems to be undocumented. systemd and docker default to `/sys/fs/cgroup` so it should be fine for most systems.
* quota will be ignored when `sched_getaffinity` doesn't work
* assumes procfs is mounted under `/proc` and cgroupfs mounted and readable somewhere in the directory tree
The ability to interoperate with C code via FFI is not limited to crates
using std; this allows using these types without std.
The existing types in `std::os::raw` become type aliases for the ones in
`core::ffi`. This uses type aliases rather than re-exports, to allow the
std types to remain stable while the core types are unstable.
This also moves the currently unstable `NonZero_` variants and
`c_size_t`/`c_ssize_t`/`c_ptrdiff_t` types to `core::ffi`, while leaving
them unstable.
use BOOL for TCP_NODELAY setsockopt value on Windows
This issue was found by the Wine project and mitigated there [^1].
Windows' setsockopt expects a BOOL (a typedef for int) for TCP_NODELAY
[^2]. Windows itself is forgiving and will accept any positive optlen and
interpret the first byte of *optval as the value, so this bug does not
affect Windows itself, but does affect systems implementing Windows'
interface more strictly, such as Wine. Wine was previously passing this
through to the host's setsockopt, where, e.g., Linux requires that
optlen be correct for the chosen option, and TCP_NODELAY expects an int.
[^1]: d6ea38f32d
[^2]: https://docs.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-setsockopt
Some improvements to the async docs
The goal here is to make the docs overall a little bit more comprehensive and add more links between the things.
One thing that's not working yet is the links to the keywords. Somehow I couldn't get them to work.
r? ````@GuillaumeGomez```` do you know how I could get the keyword links to work?
This issue was found by the Wine project and mitigated there [1].
Windows' documented interface for `setsockopt` expects a `BOOL` (a
`typedef` for `int`) for `TCP_NODELAY` [2]. Windows is forgiving and
will accept any positive length and interpret the first byte of
`*option_value` as the value, so this bug does not affect Windows
itself, but does affect systems implementing Windows' interface more
strictly, such as Wine. Wine was previously passing this through to the
host's `setsockopt`, where, e.g., Linux requires that `option_len` be
correct for the chosen option, and `TCP_NODELAY` expects an `int`.
[1]: d6ea38f32d
[2]: https://docs.microsoft.com/en-us/windows/win32/api/winsock/nf-winsock-setsockopt
POSIX allows `getsockopt` to set `*option_len` to a smaller value if
necessary. Windows will set `*option_len` to 1 for boolean options even
when the caller passes a `BOOL` (`int`) with `*option_len` as 4.
Previously `level` was named `opt` and `option_name` was named `val`,
then extra names of `payload` or `slot` were used for the option value.
This change aligns the wrapper parameters with their names in POSIX.
Winsock uses similar but more abbreviated names: `level`, `optname`,
`optval`, `optlen`.
Fix miniz_oxide types showing up in std docs
Fixes#90526.
Thanks to ```````@camelid,``````` I rediscovered `doc(masked)`, allowing us to prevent `miniz_oxide` type to show up in std docs.
r? ```````@notriddle```````
removing architecture requirements for RustyHermit
RustHermit and HermitCore is able to run on aarch64 and x86_64. In the future these operating systems will also support RISC-V. Consequently, the dependency to a specific target should be removed.
The build process of `hermit-abi` fails if the architecture isn't supported.
Add debug assertions to validate NUL terminator in c strings
The `unchecked` variants from the stdlib usually perform the check anyway if debug assertions are on (for example, `unwrap_unchecked`). This PR does the same thing for `CStr` and `CString`, validating the correctness for the NUL byte in debug mode.
Destabilise entry_insert
See: https://github.com/rust-lang/rust/pull/90345
I didn't revert the rename that was done in that PR, I left it as `entry_insert`.
Additionally, before that PR, `VacantEntry::insert_entry` seemingly had no stability attribute on it? I kept the attribute, just made it an unstable one, same as the one on `Entry`.
There didn't seem to be any mention of this in the RELEASES.md, so I don't think there's anything for me to do other than this?
kmc-solid: Use the filesystem thread-safety wrapper
Fixes the thread unsafety of the `std::fs` implementation used by the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets.
Neither the SOLID filesystem API nor built-in filesystem drivers guarantee thread safety by default. Although this may suffice in general embedded-system use cases, and in fact the API can be used from multiple threads without any problems in many cases, this has been a source of unsoundness in `std::sys::solid::fs`.
This commit updates the implementation to leverage the filesystem thread-safety wrapper (which uses a pluggable synchronization mechanism) to enforce thread safety. This is done by prefixing all paths passed to the filesystem API with `\TS`. (Note that relative paths aren't supported in this platform.)
Add MAIN_SEPARATOR_STR
Currently, if someone needs access to the path separator as a str, they need to go through this mess:
```rust
unsafe {
std::str::from_utf8_unchecked(slice::from_ref(&(MAIN_SEPARATOR as u8)))
}
```
This PR just re-exports an existing path separator str API.
Add documentation to more `From::from` implementations.
For users looking at documentation through IDE popups, this gives them relevant information rather than the generic trait documentation wording “Performs the conversion”. For users reading the documentation for a specific type for any reason, this informs them when the conversion may allocate or copy significant memory versus when it is always a move or cheap copy.
Notes on specific cases:
* The new documentation for `From<T> for T` explains that it is not a conversion at all.
* Also documented `impl<T, U> Into<U> for T where U: From<T>`, the other central blanket implementation of conversion.
* The new documentation for construction of maps and sets from arrays of keys mentions the handling of duplicates. Future work could be to do this for *all* code paths that convert an iterable to a map or set.
* I did not add documentation to conversions of a specific error type to a more general error type.
* I did not add documentation to unstable code.
This change was prepared by searching for the text "From<... for" and so may have missed some cases that for whatever reason did not match. I also looked for `Into` impls but did not find any worth documenting by the above criteria.
RustHermit and HermitCore is able to run on aarch64 and x86_64.
In the future these operating systems will also support RISC-V.
Consequently, the dependency to a specific target should be removed.
Building hermit-abi fails if the architecture isn't supported.
make `Instant::{duration_since, elapsed, sub}` saturating and remove workarounds
This removes all mutex/atomic-based workarounds for non-monotonic clocks and makes the previously panicking methods saturating instead. Additionally `saturating_duration_since` becomes deprecated since `duration_since` now fills that role.
Effectively this moves the fixup from `Instant` construction to the comparisons.
This has some observable effects, especially on platforms without monotonic clocks:
* Incorrectly ordered Instant comparisons no longer panic in release mode. This could hide some programming errors, but since debug mode still panics tests can still catch them.
* `checked_duration_since` will now return `None` in more cases. Previously it only happened when one compared instants obtained in the wrong order or manually created ones. Now it also does on backslides.
* non-monotonic intervals will not be transitive, i.e. `b.duration_since(a) + c.duration_since(b) != c.duration_since(a)`
The upsides are reduced complexity and lower overhead of `Instant::now`.
## Motivation
Currently we must choose between two poisons. One is high worst-case latency and jitter of `Instant::now()` due to explicit synchronization; see #83093 for benchmarks, the worst-case overhead is > 100x. The other is sporadic panics on specific, rare combinations of CPU/hypervisor/operating system due to platform bugs.
Use-cases where low-overhead, fine-grained timestamps are needed - such as syscall tracing, performance profiles or sensor data acquisition (drone flight controllers were mentioned in a libs meeting) in multi-threaded programs - are negatively impacted by the synchronization.
The panics are user-visible (program crashes), hard to reproduce and can be triggered by any dependency that might be using Instants for any reason.
A solution that is fast _and_ doesn't panic is desirable.
----
closes#84448closes#86470
This removes all mutex/atomics based workarounds for non-monotonic clocks and makes the previously panicking methods saturating instead.
Effectively this moves the monotonization from `Instant` construction to the comparisons.
This has some observable effects, especially on platforms without monotonic clocks:
* Incorrectly ordered Instant comparisons no longer panic. This may hide some programming errors until someone actually looks at the resulting `Duration`
* `checked_duration_since` will now return `None` in more cases. Previously it only happened when one compared instants obtained in the wrong order or
manually created ones. Now it also does on backslides.
The upside is reduced complexity and lower overhead of `Instant::now`.
Fix hashing for windows paths containing a CurDir component
* the logic only checked for / but not for \
* verbatim paths shouldn't skip items at all since they don't get normalized
* the extra branches get optimized out on unix since is_sep_byte is a trivial comparison and is_verbatim is always-false
* tests lacked windows coverage for these cases
That lead to equal paths not having equal hashes and to unnecessary collisions.
Rollup of 10 pull requests
Successful merges:
- #90955 (Rename `FilenameTooLong` to `InvalidFilename` and also use it for Windows' `ERROR_INVALID_NAME`)
- #91607 (Make `span_extend_to_prev_str()` more robust)
- #92895 (Remove some unused functionality)
- #93635 (Add missing platform-specific information on current_dir and set_current_dir)
- #93660 (rustdoc-json: Add some tests for typealias item)
- #93782 (Split `pauth` target feature)
- #93868 (Fix incorrect register conflict detection in asm!)
- #93888 (Implement `AsFd` for `&T` and `&mut T`.)
- #93909 (Fix typo: explicitely -> explicitly)
- #93910 (fix mention of moved function in `rustc_hir` docs)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Implement `AsFd` for `&T` and `&mut T`.
Add implementations of `AsFd` for `&T` and `&mut T`, so that users can
write code like this:
```rust
pub fn fchown<F: AsFd>(fd: F, uid: Option<u32>, gid: Option<u32>) -> io::Result<()> {
```
with `fd: F` rather than `fd: &F`.
And similar for `AsHandle` and `AsSocket` on Windows.
Also, adjust the `fchown` example to pass the file by reference. The
code can work either way now, but passing by reference is more likely
to be what users will want to do.
This is an alternative to #93869, and is a simpler way to achieve the
same goals: users don't need to pass borrowed-`BorrowedFd` arguments,
and it prevents a pitfall in the case where users write `fd: F` instead
of `fd: &F`.
r? ```@joshtriplett```
Rename `FilenameTooLong` to `InvalidFilename` and also use it for Windows' `ERROR_INVALID_NAME`
Address https://github.com/rust-lang/rust/issues/90940#issuecomment-970157931
`ERROR_INVALID_NAME` (i.e. "The filename, directory name, or volume label syntax is incorrect") happens if we pass an invalid filename, directory name, or label syntax, so mapping as `InvalidInput` is reasonable to me.
Stabilise `is_aarch64_feature_detected!` under `simd_aarch64` feature
Initial implementation, looking for feedback on the approach here. https://github.com/rust-lang/rust/issues/86941
One point I noticed was that I haven't seen different "since" versions for the same feature - does this mean that other features can't be added to to the `simd_aarch64` feature once this is in stable? If so it might need a more specific name.
r? `@Amanieu`
Add implementations of `AsFd` for `&T` and `&mut T`, so that users can
write code like this:
```rust
pub fn fchown<F: AsFd>(fd: F, uid: Option<u32>, gid: Option<u32>) -> io::Result<()> {
```
with `fd: F` rather than `fd: &F`.
And similar for `AsHandle` and `AsSocket` on Windows.
Also, adjust the `fchown` example to pass the file by reference. The
code can work either way now, but passing by reference is more likely
to be what users will want to do.
This is an alternative to #93869, and is a simpler way to achieve the
same goals: users don't need to pass borrowed-`BorrowedFd` arguments,
and it prevents a pitfall in the case where users write `fd: F` instead
of `fd: &F`.
kmc-solid: Fix wait queue manipulation errors in the `Condvar` implementation
This PR fixes a number of bugs in the `Condvar` wait queue implementation used by the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets. These bugs can occur when there are multiple threads waiting on the same `Condvar` and sometimes manifest as an `unwrap` failure.
Neither the SOLID filesystem API nor built-in filesystems guarantee
thread safety by default. Although this may suffice in general embedded-
system use cases, and in fact the API can be used from multiple threads
without any problems in many cases, this has been a source of
unsoundness in `std::sys::solid::fs`.
This commit updates the `std` code to leverage the filesystem thread-
safety wrapper to enforce thread safety. This is done by prefixing all
paths passed to the filesystem API with `\TS`. (Note that relative paths
aren't supported in this platform.)
Use `NtCreateFile` instead of `NtOpenFile` to open a file
Generally the internal `Nt*` functions should be avoided but when we do need to use one we should stick to the most commonly used for the job. To that end, this PR replaces `NtOpenFile` with `NtCreateFile`.
NOTE: The initial version of this comment hypothesised that this may help with some recent false positives from malware scanners. This hypothesis proved wrong. Sorry for the distraction.
Make io::Error use 64 bits on targets with 64 bit pointers.
I've wanted this for a long time, but didn't see a good way to do it without having extra allocation. When looking at it yesterday, it was more clear what to do for some reason.
This approach avoids any additional allocations, and reduces the size by half (8 bytes, down from 16). AFAICT it doesn't come additional runtime cost, and the compiler seems to do a better job with code using it.
Additionally, this `io::Error` has a niche (still), so `io::Result<()>` is *also* 64 bits (8 bytes, down from 16), and `io::Result<usize>` (used for lots of io trait functions) is 2x64 bits (16 bytes, down from 24 — this means on x86_64 it can use the nice rax/rdx 2-reg struct return). More generally, it shaves a whole 64 bit integer register off of the size of basically any `io::Result<()>`.
(For clarity: Improving `io::Result` (rather than io::Error) was most of the motivation for this)
On 32 bit (or other non-64bit) targets we still use something equivalent the old repr — I don't think think there's improving it, since one of the fields it stores is a `i32`, so we can't get below that, and it's already about as close as we can get to it.
---
### Isn't Pointer Tagging Dodgy?
The details of the layout, and why its implemented the way it is, are explained in the header comment of library/std/src/io/error/repr_bitpacked.rs. There's probably more details than there need to be, but I didn't trim it down that much, since there's a lot of stuff I did deliberately, that might have not seemed that way.
There's actually only one variant holding a pointer which gets tagged. This one is the (holder for the) user-provided error.
I believe the scheme used to tag it is not UB, and that it preserves pointer provenance (even though often pointer tagging does not) because the tagging operation is just `core::ptr::add`, and untagging is `core::ptr::sub`. The result of both operations lands inside the original allocation, so it would follow the safety contract of `core::ptr::{add,sub}`.
The other pointer this had to encode is not tagged — or rather, the tagged repr is equivalent to untagged (it's tagged with 0b00, and has >=4b alignment, so we can reuse the bottom bits). And the other variants we encode are just integers, which (which can be untagged using bitwise operations without worry — they're integers).
CC `@RalfJung` for the stuff in repr_bitpacked.rs, as my comments are informed by a lot of the UCG work, but it's possible I missed something or got it wrong (even if the implementation is okay, there are parts of the header comment that says things like "We can't do $x" which could be false).
---
### Why So Many Changes?
The repr change was mostly internal, but changed one widely used API: I had to switch how `io::Error::new_const` works.
This required switching `io::Error::new_const` to take the full message data (including the kind) as a `&'static`, rather than just the string. This would have been really tedious, but I made a macro that made it much simpler, but it was a wide change since `io::Error::new_const` is used everywhere.
This included changing files for a lot of targets I don't have easy access to (SGX? Haiku? Windows? Who has heard of these things), so I expect there to be spottiness in CI initially, unless luck is on my side.
Anyway this large only tangentially-related change is all in the first commit (although that commit also pulls the previous repr out into its own file), whereas the packing stuff is all in commit 2.
---
P.S. I haven't looked at all of this since writing it, and will do a pass over it again later, sorry for any obvious typos or w/e. I also definitely repeat myself in comments and such.
(It probably could use more tests too. I did some basic testing, and made it so we `debug_assert!` in cases the decode isn't what we encoded, but I don't know the degree which I can assume libstd's testing of IO would exercise this. That is: it wouldn't be surprising to me if libstds IO testing were minimal, especially around error cases, although I have no idea).
* the logic only checked for / but not for \
* verbatim paths shouldn't skip items at all since they don't get normalized
* the extra branches get optimized out on unix since is_sep_byte is a trivial comparison and is_verbatim is always-false
* tests lacked windows coverage for these cases
That lead to equal paths not having equal hashes and to unnecessary collisions.
Also, rename `BorrowedHandle::borrow_raw_handle` and
`BorrowedSocket::borrow_raw_socket` to `BorrowedHandle::borrow_raw` and
`BorrowedSocket::borrow_raw`.
This is just a minor rename to reduce redundancy in the user code calling
these functions, and to eliminate an inessential difference between
`BorrowedFd` code and `BorrowedHandle`/`BorrowedSocket` code.
While here, add a simple test exercising `BorrowedFd::borrow_raw_fd`.
kmc-solid: Fix off-by-one error in `SystemTime::now`
Fixes a miscalculation of `SystemTime` on the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets.
Unlike the identically-named libc counterpart `tm::tm_mon`, `SOLID_RTC_TIME::tm_mon` contains a 1-based month number.
Link `try_exists` docs to `Path::exists`
Links to the existing `Path::exists` method from both `std::Path::try_exists` and `std::fs:try_exists`.
Tracking issue for `path_try_exists`: #83186
pub use std::simd::StdFloat;
Syncs portable-simd up to commit rust-lang/portable-simd@03f6fbb21e,
Diff: 533f0fc81a...03f6fbb21e
This sync requires a little bit more legwork because it also introduces a trait into `std::simd`, so that it is no longer simply a reexport of `core::simd`. Out of simple-minded consistency and to allow more options, I replicated the pattern for the way `core::simd` is integrated in the first place, however this is not necessary if it doesn't acquire any interdependencies inside `std`: it could be a simple crate reexport. I just don't know yet if that will happen or not.
To summarize other misc changes:
- Shifts no longer panic, now wrap on too-large shifts (like `Simd` integers usually do!)
- mask16x32 will now be many i16s, not many i32s... 🙃
- `#[must_use]` is spread around generously
- Adjusts division, float min/max, and `Mask::{from,to}_array` internally to be faster
- Adds the much-requested `Simd::cast::<U>` function (equivalent to `simd.to_array().map(|lane| lane as U)`)
Support configuring whether to capture backtraces at runtime
Tracking issue: https://github.com/rust-lang/rust/issues/93346
This adds a new API to the `std::panic` module which configures whether and how the default panic hook will emit a backtrace when a panic occurs.
After discussion with `@yaahc` on [Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/backtrace.20lib.20vs.2E.20panic), this PR chooses to avoid adjusting or seeking to provide a similar API for the (currently unstable) std::backtrace API. It seems likely that the users of that API may wish to expose more specific settings rather than just a global one (e.g., emulating the `env_logger`, `tracing` per-module configuration) to avoid the cost of capture in hot code. The API added here could plausibly be copied and/or re-exported directly from std::backtrace relatively easily, but I don't think that's the right call as of now.
```rust
mod panic {
#[derive(Copy, Clone, Debug, PartialEq, Eq)]
#[non_exhaustive]
pub enum BacktraceStyle {
Short,
Full,
Off,
}
fn set_backtrace_style(BacktraceStyle);
fn get_backtrace_style() -> Option<BacktraceStyle>;
}
```
Several unresolved questions:
* Do we need to move to a thread-local or otherwise more customizable strategy for whether to capture backtraces? See [this comment](https://github.com/rust-lang/rust/pull/79085#issuecomment-727845826) for some potential use cases for this.
* Proposed answer: no, leave this for third-party hooks.
* Bikeshed on naming of all the options, as usual.
* Should BacktraceStyle be moved into `std::backtrace`?
* It's already somewhat annoying to import and/or re-type the `std::panic::` prefix necessary to use these APIs, probably adding a second module to the mix isn't worth it.
Note that PR #79085 proposed a much simpler API, but particularly in light of the desire to fully replace setting environment variables via `env::set_var` to control the backtrace API, a more complete API seems preferable. This PR likely subsumes that one.
Fix incorrect panic message in example
The panic message when calling the `connect()` should probably be a message about connection failure, not a message about binding address failure.
Document valid values of the char type
As discussed at #93392, the current documentation on what constitutes a valid char isn't very detailed and is partly on the MAX constant rather than the type itself.
This PR expands on that information, stating the actual numerical range, giving examples of what won't work, and also mentions how a `char` might be a valid USV but still not be a defined character (terminology checked against [Unicode 14.0, table 2-3](https://www.unicode.org/versions/Unicode14.0.0/ch02.pdf#M9.61673.TableTitle.Table.22.Types.of.Code.Points)).
This change weakens the descriptions of the
`{as,into,from}_raw_{fd,handle,socket}` descriptions from saying that
they *do* express ownership relations to say that they are *typically used*
in ways that express ownership relations. This needed needed since, for
example, std's own [`RawFd`] implements `{As,From,Into}Fd` without any of
the ownership relationships.
This adds proper `# Safety` comments to `from_raw_{fd,handle,socket}`,
adds the requirement that raw handles be not opened with the
`FILE_FLAG_OVERLAPPED` flag, and merges the `OwnedHandle::from_raw_handle`
comment into the main `FromRawHandle::from_raw_handle` comment.
And, this changes `HandleOrNull` and `HandleOrInvalid` to not implement
`FromRawHandle`, since they are intended for limited use in FFI situations,
and not for generic use, and they have constraints that are stronger than
the those of `FromRawHandle`.
[`RawFd`]: https://doc.rust-lang.org/stable/std/os/unix/io/type.RawFd.html
Change Termination::report return type to ExitCode
Related to https://github.com/rust-lang/rust/issues/43301
The goal of this change is to minimize the forward compatibility risks in stabilizing Termination. By using the opaque type `ExitCode` instead of an `i32` we leave room for us to evolve the API over time to provide what cross-platform consistency we can / minimize footguns when working with exit codes, where as stabilizing on `i32` would limit what changes we could make in the future in how we represent and construct exit codes.
kmc-solid: Increase the default stack size
This PR increases the default minimum stack size on the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets to 64KiB (Arm) and 128KiB (AArch64).
This value was chosen as a middle ground between supporting a relatively complex program (e.g., an application using a full-fledged off-the-shelf web server framework) with no additional configuration and minimizing resource consumption for the embedded platform that doesn't support lazily-allocated pages nor over-commitment (i.e., wasted stack spaces are wasted physical memory). If the need arises, the users can always set the `RUST_MIN_STACK` environmental variable to override the default stack size or use the platform API directly.
Errors from pthread_sigmask(3) were handled using cvt(), which expects a
return value of -1 on error and uses errno.
However, pthread_sigmask(3) returns 0 on success and an error number
otherwise.
Fix it by replacing cvt() with cvt_nz().
kmc-solid: Inherit the calling task's base priority in `Thread::new`
This PR fixes the initial priority calculation of spawned threads on the [`*-kmc-solid_*`](https://doc.rust-lang.org/nightly/rustc/platform-support/kmc-solid.html) Tier 3 targets.
Fixes a spawned task (an RTOS object on top of which threads are implemented for this target; unrelated to async tasks) getting an unexpectedly higher priority if it's spawned by a task whose priority is temporarily boosted by a priority-protection mutex.
unix: Use metadata for `DirEntry::file_type` fallback
When `DirEntry::file_type` fails to match a known `d_type`, we should
fall back to `DirEntry::metadata` instead of a bare `lstat`, because
this is faster and more reliable on targets with `fstatat`.
Fixes a spawned task getting an unexpectedly higher priority if it's
spawned by a task whose priority is temporarily boosted by a priority-
protection mutex.
When `DirEntry::file_type` fails to match a known `d_type`, we should
fall back to `DirEntry::metadata` instead of a bare `lstat`, because
this is faster and more reliable on targets with `fstatat`.
fs: Don't copy d_name from struct dirent
The dirent returned from readdir() is only guaranteed to be valid for
d_reclen bytes on common platforms. Since we copy the name separately
anyway, we can copy everything except d_name into DirEntry::entry.
Fixes#93384.
Move unstable is_{arch}_feature_detected! macros to std::arch
These macros are unstable, except for `is_x86_feature_detected` which is still exported from the crate root for backwards-compatibility.
This should unblock the stabilization of `is_aarch64_feature_detected`.
r? ```@m-ou-se```
The dirent returned from readdir() is only guaranteed to be valid for
d_reclen bytes on common platforms. Since we copy the name separately
anyway, we can copy everything except d_name into DirEntry::entry.
Fixes#93384.
kmc-solid: Implement `net::FileDesc::duplicate`
This PR implements `std::sys::solid::net::FileDesc::duplicate`, which was accidentally left out when this target was added by #86191.
Bump libc and fix remove_dir_all on Fuchsia after CVE fix
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Depends on https://github.com/rust-lang/libc/pull/2654 since we
apparently haven't needed to reference DT_UNKNOWN before this.
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Updates std's libc crate to include DT_UNKNOWN for Fuchsia.
Avoid double panics when using `TempDir` in tests
`TempDir` could panic on drop if `remove_dir_all` returns an error. If this happens while already panicking, the test process would abort and therefore not show the test results.
This PR tries to avoid such double panics.
