Make `std:🧵:available_concurrency` support process-limited number of CPUs
Use `libc::sched_getaffinity` and count the number of CPUs in the returned mask. This handles cases where the process doesn't have access to all CPUs, such as when limited via `taskset` or similar.
This also covers cgroup cpusets.
This stabilizes dereferencing immutable raw pointers in const contexts.
It does not stabilize `*mut T` dereferencing. This is placed behind the
`const_raw_mut_ptr_deref` feature gate.
In #89522 we learned that `clone3` is interacting poorly with Gentoo's
`sandbox` tool. We only need that for the unstable pidfd extensions, so
otherwise avoid that and use a normal `fork`.
Replace `std::os::raw::c_ssize_t` with `std::os::raw::c_ptrdiff_t`
The discussions in #88345 brought up that `ssize_t` is not actually the signed index type defined in stddef.h, but instead it's `ptrdiff_t`. It seems pretty clear that the use of `ssize_t` here was a mistake on my part, and that if we're going to bother having a isize-alike for FFI in `std::os::raw`, it should be `ptrdiff_t` and not `ssize_t`.
Anyway, both this and `c_size_t` are dubious in the face of the discussion in https://internals.rust-lang.org/t/pre-rfc-usize-is-not-size-t/15369, and any RFC/project-group/etc that handles those issues there should contend with these types in some manner, but that doesn't mean we shouldn't fix something wrong like this, even if it is unstable.
All that said, `size_t` is *vastly* more common in function signatures than either `ssize_t` or `ptrdiff_t`, so I'm going to update the tracking issue's list of unresolved questions to note that perhaps we only want `c_size_t` — I mostly added the signed version for symmetry, rather than to meet a need. (Given this, I'm also fine with modifying this patch to instead remove `c_ssize_t` without a replacement)
CC `@magicant` (who brought the issue up)
CC `@chorman0773` (who has a significantly firmer grasp on the minutae of the C standard than I do)
r? `@joshtriplett` (original reviewer, active in the discussions around this)
Windows thread-local keyless drop
`#[thread_local]` allows us to maintain a per-thread list of destructors. This also avoids the need to synchronize global data (which is particularly tricky within the TLS callback function).
r? `@alexcrichton`
Add JoinHandle::is_running.
This adds:
```rust
impl<T> JoinHandle<T> {
/// Checks if the the associated thread is still running its main function.
///
/// This might return `false` for a brief moment after the thread's main
/// function has returned, but before the thread itself has stopped running.
pub fn is_running(&self) -> bool;
}
```
The usual way to check if a background thread is still running is to set some atomic flag at the end of its main function. We already do that, in the form of dropping an Arc which will reduce the reference counter. So we might as well expose that information.
This is useful in applications with a main loop (e.g. a game, gui, control system, ..) where you spawn some background task, and check every frame/iteration whether the background task is finished to .join() it in that frame/iteration while keeping the program responsive.
`#[thread_local]` allows us to maintain a per-thread list of destructors. This also avoids the need to synchronize global data (which is particularly tricky within the TLS callback function).
Add #[must_use] to remaining std functions (O-Z)
I've run out of compelling reasons to group functions together across crates so I'm just going to go module-by-module. This is half of the remaining items from the `std` crate, from O-Z.
`panicking::take_hook` has a side effect: it unregisters the current panic hook, returning it. I almost ignored it, but the documentation example shows `let _ = panic::take_hook();`, so following suit I went ahead and added a `#[must_use]`.
```rust
std::panicking fn take_hook() -> Box<dyn Fn(&PanicInfo<'_>) + 'static + Sync + Send>;
```
I added these functions that clippy did not flag:
```rust
std::path::Path fn starts_with<P: AsRef<Path>>(&self, base: P) -> bool;
std::path::Path fn ends_with<P: AsRef<Path>>(&self, child: P) -> bool;
std::path::Path fn with_file_name<S: AsRef<OsStr>>(&self, file_name: S) -> PathBuf;
std::path::Path fn with_extension<S: AsRef<OsStr>>(&self, extension: S) -> PathBuf;
```
Parent issue: #89692
r? `@joshtriplett`
Add #[must_use] to expensive computations
The unifying theme for this commit is weak, admittedly. I put together a list of "expensive" functions when I originally proposed this whole effort, but nobody's cared about that criterion. Still, it's a decent way to bite off a not-too-big chunk of work.
Given the grab bag nature of this commit, the messages I used vary quite a bit. I'm open to wording changes.
For some reason clippy flagged four `BTreeSet` methods but didn't say boo about equivalent ones on `HashSet`. I stared at them for a while but I can't figure out the difference so I added the `HashSet` ones in.
```rust
// Flagged by clippy.
alloc::collections::btree_set::BTreeSet<T> fn difference<'a>(&'a self, other: &'a BTreeSet<T>) -> Difference<'a, T>;
alloc::collections::btree_set::BTreeSet<T> fn symmetric_difference<'a>(&'a self, other: &'a BTreeSet<T>) -> SymmetricDifference<'a, T>
alloc::collections::btree_set::BTreeSet<T> fn intersection<'a>(&'a self, other: &'a BTreeSet<T>) -> Intersection<'a, T>;
alloc::collections::btree_set::BTreeSet<T> fn union<'a>(&'a self, other: &'a BTreeSet<T>) -> Union<'a, T>;
// Ignored by clippy, but not by me.
std::collections::HashSet<T, S> fn difference<'a>(&'a self, other: &'a HashSet<T, S>) -> Difference<'a, T, S>;
std::collections::HashSet<T, S> fn symmetric_difference<'a>(&'a self, other: &'a HashSet<T, S>) -> SymmetricDifference<'a, T, S>
std::collections::HashSet<T, S> fn intersection<'a>(&'a self, other: &'a HashSet<T, S>) -> Intersection<'a, T, S>;
std::collections::HashSet<T, S> fn union<'a>(&'a self, other: &'a HashSet<T, S>) -> Union<'a, T, S>;
```
Parent issue: #89692
r? ```@joshtriplett```
Stabilize `is_symlink()` for `Metadata` and `Path`
I'm not fully sure about `since` version, correct me if I'm wrong
Needs update after stabilization: [cargo-test-support](8063672238/crates/cargo-test-support/src/paths.rs (L202))
Linked issue: #85748
Use libc::sched_getaffinity and count the number of CPUs in the returned
mask. This handles cases where the process doesn't have access to all
CPUs, such as when limited via taskset or similar.
Automatically convert paths to verbatim for filesystem operations that support it
This allows using longer paths without the user needing to `canonicalize` or manually prefix paths. If the path is already verbatim then this has no effect.
Fixes: #32689
Rollup of 5 pull requests
Successful merges:
- #90239 (Consistent big O notation in map.rs)
- #90267 (fix: inner attribute followed by outer attribute causing ICE)
- #90288 (Add hint for people missing `TryFrom`, `TryInto`, `FromIterator` import pre-2021)
- #90304 (Add regression test for #75961)
- #90344 (Add tracking issue number to const_cstr_unchecked)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Clean up special function const checks
Mark them as const and `#[rustc_do_not_const_check]` instead of hard-coding them in const-eval checks.
r? `@oli-obk`
`@rustbot` label A-const-eval T-compiler
Remove fNN::lerp
Lerp is [surprisingly complex with multiple tradeoffs depending on what guarantees you want to provide](https://github.com/rust-lang/rust/issues/86269#issuecomment-869108301) (and what you're willing to drop for raw speed), so we don't have consensus on what implementation to use, let alone what signature - `t.lerp(a, b)` nicely puts `a, b` together, but makes dispatch to lerp custom types with the same signature basically impossible, and major ecosystem crates (e.g. nalgebra, glium) use `a.lerp(b, t)`, which is easily confusable. It was suggested to maybe provide a `Lerp<T>` trait and `t.lerp([a, b])`, which _could_ be implemented by downstream math libraries for their types, but also significantly raises the bar from a simple fNN method to a full trait, and does nothing to solve the implementation question. (It also raises the question of whether we'd support higher-order bezier interpolation.)
The only consensus we have is the lack of consensus, and the [general temperature](https://github.com/rust-lang/rust/issues/86269#issuecomment-951347135) is that we should just remove this method (giving the method space back to 3rd party libs) and revisit this if (and likely only if) IEEE adds lerp to their specification.
If people want a lerp, they're _probably_ already using (or writing) a math support library, which provides a lerp function for its custom math types and can provide the same lerp implementation for the primitive types via an extension trait.
See also [previous Zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/lerp.20API.20design)
cc ``@clarfonthey`` (original PR author), ``@m-ou-se`` (original r+), ``@scottmcm`` (last voice in tracking issue, prompted me to post this)
Closes#86269 (removed)
Stabilise unix_process_wait_more, extra ExitStatusExt methods
This stabilises the feature `unix_process_wait_more`. Tracking issue #80695, FCP needed.
This was implemented in #79982 and merged in January.
My change to use `Type::def_id()` (formerly `Type::def_id_full()`) in
more places caused some docs to show up that used to be missed by
rustdoc. Those docs contained unescaped square brackets, which triggered
linkcheck errors. This commit escapes the square brackets and adds this
particular instance to the linkcheck exception list.
Inline CStr::from_ptr
Inlining this function is valuable, as it allows LLVM to apply `strlen`-specific optimizations without having to enable LTO.
For instance, the following function:
```rust
pub fn f(p: *const c_char) -> Option<u8> {
unsafe { CStr::from_ptr(p) }.to_bytes().get(0).copied()
}
```
Looks like this if `CStr::from_ptr` is allowed to be inlined.
```asm
before:
push rax
call qword ptr [rip + std::ffi::c_str::CStr::from_ptr@GOTPCREL]
mov rcx, rax
cmp rdx, 1
sete dl
test rax, rax
sete al
or al, dl
jne .LBB1_2
mov dl, byte ptr [rcx]
.LBB1_2:
xor al, 1
pop rcx
ret
after:
mov dl, byte ptr [rdi]
test dl, dl
setne al
ret
```
Note that optimization turned this from O(N) to O(1) in terms of performance, as LLVM knows that it doesn't really need to call `strlen` to determine whether a string is empty or not.
Stabilize feature `saturating_div` for rust 1.58.0
The tracking issue is #89381
This seems like a reasonable simple change(?). The feature `saturating_div` was added as part of the ongoing effort to implement a `Saturating` integer type (see #87921). The implementation has been discussed [here](https://github.com/rust-lang/rust/pull/87921#issuecomment-899357720) and [here](https://github.com/rust-lang/rust/pull/87921#discussion_r691888556). It extends the list of saturating operations on integer types (like `saturating_add`, `saturating_sub`, `saturating_mul`, ...) by the function `fn saturating_div(self, rhs: Self) -> Self`.
The stabilization of the feature `saturating_int_impl` (for the `Saturating` type) needs to have this stabilized first.
Closes#89381
Previously, it wasn't clear whether "This could include" was referring
to logic errors, or undefined behaviour. Tweak wording to clarify this
sentence does not relate to UB.
Stabilize CString::from_vec_with_nul[_unchecked]
Closes the tracking issue #73179. I am keeping this in _draft_ mode until the FCP has ended.
This is my first time stabilizing a feature, so I would appreciate any guidance on things I should do differently.
Closes#73179
Remove unnecessary condition in Barrier::wait()
This is my first pull request for Rust, so feel free to call me out if anything is amiss.
After some examination, I realized that the second condition of the "spurious-wakeup-handler" loop in ``std::sync::Barrier::wait()`` should always evaluate to ``true``, making it redundant in the ``&&`` expression.
Here is the affected function before the fix:
```rust
#[stable(feature = "rust1", since = "1.0.0")]
pub fn wait(&self) -> BarrierWaitResult {
let mut lock = self.lock.lock().unwrap();
let local_gen = lock.generation_id;
lock.count += 1;
if lock.count < self.num_threads {
// We need a while loop to guard against spurious wakeups.
// https://en.wikipedia.org/wiki/Spurious_wakeup
while local_gen == lock.generation_id && lock.count < self.num_threads { // fixme
lock = self.cvar.wait(lock).unwrap();
}
BarrierWaitResult(false)
} else {
lock.count = 0;
lock.generation_id = lock.generation_id.wrapping_add(1);
self.cvar.notify_all();
BarrierWaitResult(true)
}
}
```
At first glance, it seems that the check that ``lock.count < self.num_threads`` would be necessary in order for a thread A to detect when another thread B has caused the barrier to reach its thread count, making thread B the "leader".
However, the control flow implicitly results in an invariant that makes observing ``!(lock.count < self.num_threads)``, i.e. ``lock.count >= self.num_threads`` impossible from thread A.
When thread B, which will be the leader, calls ``.wait()`` on this shared instance of the ``Barrier``, it locks the mutex in the first line and saves the ``MutexGuard`` in the ``lock`` variable. It then increments the value of ``lock.count``. However, it then proceeds to check if ``lock.count < self.num_threads``. Since it is the leader, it is the case that (after the increment of ``lock.count``), the lock count is *equal* to the number of threads. Thus, the second branch is immediately taken and ``lock.count`` is zeroed. Additionally, the generation ID is incremented (with wrap). Then, the condition variable is signalled. But, the other threads are waiting at the line ``lock = self.cvar.wait(lock).unwrap();``, so they cannot resume until thread B's call to ``Barrier::wait()`` returns, which drops the ``MutexGuard`` acquired in the first ``let`` statement and unlocks the mutex.
The order of events is thus:
1. A thread A calls `.wait()`
2. `.wait()` acquires the mutex, increments `lock.count`, and takes the first branch
3. Thread A enters the ``while`` loop since the generation ID has not changed and the count is less than the number of threads for the ``Barrier``
3. Spurious wakeups occur, but both conditions hold, so the thread A waits on the condition variable
4. This process repeats for N - 2 additional times for non-leader threads A'
5. *Meanwhile*, Thread B calls ``Barrier::wait()`` on the same barrier that threads A, A', A'', etc. are waiting on. The thread count reaches the number of threads for the ``Barrier``, so all threads should now proceed, with B being the leader. B acquires the mutex and increments the value ``lock.count`` only to find that it is not less than ``self.num_threads``. Thus, it immediately clamps ``self.num_threads`` back down to 0 and increments the generation. Then, it signals the condvar to tell the A (prime) threads that they may continue.
6. The A, A', A''... threads wake up and attempt to re-acquire the ``lock`` as per the internal operation of a condition variable. When each A has exclusive access to the mutex, it finds that ``lock.generation_id`` no longer matches ``local_generation`` **and the ``&&`` expression short-circuits -- and even if it were to evaluate it, ``self.count`` is definitely less than ``self.num_threads`` because it has been reset to ``0`` by thread B *before* B dropped its ``MutexGuard``**.
Therefore, it my understanding that it would be impossible for the non-leader threads to ever see the second boolean expression evaluate to anything other than ``true``. This PR simply removes that condition.
Any input would be appreciated. Sorry if this is terribly verbose. I'm new to the Rust community and concurrency can be hard to explain in words. Thanks!
Reject octal zeros in IPv4 addresses
This fixes#86964 by rejecting octal zeros in IP addresses, such that `192.168.00.00000000` is rejected with a parse error, since having leading zeros in front of another zero indicates it is a zero written in octal notation, which is not allowed in the strict mode specified by RFC 6943 3.1.1. Octal rejection was implemented in #83652, but due to the way it was implemented octal zeros were still allowed.
As a security measure, Windows 11 introduces a new temporary directory API, GetTempPath2.
When the calling process is running as SYSTEM, a separate temporary directory
will be returned inaccessible to non-SYSTEM processes. For non-SYSTEM processes
the behavior will be the same as before.
removing TLS support in x86_64-unknown-none-hermitkernel
HermitCore's kernel itself doesn't support TLS. Consequently, the entries in x86_64-unknown-none-hermitkernel should be removed. This commit should help to finalize #89062.
linux/aarch64 Now() should be actually_monotonic()
While issues have been seen on arm64 platforms the Arm architecture requires
that the counter monotonically increases and that it must provide a uniform
view of system time (e.g. it must not be possible for a core to receive a
message from another core with a time stamp and observe time going backwards
(ARM DDI 0487G.b D11.1.2). While there have been a few 64bit SoCs that have
bugs (#49281, #56940) which cause time to not monotonically increase, these have
been fixed in the Linux kernel and we shouldn't penalize all Arm SoCs for those
who refuse to update their kernels:
SUN50I_ERRATUM_UNKNOWN1 - Allwinner A64 / Pine A64 - fixed in 5.1
FSL_ERRATUM_A008585 - Freescale LS2080A/LS1043A - fixed in 4.10
HISILICON_ERRATUM_161010101 - Hisilicon 1610 - fixed in 4.11
ARM64_ERRATUM_858921 - Cortex A73 - fixed in 4.12
255a3f3e18 std: Force `Instant::now()` to be monotonic added a Mutex to work around
this problem and a small test program using glommio shows the majority of time spent
acquiring and releasing this Mutex. 3914a7b0da tries to improve this, but actually
makes it worse on big systems as for 128b atomics a ldxp/stxp pair (and successful loop)
for v8.4 systems that don't support FEAT_LSE2 is required which is expensive as a lock
and because of how the load/store-exclusives scale on large Arm systems is both unfair
to threads and tends to go backwards in performance.
A small sample program using glommio improves by 70x on a 32 core Graviton2
system with this change.
[fuchsia] Update process info struct
The fuchsia platform is in the process of softly transitioning over to
using a new value for ZX_INFO_PROCESS with a new corresponding struct.
This change migrates libstd.
See [fxrev.dev/510478](https://fxrev.dev/510478) and [fxbug.dev/30751](https://fxbug.dev/30751) for more detail.
Add abstract namespace support for Unix domain sockets
Hello! The other day I wanted to mess around with UDS in Rust and found that abstract namespaces ([unix(7)](https://man7.org/linux/man-pages/man7/unix.7.html)) on Linux still needed development. I took the approach of adding `_addr` specific public functions to reduce conflicts.
Feature name: `unix_socket_abstract`
Tracking issue: #85410
Further context: #42048
## Non-platform specific additions
`UnixListener::bind_addr(&SocketAddr) -> Result<UnixListener>`
`UnixStream::connect_addr(&SocketAddr) -> Result<()>`
`UnixDatagram::bind_addr(&SocketAddr) -> Result<UnixDatagram>`
`UnixDatagram::connect_addr(&SocketAddr) -> Result<()>`
`UnixDatagram::send_to_addr(&self, &[u8], &SocketAddr) -> Result<usize>`
## Platform-specific (Linux) additions
`SocketAddr::from_abstract_namespace(&[u8]) -> SocketAddr`
`SockerAddr::as_abstract_namespace() -> Option<&[u8]>`
## Example
```rust
#![feature(unix_socket_abstract)]
use std::os::unix::net::{UnixListener, SocketAddr};
fn main() -> std::io::Result<()> {
let addr = SocketAddr::from_abstract_namespace(b"namespace")?; // Linux only
let listener = match UnixListener::bind_addr(&addr) {
Ok(sock) => sock,
Err(err) => {
println!("Couldn't bind: {:?}", err);
return Err(err);
}
};
Ok(())
}
```
## Further Details
The main inspiration for the implementation came from the [nix-rust](https://github.com/nix-rust/nix/blob/master/src/sys/socket/addr.rs#L558) crate but there are also other [historical](c4db0685b1) [attempts](https://github.com/tormol/uds/blob/master/src/addr.rs#L324) with similar approaches.
A comment I did have was with this change, we now allow a `SocketAddr` to be constructed explicitly rather than just used almost as a handle for the return of `peer_addr` and `local_addr`. We could consider adding other explicit constructors (e.g. `SocketAddr::from_pathname`, `SockerAddr::from_unnamed`).
Cheers!
Use BCryptGenRandom instead of RtlGenRandom on Windows.
This removes usage of RtlGenRandom on Windows, in favour of BCryptGenRandom.
BCryptGenRandom isn't available on XP, but we dropped XP support a while ago.
The fuchsia platform is in the process of softly transitioning over to
using a new value for ZX_INFO_PROCESS with a new corresponding struct.
This change migrates libstd.
See fxrev.dev/510478 and fxbug.dev/30751 for more detail.
Fix ctrl-c causing reads of stdin to return empty on Windows.
Pressing ctrl+c (or ctrl+break) on Windows caused a blocking read of stdin to unblock and return empty, unlike other platforms which continue to block.
On ctrl-c, `ReadConsoleW` will return success, but also set `LastError` to `ERROR_OPERATION_ABORTED`.
This change detects this case, and re-tries the call to `ReadConsoleW`.
Fixes#89177. See issue for further details.
Tested on Windows 7 and Windows 10 with both MSVC and GNU toolchains
Improve `std:🧵:available_parallelism` docs
_Tracking issue: https://github.com/rust-lang/rust/issues/74479_
This PR reworks the documentation of `std:🧵:available_parallelism`, as requested [here](https://github.com/rust-lang/rust/pull/89324#issuecomment-934343254).
## Changes
The following changes are made:
- We've removed prior mentions of "hardware threads" and instead centers the docs around "parallelism" as a resource available to a program.
- We now provide examples of when `available_parallelism` may return numbers that differ from the number of CPU cores in the host machine.
- We now mention that the amount of available parallelism may change over time.
- We make note of which platform components we don't take into account which more advanced users may want to take note of.
- The example has been updated, which should be a bit easier to use.
- We've added a docs alias to `num-cpus` which provides similar functionality to `available_parallelism`, and is one of the most popular crates on crates.io.
---
Thanks!
r? `@BurntSushi`
The unifying theme for this commit is weak, admittedly. I put together a
list of "expensive" functions when I originally proposed this whole
effort, but nobody's cared about that criterion. Still, it's a decent
way to bite off a not-too-big chunk of work.
Given the grab bag nature of this commit, the messages I used vary quite
a bit.
Add #[must_use] to is_condition tests
I threw in `std::path::Path::has_root` for funsies.
A continuation of #89718.
Parent issue: #89692
r? ```@joshtriplett```
Add #[must_use] to non-mutating verb methods
These are methods that could be misconstrued to mutate their input, similar to #89694. I gave each one a different custom message.
I wrote that `upgrade` and `downgrade` don't modify the input pointers. Logically they don't, but technically they do...
Parent issue: #89692
r? ```@joshtriplett```
Add #[must_use] to from_value conversions
I added two methods to the list myself. Clippy did not flag them because they take `mut` args, but neither modifies their argument.
```rust
core::str const unsafe fn from_utf8_unchecked_mut(v: &mut [u8]) -> &mut str;
std::ffi::CString unsafe fn from_raw(ptr: *mut c_char) -> CString;
```
I put a custom note on `from_raw`:
```rust
#[must_use = "call `drop(from_raw(ptr))` if you intend to drop the `CString`"]
pub unsafe fn from_raw(ptr: *mut c_char) -> CString {
```
Parent issue: #89692
r? ``@joshtriplett``
Add new tier-3 target: armv7-unknown-linux-uclibceabihf
This change adds a new tier-3 target: armv7-unknown-linux-uclibceabihf
This target is primarily used in embedded linux devices where system resources are slim and glibc is deemed too heavyweight. Cross compilation C toolchains are available [here](https://toolchains.bootlin.com/) or via [buildroot](https://buildroot.org).
The change is based largely on a previous PR #79380 with a few minor modifications. The author of that PR was unable to push the PR forward, and graciously allowed me to take it over.
Per the [target tier 3 policy](https://github.com/rust-lang/rfcs/blob/master/text/2803-target-tier-policy.md), I volunteer to be the "target maintainer".
This is my first PR to Rust itself, so I apologize if I've missed things!
Add #[must_use] to string/char transformation methods
These methods could be misconstrued as modifying their arguments instead of returning new values.
Where possible I made the note recommend a method that does mutate in place.
Parent issue: #89692
Fix minor std::thread documentation typo
callers of spawn_unchecked() need to make sure that the thread
not outlive references in the passed closure, not the other way around.
Optimize File::read_to_end and read_to_string
Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity.
`std::fs::read` and `std::fs::read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the documentation for both the `Read` trait and the `Read::read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as `impl Read` or `dyn Read` will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked `unsafe`.
- `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added `@camelid's` suggested wording from https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502.
r? `@joshtriplett`
These methods could be misconstrued as modifying their arguments instead
of returning new values.
Where possible I made the note recommend a method that does mutate in
place.
Reading a file into an empty vector or string buffer can incur
unnecessary `read` syscalls and memory re-allocations as the buffer
"warms up" and grows to its final size. This is perhaps a necessary evil
with generic readers, but files can be read in smarter by checking the
file size and reserving that much capacity.
`std::fs::read` and `read_to_string` already perform this optimization:
they open the file, reads its metadata, and call `with_capacity` with
the file size. This ensures that the buffer does not need to be resized
and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls
`file.read_to_end` or `file.read_to_string` they do not get this
optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of
examples of this. They're found all over in unit tests, which isn't a
big deal, but there are also several real instances in the compiler and
in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the `Read` trait and the `read_to_end` method both show
this exact pattern as examples of how to use readers. What this says to
me is that this shouldn't be solved by simply fixing the instances of it
in this codebase. If it's here it's certain to be prevalent in the wider
Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and
`read_to_string` directly on `File`. This way it's no longer a minor
footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as
a bare `Read` constraint or via a `dyn Read` trait object will benefit.
For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to
`self.inner`'s methods. That way it can call `File`'s optimized
implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked
`unsafe`.
- `File::read_to_string` being more efficient means that the performance
note for `io::read_to_string` can be softened. I've added @camelid's
suggested wording from:
https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
Make cfg imply doc(cfg)
This is a reopening of #79341, rebased and modified a bit (we made a lot of refactoring in rustdoc's types so they needed to be reflected in this PR as well):
* `hidden_cfg` is now in the `Cache` instead of `DocContext` because `cfg` information isn't stored anymore on `clean::Attributes` type but instead computed on-demand, so we need this information in later parts of rustdoc.
* I removed the `bool_to_options` feature (which makes the code a bit simpler to read for `SingleExt` trait implementation.
* I updated the version for the feature.
There is only one thing I couldn't figure out: [this comment](https://github.com/rust-lang/rust/pull/79341#discussion_r561855624)
> I think I'll likely scrap the whole `SingleExt` extension trait as the diagnostics for 0 and >1 items should be different.
How/why should they differ?
EDIT: this part has been solved, the current code was fine, just needed a little simplification.
cc `@Nemo157`
r? `@jyn514`
Original PR description:
This is only active when the `doc_cfg` feature is active.
The implicit cfg can be overridden via `#[doc(cfg(...))]`, so e.g. to hide a `#[cfg]` you can use something like:
```rust
#[cfg(unix)]
#[doc(cfg(all()))]
pub struct Unix;
```
By adding `#![doc(cfg_hide(foobar))]` to the crate attributes the cfg `#[cfg(foobar)]` (and _only_ that _exact_ cfg) will not be implicitly treated as a `doc(cfg)` to render a message in the documentation.
Rename `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`
_Tracking issue: https://github.com/rust-lang/rust/issues/74479_
This PR renames `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`.
## Rationale
The API was initially named `std:🧵:hardware_concurrency`, mirroring the [C++ API of the same name](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency). We eventually decided to omit any reference to the word "hardware" after [this comment](https://github.com/rust-lang/rust/pull/74480#issuecomment-662045841). And so we ended up with `available_concurrency` instead.
---
For a talk I was preparing this week I was reading through ["Understanding and expressing scalable concurrency" (A. Turon, 2013)](http://aturon.github.io/academic/turon-thesis.pdf), and the following passage stood out to me (emphasis mine):
> __Concurrency is a system-structuring mechanism.__ An interactive system that deals with disparate asynchronous events is naturally structured by division into concurrent threads with disparate responsibilities. Doing so creates a better fit between problem and solution, and can also decrease the average latency of the system by preventing long-running computations from obstructing quicker ones.
> __Parallelism is a resource.__ A given machine provides a certain capacity for parallelism, i.e., a bound on the number of computations it can perform simultaneously. The goal is to maximize throughput by intelligently using this resource. For interactive systems, parallelism can decrease latency as well.
_Chapter 2.1: Concurrency is not Parallelism. Page 30._
---
_"Concurrency is a system-structuring mechanism. Parallelism is a resource."_ — It feels like this accurately captures the way we should be thinking about these APIs. What this API returns is not "the amount of concurrency available to the program" which is a property of the program, and thus even with just a single thread is effectively unbounded. But instead it returns "the amount of _parallelism_ available to the program", which is a resource hard-constrained by the machine's capacity (and can be further restricted by e.g. operating systems).
That's why I'd like to propose we rename this API from `available_concurrency` to `available_parallelism`. This still meets the criteria we previously established of not attempting to define what exactly we mean by "hardware", "threads", and other such words. Instead we only talk about "concurrency" as an abstract resource available to our program.
r? `@joshtriplett`
Use the 64b inner:monotonize() implementation not the 128b one for aarch64
aarch64 prior to v8.4 (FEAT_LSE2) doesn't have an instruction that guarantees
untorn 128b reads except for completing a 128b load/store exclusive pair
(ldxp/stxp) or compare-and-swap (casp) successfully. The requirement to
complete a 128b read+write atomic is actually more expensive and more unfair
than the previous implementation of monotonize() which used a Mutex on aarch64,
especially at large core counts. For aarch64 switch to the 64b atomic
implementation which is about 13x faster for a benchmark that involves many
calls to Instant::now().
path.push() should work as expected on windows verbatim paths
On Windows, std::fs::canonicalize() returns an so-called UNC path. UNC paths differ with regular paths because:
- This type of path can much longer than a non-UNC path (32k vs 260 characters).
- The prefix for a UNC path is ``Component::Prefix(Prefix::DiskVerbatim(..)))``
- No `/` is allowed
- No `.` is allowed
- No `..` is allowed
Rust has poor handling of such paths. If you join a UNC path with a path with any of the above, then this will not work.
I've implemented a new method `fn join_fold()` which joins paths and also removes any `.` and `..` from it, and replaces `/` with `\` on Windows. Using this function it is possible to use UNC paths without issue. In addition, this function is useful on Linux too; paths can be appended without having to call `canonicalize()` to remove the `.` and `..`.
This PR needs test cases, which can I add. I hope this will a start of a discussion.
Manual Debug for Unix ExitCode ExitStatus ExitStatusError
These structs have misleading names. An ExitStatus[Error] is actually a Unix wait status; an ExitCode is actually an exit status. These misleading names appear in the `Debug` output.
The `Display` impls on Unix have been improved, but the `Debug` impls are still misleading, as reported in #74832.
Fix this by pretending that these internal structs are called `unix_exit_status` and `unix_wait_status` as applicable. (We can't actually rename the structs because of the way that the cross-platform machinery works: the names are cross-platform.)
After this change, this program
```
#![feature(exit_status_error)]
fn main(){
let x = std::process::Command::new("false").status().unwrap();
dbg!(x.exit_ok());
eprintln!("x={:?}",x);
}
```
produces this output
```
[src/main.rs:4] x.exit_ok() = Err(
ExitStatusError(
unix_wait_status(
256,
),
),
)
x=ExitStatus(unix_wait_status(256))
```
Closes#74832
Remove unnecessary unsafe block in `process_unix`
Because it's nested under this unsafe fn!
This block isn't detected as unnecessary because of a bug in the compiler: #88260.
Add `Ipv6Addr::is_benchmarking`
This PR adds the unstable method `Ipv6Addr::is_benchmarking`. This method is added for parity with `Ipv4Addr::is_benchmarking`, and I intend to use it in a future rework of `Ipv6Addr::is_global` (edit: #86634) to more accurately follow the [IANA Special Address Registry](https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml) (like is done in `Ipv4Addr::is_global`).
With `Ipv6Addr::is_benchmarking` and `Ipv4Addr::is_benchmarking` now both existing, `IpAddr::is_benchmarking` is also added.
Fix read_to_end to not grow an exact size buffer
If you know how much data to expect and use `Vec::with_capacity` to pre-allocate a buffer of that capacity, `Read::read_to_end` will still double its capacity. It needs some space to perform a read, even though that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading into a small "probe" buffer instead. If that read returns `0` then it's confirmed that the buffer was the perfect size. If it doesn't, the probe buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in `initial_buffer_size`.
Curiously, there was a unit test that specifically checked that `Read::read_to_end` *does* over-allocate. I removed that test, too.
This allows using longer paths for filesystem operations without the user needing to `canonicalize` or manually prefix paths.
If the path is already verbatim than this has no effect.
On MinGW toolchains the various features (such as function sections)
necessary to eliminate dead function references are disabled due to
various bugs. This means that the windows sockets library will most
likely remain linked to any mingw toolchain built program that also
utilizes libstd.
That said, I made an attempt to also enable `function-sections` and
`--gc-sections` during my experiments, but the symbol references
remained, sadly.
Restructure std::rt
These changes should reduce binary size slightly while at the same slightly improving performance of startup, thread spawning and `std:🧵:current()`. I haven't verified if the compiler is able to optimize some of these cases already, but at least for some others the compiler is unable to do these optimizations as they slightly change behavior in cases where program startup would crash anyway by omitting a backtrace and panic location.
I can remove 6f6bb16 if preferred.
The reference automatically coerces to a pointer. Writing an explicit
cast here is slightly misleading because that's most commonly used when
a pointer needs to be converted from one pointer type to another, e.g.
`*const c_void` to `*const sigaction` or vice versa.
Add SOLID targets
This PR introduces new tier 3 targets for [SOLID](https://www.kmckk.co.jp/eng/SOLID/) embedded development platform by Kyoto Microcomputer Co., Ltd.
| Target name | `target_arch` | `target_vendor` | `target_os` |
|--------------------------------|---------------|-----------------|--------------|
| `aarch64-kmc-solid_asp3` | `aarch64` | `kmc` | `solid_asp3` |
| `armv7a-kmc-solid_asp3-eabi` | `arm` | `kmc` | `solid_asp3` |
| `armv7a-kmc-solid_asp3-eabihf` | `arm` | `kmc` | `solid_asp3` |
## Related PRs
- [ ] `libc`: https://github.com/rust-lang/libc/pull/2227
- [ ] `cc`: https://github.com/alexcrichton/cc-rs/pull/609
## Non-blocking Issues
- [ ] The target kernel can support `Thread::unpark` directly, but this property is not utilized because the underlying kernel feature is used to implement `Condvar` and it's unclear whether `std` should guarantee that parking tokens are not clobbered by other synchronization primitives.
- [ ] The rustc book: The page title "\*-kmc-solid-\*" shows up as "-kmc-solid-" in TOC
## Tier 3 Target Policy
As tier 3 targets, the new targets are required to adhere to [the tier 3 target policy](https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html#tier-3-target-policy) requirements. This section quotes each requirement in entirety and describes how they are met.
> - A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)
See [`src/doc/rustc/src/platform-support/kmc-solid.md`](https://github.com/kawadakk/rust/blob/release-add-solid-support/src/doc/rustc/src/platform-support/kmc-solid.md).
> - Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.
> - Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.
The new target names follow this format: `$ARCH-$VENDOR-$OS-$ABI`, which is already adopted by most existing targets. `$ARCH` and `$ABI` follow the convention: `aarch64-*` for AArch64, `armv7a-*-eabi` for Armv7-A with EABI. `$OS` is used to distinguish multiple variations of the platform in a somewhat similar way to the Apple targets, though we are only adding one variation in this PR. `$VENDOR` denotes the platform vendor name similarly to the Apple, Solaris, SGX, and VxWorks targets.
`$OS` corresponds to the value of `target_os` and takes the format `solid-$KERNEL`. The inclusion of a hyphen prevents unique decomposition of target names, though the mapping between target names and target attributes isn't trivial in the first place, e.g., because of the Android targets.
More targets may be added later, as we support other base kernels (there are at least three at the point of writing) and are interested in supporting other processor architectures in the future.
> - Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.
> - The target must not introduce license incompatibilities.
> - Anything added to the Rust repository must be under the standard Rust license (`MIT OR Apache-2.0`).
> - The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the `tidy` tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
> - If the target supports building host tools (such as `rustc` or `cargo`), those host tools must not depend on proprietary (non-FOSS) libraries, other than ordinary runtime libraries supplied by the platform and commonly used by other binaries built for the target. For instance, `rustc` built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
> - Targets should not require proprietary (non-FOSS) components to link a functional binary or library.
> - "onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are *not* limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.
We intend to make the contribution fully available under the standard Rust license with no additional legal restrictions whatsoever. This PR does not introduce any new dependency less permissive than the Rust license policy, and we are willing to ensure this doesn't happen for future contributions regarding the new targets.
The new targets don't support building host tools.
Although the new targets use a platform-provided C compiler toolchain, it can be substituted by [GNU Arm Embedded Toolchain](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm) for testing purposes.
> - Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (`core` for most targets, `alloc` for targets that can support dynamic memory allocation, `std` for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.
Most features are implemented. The following features are not implemented due to the lack of native support:
- `fs::File::{file_attr, truncate, duplicate, set_permissions}`
- `fs::{symlink, link, canonicalize}`
- Process creation
- Command-line arguments
~~Networking is not implemented yet, and we intend to add it as soon as it's ready.~~
Edit (2021-07-07): Networking is now implemented.
Backtrace generation is not really a good fit for embedded targets, so it's intentionally left unimplemented. Unwinding is functional, however.
> - The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running tests (even if they do not pass), the documentation must explain how to run tests for the target, using emulation if possible or dedicated hardware if necessary.
See [`src/doc/rustc/src/platform-support/kmc-solid.md`](https://github.com/kawadakk/rust/blob/release-add-solid-support/src/doc/rustc/src/platform-support/kmc-solid.md). Running tests is not supported.
> - Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
> - This requirement does not prevent part or all of this policy from being cited in an explicit contract or work agreement (e.g. to implement or maintain support for a target). This requirement exists to ensure that a developer or team responsible for reviewing and approving a target does not face any legal threats or obligations that would prevent them from freely exercising their judgment in such approval, even if such judgment involves subjective matters or goes beyond the letter of these requirements.
> - Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via ``@`)` to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
> - Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.
> - Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
> - In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.
We acknowledge these requirements and intend to ensure they are met.
There are no closely related targets at the moment.
SOLID[1] is an embedded development platform provided by Kyoto
Microcomputer Co., Ltd. This commit introduces a basic Tier 3 support
for SOLID.
# New Targets
The following targets are added:
- `aarch64-kmc-solid_asp3`
- `armv7a-kmc-solid_asp3-eabi`
- `armv7a-kmc-solid_asp3-eabihf`
SOLID's target software system can be divided into two parts: an
RTOS kernel, which is responsible for threading and synchronization,
and Core Services, which provides filesystems, networking, and other
things. The RTOS kernel is a μITRON4.0[2][3]-derived kernel based on
the open-source TOPPERS RTOS kernels[4]. For uniprocessor systems
(more precisely, systems where only one processor core is allocated for
SOLID), this will be the TOPPERS/ASP3 kernel. As μITRON is
traditionally only specified at the source-code level, the ABI is
unique to each implementation, which is why `asp3` is included in the
target names.
More targets could be added later, as we support other base kernels
(there are at least three at the point of writing) and are interested
in supporting other processor architectures in the future.
# C Compiler
Although SOLID provides its own supported C/C++ build toolchain, GNU Arm
Embedded Toolchain seems to work for the purpose of building Rust.
# Unresolved Questions
A μITRON4 kernel can support `Thread::unpark` natively, but it's not
used by this commit's implementation because the underlying kernel
feature is also used to implement `Condvar`, and it's unclear whether
`std` should guarantee that parking tokens are not clobbered by other
synchronization primitives.
# Unsupported or Unimplemented Features
Most features are implemented. The following features are not
implemented due to the lack of native support:
- `fs::File::{file_attr, truncate, duplicate, set_permissions}`
- `fs::{symlink, link, canonicalize}`
- Process creation
- Command-line arguments
Backtrace generation is not really a good fit for embedded targets, so
it's intentionally left unimplemented. Unwinding is functional, however.
## Dynamic Linking
Dynamic linking is not supported. The target platform supports dynamic
linking, but enabling this in Rust causes several problems.
- The linker invocation used to build the shared object of `std` is
too long for the platform-provided linker to handle.
- A linker script with specific requirements is required for the
compiled shared object to be actually loadable.
As such, we decided to disable dynamic linking for now. Regardless, the
users can try to create shared objects by manually invoking the linker.
## Executable
Building an executable is not supported as the notion of "executable
files" isn't well-defined for these targets.
[1] https://solid.kmckk.com/SOLID/
[2] http://ertl.jp/ITRON/SPEC/mitron4-e.html
[3] https://en.wikipedia.org/wiki/ITRON_project
[4] https://toppers.jp/
Fix spacing of links in inline code.
Similar to #80733, but the focus is different. This PR eliminates all occurrences of pieced-together inline code blocks like [`Box`]`<`[`Option`]`<T>>` and replaces them with good-looking ones (using HTML-syntax), like <code>[Box]<[Option]\<T>></code>. As far as I can tell, I should’ve found all of these in the standard library (regex search with `` r"`\]`|`\[`" ``) \[except for in `core::convert` where I’ve noticed other things in the docs that I want to fix in a separate PR]. In particular, unlike #80733, I’ve added almost no new instance of inline code that’s broken up into multiple links (or some link and some link-free part). I also added tooltips (the stuff in quotes for the markdown link listings) in places that caught my eye, but that’s by no means systematic, just opportunistic.
[Box]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[`Box`]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[Option]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
[`Option`]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
Context: I got annoyed by repeatedly running into new misformatted inline code while reading the standard library docs. I know that once issue #83997 (and/or related ones) are resolved, these changes become somewhat obsolete, but I fail to notice much progress on that end right now.
r? `@jyn514`
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync}
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::string
----------
Fix spacing for links inside code blocks in alloc::vec
----------
Fix spacing for links inside code blocks in core::option
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in core::result
----------
Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll}
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path}
----------
Fix spacing for links inside code blocks in std::{collections, time}
----------
Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str}
----------
Fix spacing for links inside code blocks, and improve link tooltips in std::ffi
----------
Fix spacing for links inside code blocks, and improve a few link tooltips
in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}}
----------
Fix typo in link to `into` for `OsString` docs
----------
Remove tooltips that will probably become redundant in the future
----------
Apply suggestions from code review
Replacing `…std/primitive.reference.html` paths with just `reference`
Co-authored-by: Joshua Nelson <github@jyn.dev>
----------
Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
Add a better error message for #39364
There is a known bug in the implementation of mpsc channels in rust.
This adds a clearer error message when the bug occurs, so that developers don't lose too much time looking for the origin of the bug.
See https://github.com/rust-lang/rust/issues/39364
Fix WinUWP std compilation errors due to I/O safety
I/O safety for Windows has landed in #87329. However, it does not cover UWP specific parts and prevents all UWP targets from building. See https://github.com/YtFlow/Maple/issues/18. This PR fixes these compile errors when building std for UWP targets.
If you know how much data to expect and use `Vec::with_capacity` to
pre-allocate a buffer of that capacity, `Read::read_to_end` will still
double its capacity. It needs some space to perform a read, even though
that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into
memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading
into a small "probe" buffer instead. If that read returns `0` then it's
confirmed that the buffer was the perfect size. If it doesn't, the probe
buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be
removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the
previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in
`initial_buffer_size`.
Curiously, there was a unit test that specifically checked that
`Read::read_to_end` *does* over-allocate. I removed that test, too.
Impl `Error` for `FromSecsError` without foreign type
Using it through the crate-local path in `std` means that it shouldn't make an "Implementations on Foreign Types" section in the `std::error::Error` docs.
Fixes a technicality regarding the size of C's `char` type
Specifically, ISO/IEC 9899:2018 — better known as "C18" — (and at least
C11, C99 and C89) do not specify the size of `byte` in bits.
Section 3.6 defines "byte" as "addressable unit of data storage" while
section 6.2.5 ("Types") only defines "char" as "large enough to store
any member of the basic execution set" giving it a lower bound of 7 bit
(since there are 96 characters in the basic execution set).
With section 6.5.3.4 paragraph 4 "When sizeof is applied to an operant
that has type char […] the result is 1" you could read this as the size
of `char` in bits being defined as exactly the same as the number of
bits in a byte but it's also valid to read that as an exception.
In general implementations take `char` as the smallest unit of
addressable memory, which for modern byte-addressed architectures is
overwhelmingly 8 bits to the point of this convention being completely
cemented into just about all of our software.
So is any of this actually relevant at all? I hope not. I sincerely hope
that this never, ever comes up.
But if for some reason a poor rustacean is having to interface with C
code running on a Cray X1 that in 2003 is still doing word-addressed
memory with 64-bit chars and they trust the docs here blindly it will
blow up in her face. And I'll be truly sorry for her to have to deal
with … all of that.
Using it through the crate-local path in `std` means that it shouldn't make an "Implementations on Foreign Types" section in the `std::error::Error` docs.
Add initial support for m68k
This patch series adds initial support for m68k making use of the new M68k
backend introduced with LLVM-13. Additional changes will be needed to be
able to actually use the backend for this target.
Specifically, ISO/IEC 9899:2018 — better known as "C18" — (and at least
C11, C99 and C89) do not specify the size of `byte` in bits.
Section 3.6 defines "byte" as "addressable unit of data storage" while
section 6.2.5 ("Types") only defines "char" as "large enough to store
any member of the basic execution set" giving it a lower bound of 7 bit
(since there are 96 characters in the basic execution set).
With section 6.5.3.4 paragraph 4 "When sizeof is applied to an operant
that has type char […] the result is 1" you could read this as the size
of `char` in bits being defined as exactly the same as the number of
bits in a byte but it's also valid to read that as an exception.
In general implementations take `char` as the smallest unit of
addressable memory, which for modern byte-addressed architectures is
overwhelmingly 8 bits to the point of this convention being completely
cemented into just about all of our software.
So is any of this actually relevant at all? I hope not. I sincerely hope
that this never, ever comes up.
But if for some reason a poor rustacean is having to interface with C
code running on a Cray X1 that in 2003 is still doing word-addressed
memory with 64-bit words and they trust the docs here blindly it will
blow up in her face. And I'll be truly sorry for her to have to deal
with … all of that.
Add intra-doc links and small changes to `std::os` to be more consistent
I believe that a few items in `std::os` should be linked. I've also added a basic example in `std::os::windows`.
fix potential race in AtomicU64 time monotonizer
The AtomicU64-based monotonizer introduced in #83093 is incorrect because several threads could try to update the value concurrently and a thread which doesn't have the newest value among all the updates could win.
That bug probably has little real world impact since it doesn't make observed time worse than hardware clocks. The worst case would probably be a thread which has a clock that is behind by several cycles observing several inconsistent fixups, which should be similar to observing the unfiltered backslide in the first place.
New benchmarks, they don't look as good as the original PR but still an improvement compared to the mutex.
I don't know why the contended mutex case is faster now than in the previous benchmarks.
```
actually_monotonic() == true:
test time::tests::instant_contention_01_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 45 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads ... bench: 45 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads ... bench: 45 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads ... bench: 46 ns/iter (+/- 0)
atomic u64:
test time::tests::instant_contention_01_threads ... bench: 66 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 287 ns/iter (+/- 14)
test time::tests::instant_contention_04_threads ... bench: 296 ns/iter (+/- 43)
test time::tests::instant_contention_08_threads ... bench: 604 ns/iter (+/- 163)
test time::tests::instant_contention_16_threads ... bench: 1,147 ns/iter (+/- 29)
mutex:
test time::tests::instant_contention_01_threads ... bench: 78 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 652 ns/iter (+/- 275)
test time::tests::instant_contention_04_threads ... bench: 900 ns/iter (+/- 32)
test time::tests::instant_contention_08_threads ... bench: 1,927 ns/iter (+/- 62)
test time::tests::instant_contention_16_threads ... bench: 3,748 ns/iter (+/- 146)
```
Avoid codegen for Result::into_ok in lang_start
This extra codegen seems to be the cause for the regressions in max-rss on #86034. While LLVM will certainly optimize the dead code away, avoiding it's generation in the first place seems good, particularly when it is so simple.
#86034 produced this [diff](https://gist.github.com/Mark-Simulacrum/95c7599883093af3b960c35ffadf4dab#file-86034-diff) for a simple `fn main() {}`. With this PR, that diff [becomes limited to just a few extra IR instructions](https://gist.github.com/Mark-Simulacrum/95c7599883093af3b960c35ffadf4dab#file-88988-from-pre-diff) -- no extra functions.
Note that these are pre-optimization; LLVM surely will eliminate this during optimization. However, that optimization can end up generating more work and bump memory usage, and this eliminates that.
Allow `panic!("{}", computed_str)` in const fn.
Special-case `panic!("{}", arg)` and translate it to `panic_display(&arg)`. `panic_display` will behave like `panic_any` in cosnt eval and behave like `panic!(format_args!("{}", arg))` in runtime.
This should bring Rust 2015 and 2021 to feature parity in terms of `const_panic`; and hopefully would unblock the stabilisation of #51999.
`@rustbot` modify labels: +T-compiler +T-libs +A-const-eval +A-const-fn
r? `@oli-obk`
Add chown functions to std::os::unix::fs to change the owner and group of files
This is a straightforward wrapper that uses the existing helpers for C
string handling and errno handling.
Having this available is convenient for UNIX utility programs written in
Rust, and avoids having to call unsafe functions like `libc::chown`
directly and handle errors manually, in a program that may otherwise be
entirely safe code.
In addition, these functions provide a more Rustic interface by
accepting appropriate traits and using `None` rather than `-1`.
Add TcpListener::into_incoming and IntoIncoming
The `incoming` method is really useful, however for some use cases the borrow
this introduces is needlessly restricting. Thus, an owned variant is added.
r? ``@joshtriplett``
The RefCell is now borrowed exactly once. In addition a code sequence
that contains an unwrap that is guaranteed to never panic at runtime is
replaced with get_or_insert_with, which makes the intended behavior
clearer and will not emit code to panic even without optimizations.
Previously the thread name would first be heap allocated and then
re-allocated to add a nul terminator. Now it will be heap allocated only
once with nul terminator added form the start.
This is a straightforward wrapper that uses the existing helpers for C
string handling and errno handling.
Having this available is convenient for UNIX utility programs written in
Rust, and avoids having to call unsafe functions like `libc::chown`
directly and handle errors manually, in a program that may otherwise be
entirely safe code.
In addition, these functions provide a more Rustic interface by
accepting appropriate traits and using `None` rather than `-1`.
This works by doing two things:
- Adding links that are specific to the crate. Since not all primitive
items are defined in `core` (due to lang_items), these need to use
relative links and not intra-doc links.
- Duplicating `primitive_docs` in both core and std. This allows not needing CARGO_PKG_NAME to build the standard library. It also adds a tidy check to make sure they stay the same.
As discussed in #88576, raw handle values in Windows can be null, such
as in `windows_subsystem` mode, or when consoles are detached from a
process. So, don't use `NonNull` to hold them, don't assert that they're
not null, and remove `OwnedHandle`'s `repr(transparent)`. Introduce a
new `HandleOrNull` type, similar to `HandleOrInvalid`, to cover the FFI
use case.
More symbolic doc aliases
A bunch of small changes, mostly adding `#[doc(alias = "…")]` entries for symbolic `"…"`.
Also a small change in documentation of `const` keywords.
Document when to use Windows' `symlink_dir` vs. `symlink_file`
It was previously unclear why there are two functions and when they should be used.
Fixes: #88635
Add links in docs for some primitive types
This pull request adds additional links in existing documentation of some of the primitive types.
Where items are linked only once, I have used the `[link](destination)` format. For items in `std`, I have linked directly to the HTML, since although the primitives are in `core`, they are not displayed on `core` documentation. I was unsure of what length I should keep lines of documentation to, so I tried to keep them within reason.
Additionally, I have avoided excessively linking to keywords like `self` when they are not relevant to the documentation. I can add these links if it would be an improvement.
I hope this can improve Rust. Please let me know if there's anything I did wrong!
While issues have been seen on arm64 platforms the Arm architecture requires
that the counter monotonically increases and that it must provide a uniform
view of system time (e.g. it must not be possible for a core to receive a
message from another core with a time stamp and observe time going backwards
(ARM DDI 0487G.b D11.1.2). While there have been a few 64bit SoCs that have
bugs (#49281, #56940) which cause time to not monotonically increase, these have
been fixed in the Linux kernel and we shouldn't penalize all Arm SoCs for those
who refuse to update their kernels:
SUN50I_ERRATUM_UNKNOWN1 - Allwinner A64 / Pine A64 - fixed in 5.1
FSL_ERRATUM_A008585 - Freescale LS2080A/LS1043A - fixed in 4.10
HISILICON_ERRATUM_161010101 - Hisilicon 1610 - fixed in 4.11
ARM64_ERRATUM_858921 - Cortex A73 - fixed in 4.12
255a3f3e18 std: Force `Instant::now()` to be monotonic added a mutex to work around
this problem and a small test program using glommio shows the majority of time spent
acquiring and releasing this Mutex. 3914a7b0da tries to improve this, but actually
makes it worse on big systems as for 128b atomics a ldxp/stxp pair (and
successful loop) is required which is expensive as a lock and because of how
the load/store-exclusives scale on large Arm systems is both unfair to threads
and tends to go backwards in performance.
aarch64 prior to v8.4 (FEAT_LSE2) doesn't have an instruction that guarantees
untorn 128b reads except for completing a 128b load/store exclusive pair
(ldxp/stxp) or compare-and-swap (casp) successfully. The requirement to
complete a 128b read+write atomic is actually more expensive and more unfair
than the previous implementation of monotonize() which used a Mutex on aarch64,
especially at large core counts. For aarch64 switch to the 64b atomic
implementation which is about 13x faster for a benchmark that involves many
calls to Instant::now().
There is a known bug in the implementation of mpsc channels in rust.
This adds a clearer error message when the bug occurs, so that developers don't lose too much time looking for the origin of the bug.
See https://github.com/rust-lang/rust/issues/39364
Stabilize std::os::unix::fs::chroot
I've verified that this works as documented, and I've tested it in (a nightly
build of) production software as a replacement for an unsafe call to
`libc::chroot`. It's been available in nightly for a few releases. I think it's
ready to stabilize.
---
Tracking issue: https://github.com/rust-lang/rust/issues/84715
Allow writing of incomplete UTF-8 sequences to the Windows console via stdout/stderr
# Problem
Writes of just an incomplete UTF-8 byte sequence (e.g. `b"\xC3"` or `b"\xF0\x9F"`) to stdout/stderr with a Windows console attached error with `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` even though further writes could complete the codepoint. This is currently a rare occurence since the [linewritershim](2c56ea38b0/library/std/src/io/buffered/linewritershim.rs) implementation flushes complete lines immediately and buffers up to 1024 bytes for incomplete lines. It can still happen as described in #83258.
The problem will become more pronounced once the developer can switch stdout/stderr from line-buffered to block-buffered or immediate when the changes in the "Switchable buffering for Stdout" pull request (#78515) get merged.
# Patch description
If there is at least one valid UTF-8 codepoint all valid UTF-8 is passed through to the extracted `write_valid_utf8_to_console()` fn. The new code only comes into play if `write()` is being passed a short byte slice comprising an incomplete UTF-8 codepoint. In this case up to three bytes are buffered in the `IncompleteUtf8` struct associated with `Stdout` / `Stderr`. The bytes are accepted one at a time. As soon as an error can be detected `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` is returned. Once a complete UTF-8 codepoint is received it is passed to the `write_valid_utf8_to_console()` and the buffer length is set to zero.
Calling `flush()` will neither error nor write anything if an incomplete codepoint is present in the buffer.
# Tests
Currently there are no Windows-specific tests for console writing code at all. Writing (regression) tests for this problem is a bit challenging since unit tests and UI tests don't run in a console and suddenly popping up another console window might be surprising to developers running the testsuite and it might not work at all in CI builds. To just test the new functionality in unit tests the code would need to be refactored. Some guidance on how to proceed would be appreciated.
# Public API changes
* `std::str::verifications::utf8_char_width()` would be exposed as `std::str::utf8_char_width()` behind the "str_internals" feature gate.
# Related issues
* Fixes#83258.
* PR #78515 will exacerbate the problem.
# Open questions
* Add tests?
* Squash into one commit with better commit message?
Stabilize `UnsafeCell::raw_get()`
This PR stabilizes the associated function `UnsafeCell::raw_get()`. The FCP has [already completed](https://github.com/rust-lang/rust/issues/66358#issuecomment-899095068). While there was some discussion about the naming after the close of the FCP, it looks like people have agreed on this name. Still, it would probably be best if a `libs-api` member had a look at this and stated whether more discussion is needed.
While I was at it, I added some tests for `UnsafeCell`, because there were barely any.
Closes#66358.
Use the return value of readdir_r() instead of errno
POSIX says:
> If successful, the readdir_r() function shall return zero; otherwise,
> an error number shall be returned to indicate the error.
But we were previously using errno instead of the return value. This
led to issue #86649.
POSIX says:
> If successful, the readdir_r() function shall return zero; otherwise,
> an error number shall be returned to indicate the error.
But we were previously using errno instead of the return value. This
led to issue #86649.
Add `TcpStream::set_linger` and `TcpStream::linger`
Adds methods for getting/setting the `SO_LINGER` option on TCP sockets. Behavior is consistent across Unix and Windows.
r? `@joshtriplett` (I noticed you've been reviewing net related PRs)
Document `std::env::current_exe` possible rename behaviour
It might not be obvious that the "path of the current running executable" may (or may not) imply "at the time it was loaded".
This came up recently in chat so I thought it might be worth documenting.
Add Saturating type (based on Wrapping type)
Tracking #87920
### Unresolved Questions
<!--
Include any open questions that need to be answered before the feature can be
stabilised.
-->
- [x] ~`impl Div for Saturating<T>` falls back on inner integer division - which seems alright?~
- [x] add `saturating_div`? (to respect division by `-1`)
- [x] There is no `::saturating_shl` and `::saturating_shr`. (How to) implement `Shl`, `ShlAssign`, `Shr` and `ShrAssign`?
- [naively](3f7d2ce28f)
- [x] ~`saturating_neg` is only implemented on [signed integer types](https://doc.rust-lang.org/std/?search=saturating_n)~
- [x] Is the implementation copied over from the `Wrapping`-type correct for `Saturating`?
- [x] `Saturating::rotate_left`
- [x] `Saturating::rotate_right`
- [x] `Not`
- [x] `BitXorOr` and `BitXorOrAssign`
- [x] `BitOr` and `BitOrAssign`
- [x] `BitAnd` and `BitAndAssign`
- [x] `Saturating::swap_bytes`
- [x] `Saturating::reverse_bits`
The `incoming` method is really useful, however for some use cases the borrow
this introduces is needlessly restricting. Thus, an owned variant is added.
Add `c_size_t` and `c_ssize_t` to `std::os::raw`.
Apparently these aren't guaranteed to be the same, and are merely "always the same in practice" (see https://rust-lang.zulipchat.com/#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/.60usize.60.20vs.20.60size_t.60).
This is a big footgun, but I suspect it can be alleviated if we expose this and start migrating people to it in advance of any platforms that ever have this as different.
I'll file a tracking issue after this gets some traction.
Stabilise BufWriter::into_parts
The FCP for this has already completed, in #80690.
This was just blocked on #85901 (which changed the name), which is now merged. The original stabilisation MR was #84770 but that has a lot of noise in it, and I also accidentally deleted the branch while trying to tidy up. So here is a new MR. Sorry for the noise.
Closes#80690
Errorkind reorder
I was doing a bit more work in this area and the untidiness of these two orderings bothered me.
The commit messages have the detailed rationale. For your convenience, I c&p them here:
```
io::ErrorKind: rationalise ordering in main enum
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
```
```
io::Error: alphabeticise the match in as_str()
There was no rationale for the previous ordering.
```
r? kennytm since that's who rust-highfive picked before, in #88294 which I accidentally closed.
These structs have misleading names. An ExitStatus[Error] is actually
a Unix wait status; an ExitCode is actually an exit status.
The Display impls are fixed, but the Debug impls are still misleading,
as reported in #74832.
Fix this by pretending that these internal structs are called
`unix_exit_status` and `unix_wait_status` as applicable. (We can't
actually rename the structs because of the way that the cross-platform
machinery works: the names are cross-platform.)
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Implement `AsFd` etc. for `UnixListener`.
Implement `AsFd`, `From<OwnedFd>`, and `Into<OwnedFd>` for
`UnixListener`. This is a follow-up to #87329.
r? `@joshtriplett`
add file_prefix method to std::path
This is an initial implementation of `std::path::Path::file_prefix`. It is effectively a "left" variant of the existing [`file_stem`](https://doc.rust-lang.org/std/path/struct.Path.html#method.file_stem) method. An illustration of the difference is
```rust
use std::path::Path;
let path = Path::new("foo.tar.gz");
assert_eq!(path.file_stem(), Some("foo.tar"));
assert_eq!(path.file_prefix(), Some("foo"));
```
In my own development, I generally find I almost always want the prefix, rather than the stem, so I thought it might be best to suggest it's addition to libstd.
Of course, as this is my first contribution, I expect there is probably more work that needs to be done. Additionally, if the libstd team feel this isn't appropriate then so be it.
There has been some [discussion about this on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/file_lstem/near/238076313) and a user there suggested I open a PR to see whether someone in the libstd team thinks it is worth pursuing.
where available use AtomicU{64,128} instead of mutex for Instant backsliding protection
This decreases the overhead of backsliding protection on x86 systems with unreliable TSC, e.g. windows. And on aarch64 systems where 128bit atomics are available.
The following benchmarks were taken on x86_64 linux though by overriding `actually_monotonic()`, the numbers may look different on other platforms
```
# actually_monotonic() == true
test time::tests::instant_contention_01_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads ... bench: 44 ns/iter (+/- 0)
# 1x AtomicU64
test time::tests::instant_contention_01_threads ... bench: 65 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 157 ns/iter (+/- 20)
test time::tests::instant_contention_04_threads ... bench: 281 ns/iter (+/- 53)
test time::tests::instant_contention_08_threads ... bench: 555 ns/iter (+/- 77)
test time::tests::instant_contention_16_threads ... bench: 883 ns/iter (+/- 107)
# mutex
test time::tests::instant_contention_01_threads ... bench: 60 ns/iter (+/- 2)
test time::tests::instant_contention_02_threads ... bench: 770 ns/iter (+/- 231)
test time::tests::instant_contention_04_threads ... bench: 1,347 ns/iter (+/- 45)
test time::tests::instant_contention_08_threads ... bench: 2,693 ns/iter (+/- 114)
test time::tests::instant_contention_16_threads ... bench: 5,244 ns/iter (+/- 487)
```
Since I don't have an arm machine with 128bit atomics I wasn't able to benchmark the AtomicU128 implementation.
I/O safety.
Introduce `OwnedFd` and `BorrowedFd`, and the `AsFd` trait, and
implementations of `AsFd`, `From<OwnedFd>` and `From<T> for OwnedFd`
for relevant types, along with Windows counterparts for handles and
sockets.
Tracking issue: <https://github.com/rust-lang/rust/issues/87074>
RFC: <https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md>
Highlights:
- The doc comments at the top of library/std/src/os/unix/io/mod.rs and library/std/src/os/windows/io/mod.rs
- The new types and traits in library/std/src/os/unix/io/fd.rs and library/std/src/os/windows/io/handle.rs
- The removal of the `RawHandle` struct the Windows impl, which had the same name as the `RawHandle` type alias, and its functionality is now folded into `Handle`.
Managing five levels of wrapping (File wraps sys::fs::File wraps sys::fs::FileDesc wraps OwnedFd wraps RawFd, etc.) made for a fair amount of churn and verbose as/into/from sequences in some places. I've managed to simplify some of them, but I'm open to ideas here.
r? `@joshtriplett`
Add fast path for Path::cmp that skips over long shared prefixes
```
# before
test path::tests::bench_path_cmp_fast_path_buf_sort ... bench: 60,811 ns/iter (+/- 865)
test path::tests::bench_path_cmp_fast_path_long ... bench: 6,459 ns/iter (+/- 275)
test path::tests::bench_path_cmp_fast_path_short ... bench: 1,777 ns/iter (+/- 34)
# after
test path::tests::bench_path_cmp_fast_path_buf_sort ... bench: 38,140 ns/iter (+/- 211)
test path::tests::bench_path_cmp_fast_path_long ... bench: 1,471 ns/iter (+/- 24)
test path::tests::bench_path_cmp_fast_path_short ... bench: 1,106 ns/iter (+/- 9)
```
The name (and updated documentation) make the FFI-only usage clearer, and wrapping Option<OwnedHandle> avoids the need to write a separate Drop or Debug impl.
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
Add TcpStream type to TcpListener::incoming docs
## Context
While going through the "The Rust Programming Language" book (Klabnik & Nichols), the TCP server example directs us to use TcpListener::incoming. I was curious how I could pass this value to a function (before reading ahead in the book), so I looked up the docs to determine the signature.
When I opened the docs, I found https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming, which didn't mention TcpStream anywhere in the example.
Eventually, I clicked on https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.accept in the docs (after clicking a few other locations first), and was able to surmise that the value contained TcpStream.
## Opportunity
While this type is mentioned several times in this doc, I feel that someone should be able to fully use the results of the TcpListner::incoming iterator based solely on the docs of just this method.
## Implementation
I took the code from the top-level TcpListener https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming and blended it with the existing docs for TcpListener::incoming https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming.
It does make the example a little longer, and it also introduces a little duplication. It also gives the reader the type signatures they need to move on to the next step.
## Additional considerations
I noticed that in this doc, `handle_connection` and `handle_client` are both used to accept a TcpStream in the docs on this page. I want to standardize on one function name convention, so readers don't accidentally think two different concepts are being referenced. I didn't want to cram do too much in one PR, I can update this PR to make that change, or I could send another PR (if you would like).
First attempted contribution to Rust (and I'm also still very new, hence reading through the rust book for the first time)! Would you please let me know what you think?
Update the backtrace crate in libstd
This commit updates the backtrace crate in libstd now that dependencies
have been updated to use `memchr` from the standard library as well.
This is mostly just making sure deps are up-to-date and have all the
latest-and-greatest fixes and such.
Closesrust-lang/backtrace-rs#432
This commit updates the backtrace crate in libstd now that dependencies
have been updated to use `memchr` from the standard library as well.
This is mostly just making sure deps are up-to-date and have all the
latest-and-greatest fixes and such.
Closesrust-lang/backtrace-rs#432
Unbox mutexes, condvars and rwlocks on hermit
[RustyHermit](https://github.com/hermitcore/rusty-hermit) provides now movable synchronization primitives and we are able to unbox mutexes and condvars.
Fix environment variable getter docs
`@RalfJung` pointed out a number of errors and suboptimal choices I made in my documentation for #86183. This PR should (hopefully) fix the problems they've identified.
Change WASI's `RawFd` from `u32` to `c_int` (`i32`).
WASI previously used `u32` as its `RawFd` type, since its "file descriptors"
are unsigned table indices, and there's no fundamental reason why WASI can't
have more than 2^31 handles.
However, this creates myriad little incompability problems with code
that also supports Unix platforms, where `RawFd` is `c_int`. While WASI
isn't a Unix, it often shares code with Unix, and this difference made
such shared code inconvenient. #87329 is the most recent example of such
code.
So, switch WASI to use `c_int`, which is `i32`. This will mean that code
intending to support WASI should ideally avoid assuming that negative file
descriptors are invalid, even though POSIX itself says that file descriptors
are never negative.
This is a breaking change, but `RawFd` is considerd an experimental
feature in [the documentation].
[the documentation]: https://doc.rust-lang.org/stable/std/os/wasi/io/type.RawFd.html
r? `@alexcrichton`
The libs-api team agrees to allow const_trait_impl to appear in the
standard library as long as stable code cannot be broken (they are
properly gated) this means if the compiler teams thinks it's okay, then
it's okay.
My priority on constifying would be:
1. Non-generic impls (e.g. Default) or generic impls with no
bounds
2. Generic functions with bounds (that use const impls)
3. Generic impls with bounds
4. Impls for traits with associated types
For people opening constification PRs: please cc me and/or oli-obk.
WASI previously used `u32` as its `RawFd` type, since its "file descriptors"
are unsigned table indices, and there's no fundamental reason why WASI can't
have more than 2^31 handles.
However, this creates myriad little incompability problems with code
that also supports Unix platforms, where `RawFd` is `c_int`. While WASI
isn't a Unix, it often shares code with Unix, and this difference made
such shared code inconvenient. #87329 is the most recent example of such
code.
So, switch WASI to use `c_int`, which is `i32`. This will mean that code
intending to support WASI should ideally avoid assuming that negative file
descriptors are invalid, even though POSIX itself says that file descriptors
are never negative.
This is a breaking change, but `RawFd` is considerd an experimental
feature in [the documentation].
[the documentation]: https://doc.rust-lang.org/stable/std/os/wasi/io/type.RawFd.html
STD support for the ESP-IDF framework
Dear all,
This PR is implementing libStd support for the [ESP-IDF](https://github.com/espressif/esp-idf) newlib-based framework, which is the open source SDK provided by Espressif for their MCU family (esp32, esp32s2, esp32c3 and all other forthcoming ones).
Note that this PR has a [sibling PR](https://github.com/rust-lang/libc/pull/2310) against the libc crate, which implements proper declarations for all ESP-IDF APIs which are necessary for libStd support.
# Implementation approach
The ESP-IDF framework - despite being bare metal - offers a relatively complete POSIX API based on newlib. `pthread`, BSD sockets, file descriptors, and even a small file-system VFS layer. Perhaps the only significant exception is the lack of support for processes, which is to be expected of course on bare metal.
Therefore, the libStd support is implemented as a set of (hopefully small) changes to the `sys/unix` family of modules, in the form of conditional-compilation branches based either on `target_os = "espidf"` or in a couple of cases - based on `target_env = "newlib"` (the latter was already there actually and is not part of this patch).
The PR also contains two new targets:
- `riscv32imc-esp-espidf`
- `riscv32imac-esp-espidf`
... which are essentially copies of `riscv32imc-unknown-none-elf` and `riscv32imac-unknown-none-elf`, but enriched with proper `linker`, `linker_flavor`, `families`, `os`, `env` etc. specifications so that (a) the proper conditional compilation branches in libStd are selected when compiling with these targets and (b) the correct linker is used.
Since support for atomics is a precondition for libStd, the `riscv32imc-esp-espidf` target additionally is configured in such a way, so as to emit libcalls to the `__sync*` & `__atomic*` GCC functions, which are already implemented in the ESP-IDF framework. If this modification is not acceptable, we can also live with only the `riscv32imac-esp-espidf` target as well. While the RiscV chips of Espressif lack native atomics support, the relevant instructions are transparently emulated in the ESP-IDF framework using invalid instruction trap. This modification was implemented specifically with Rust support in mind.
# Target maintainers
In case this PR eventually gets merged, you can list myself as a Target Maintainer.
More importantly, Espressif (the chip vendor) is now actively involved and [embracing](https://github.com/espressif/rust-esp32-example/blob/main/docs/rust-on-xtensa.md) all [Rust-related efforts](https://github.com/esp-rs) which were originally a community effort. In light of that, I suppose `@MabezDev` - who initiated the Rust-on-Espressif efforts back in time and who now works for Espressif won't object to being listed as a maintainer as well.
**EDIT:** I was hinted (thanks, `@Urgau)` that answering the Tier 3 policy explicitly might be helpful. Answers below.
# Tier 3 Target Policy - answers
> A proposed target or target-specific patch that substantially changes code shared with other targets (not just target-specific code) must be reviewed and approved by the appropriate team for that shared code before acceptance.
Hopefully, the changes introduced by the ESP-IDF libStd support are rather on the small side. They are completely contained within the `sys/unix` set of modules (that is, aside from the obviously necessary one-liners in the `unwind` crate and in `build.rs`).
> A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)
`@ivmarkov`
`@MabezDev`
> Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.
The two introduced targets follow as much as possible the naming conventions of the other targets. I.e. taking the bare-metal `riscv32imac_unknown_none_elf` as a base:
* The name of the new target was derived by replacing `none` with `espidf` to designate the `target_os`.
* `_elf` was removed, as the non-bare metal targets seem not to have it
* `-newlib` was deliberately NOT added at the end, as I believe the chance of having two simultaneously active separate targets for the ESP-IDF framework with different C libraries (say, newlib vs musl) is way too small
* Finally, we replaced the middle `unknown` with `esp` which is kind of the name of the whole chipset MCU family (and abbreviation from Espressif which is too long). It will stay `esp` for all RiscV32-based MCUs of the company, as they all use the riscv32imc instruction set. By necessity however (disambiguation), it will be `esp32` or `esp32s2` or `esp32s3` for the Xtensa-based MCUs as all of these have their own variation of the Xtensa architecture. (The Xtensa targets are not part of this PR, even though they would use 1:1 the same LibStd implementation provided here, as they depend on the upstreaming of the Xtensa architecture support in LLVM; this upstreaming this is currently in progress.)
There was also a preceding discussion on the topic [here](https://github.com/espressif/rust-esp32-example/issues/14).
> Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.
We are explicitly putting an `-espidf` suffix to designate that the target is *specifically* for Rust + ESP-IDF
> Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.
Agreed.
> The target must not introduce license incompatibilities.
To the best of our knowledge, it doesn't.
> Anything added to the Rust repository must be under the standard Rust license (MIT OR Apache-2.0).
MIT + Apache 2.0
> The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the tidy tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
Requirements are not changed for any other target.
> If the target supports building host tools (such as rustc or cargo), those host tools must not depend on proprietary (non-FOSS) libraries, other than ordinary runtime libraries supplied by the platform and commonly used by other binaries built for the target. For instance, rustc built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
The targets are for bare-metal environment which is not hosting build tools or a compiler.
> Targets should not require proprietary (non-FOSS) components to link a functional binary or library.
The linker used by the targets is the GCC linker from the GCC toolchain cross-compiled for riscv. GNU GPL.
> "onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are not limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.
> Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
> This requirement does not prevent part or all of this policy from being cited in an explicit contract or work agreement (e.g. to implement or maintain support for a target). This requirement exists to ensure that a developer or team responsible for reviewing and approving a target does not face any legal threats or obligations that would prevent them from freely exercising their judgment in such approval, even if such judgment involves subjective matters or goes beyond the letter of these requirements.
Agreed.
> Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (core for most targets, alloc for targets that can support dynamic memory allocation, std for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.
The targets implement libStd almost in its entirety, except for the missing support for process, as this is a bare metal platform. The process `sys\unix` module is currently stubbed to return "not implemented" errors.
> The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running tests (even if they do not pass), the documentation must explain how to run tests for the target, using emulation if possible or dedicated hardware if necessary.
Target does not (yet) support running tests. We would gladly provide all documentation how to build for the target (where?). It is currently hosted in this [README.md](https://github.com/ivmarkov/rust-esp32-std-hello) file, but will likely be moved to the [esp-rs](https://github.com/esp-rs) organization. Since the build for the target is driven by cargo and [all other tooling is downloaded automatically during the build](https://github.com/esp-rs/esp-idf-sys/blob/master/build.rs), there is no need for extensive documentation.
> Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via `@)` to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
Agreed.
> Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.
Agreed.
> Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
To the best of our knowledge, we believe we are not breaking any other target (be it tier 1, 2 or 3).
> In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.
To the best of our knowledge, we have not introduced any unconditional use of a feature that affects any other target.
> If a tier 3 target stops meeting these requirements, or the target maintainers no longer have interest or time, or the target shows no signs of activity and has not built for some time, or removing the target would improve the quality of the Rust codebase, we may post a PR to remove it; any such PR will be CCed to the target maintainers (and potentially other people who have previously worked on the target), to check potential interest in improving the situation.
Agreed.
removed references to parent/child from std::thread documentation
- also clarifies how thread.join and detaching of threads works
- the previous prose implied that there is a relationship between a
spawning thread and the thread being spawned, and that "child" threads
couldn't outlive their "parents" unless detached, which is incorrect.
Replace read_to_string with read_line in Stdin example
The current example results in infinitely reading from stdin, which can confuse newcomers trying to read from stdin.
(`@razmag` encountered this while learning the language from the docs)
While going through the "The Rust Programming Language" book (Klabnik & Nichols), the TCP server example directs us to use TcpListener::incoming. I was curious how I could pass this value to a function (before reading ahead in the book), so I looked up the docs to determine the signature.
When I opened the docs, I found https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming, which didn't mention TcpStream anywhere in the example.
Eventually, I clicked on https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.accept in the docs (after clicking a few other locations first), and was able to surmise that the value contained TcpStream.
## Opportunity
While this type is mentioned several times in this doc, I feel that someone should be able to fully use the results of the TcpListner::incoming iterator based solely on the docs of just this method.
## Implementation
I took the code from the top-level TcpListener https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming and blended it with the existing docs for TcpListener::incoming https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming.
It does make the example a little longer, and it also introduces a little duplication. It also gives the reader the type signatures they need to move on to the next step.
## Additional considerations
I noticed that in this doc, `handle_connection` and `handle_client` are both used to accept a TcpStream in the docs on this page. I want to standardize on one function name convention, so readers don't accidentally think two different concepts are being referenced. I didn't want to cram do too much in one PR, I can update this PR to make that change, or I could send another PR (if you would like).
First attempted contribution to Rust (and I'm also still very new, hence reading through the rust book for the first time)! Would you please let me know what you think?
Stabilize Vec<T>::shrink_to
This PR stabilizes `shrink_to` feature and closes the corresponding issue. The second point was addressed already, and no `panic!` should occur.
Closes#56431.
impl Default, Copy, Clone for std::io::Sink and Empty
The omission of `Sink: Default` is causing me a slight inconvenience in a test harness. There seems little reason for this and `Empty` not to be `Clone` and `Copy` too.
I have made all three of these insta-stable, because:
AIUI `Copy` can only be derived, and I was not able to find any examples of how to unstably derive it. I think it is probably not possible.
I hunted through the git history for precedent and found
> 79b8ad84c8
> Implement `Copy` for `IoSlice`
> https://github.com/rust-lang/rust/pull/69403
which was also insta-stable.
Document that fs::read_dir skips . and ..
Hi,
I think this is worth noting in the docs since it differs from POSIX `readdir`. I didn’t put it under platform-specific notes because it seems to be consistent across platforms, and changing this behavior in the future could cause pretty nasty bugs.
Thanks!
- also clarifies how thread.join and detaching of threads works
- the previous prose implied that there is a relationship between a
spawning thread and the thread being spawned, and that "child" threads
couldn't outlive their parents unless detached, which is incorrect.
Hide allocator details from TryReserveError
I think there's [no need for TryReserveError to carry detailed information](https://github.com/rust-lang/rust/issues/48043#issuecomment-825139280), but I wouldn't want that issue to delay stabilization of the `try_reserve` feature.
So I'm proposing to stabilize `try_reserve` with a `TryReserveError` as an opaque structure, and if needed, expose error details later.
This PR moves the `enum` to an unstable inner `TryReserveErrorKind` that lives under a separate feature flag. `TryReserveErrorKind` could possibly be left as an implementation detail forever, and the `TryReserveError` get methods such as `allocation_size() -> Option<usize>` or `layout() -> Option<Layout>` instead, or the details could be dropped completely to make try-reserve errors just a unit struct, and thus smaller and cheaper.
Rollup of 9 pull requests
Successful merges:
- #87561 (thread set_name haiku implementation.)
- #87715 (Add long error explanation for E0625)
- #87727 (explicit_generic_args_with_impl_trait: fix min expected number of generics)
- #87742 (Validate FFI-safety warnings on naked functions)
- #87756 (Add back -Zno-profiler-runtime)
- #87759 (Re-use std::sealed::Sealed in os/linux/process.)
- #87760 (Promote `aarch64-apple-ios-sim` to Tier 2)
- #87770 (permit drop impls with generic constants in where clauses)
- #87780 (alloc: Use intra doc links for the reserve function)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
alloc: Use intra doc links for the reserve function
The sentence exists to highlight the existence of a
performance footgun of repeated calls of the
reserve_exact function.
Re-use std::sealed::Sealed in os/linux/process.
This uses `std::sealed::Sealed` in `std::os::linux::process` instead of defining new `Sealed` traits there.
rustc: Fill out remaining parts of C-unwind ABI
This commit intends to fill out some of the remaining pieces of the
C-unwind ABI. This has a number of other changes with it though to move
this design space forward a bit. Notably contained within here is:
* On `panic=unwind`, the `extern "C"` ABI is now considered as "may
unwind". This fixes a longstanding soundness issue where if you
`panic!()` in an `extern "C"` function defined in Rust that's actually
UB because the LLVM representation for the function has the `nounwind`
attribute, but then you unwind.
* Whether or not a function unwinds now mainly considers the ABI of the
function instead of first checking the panic strategy. This fixes a
miscompile of `extern "C-unwind"` with `panic=abort` because that ABI
can still unwind.
* The aborting stub for non-unwinding ABIs with `panic=unwind` has been
reimplemented. Previously this was done as a small tweak during MIR
generation, but this has been moved to a separate and dedicated MIR
pass. This new pass will, for appropriate functions and function
calls, insert a `cleanup` landing pad for any function call that may
unwind within a function that is itself not allowed to unwind. Note
that this subtly changes some behavior from before where previously on
an unwind which was caught-to-abort it would run active destructors in
the function, and now it simply immediately aborts the process.
* The `#[unwind]` attribute has been removed and all users in tests and
such are now using `C-unwind` and `#![feature(c_unwind)]`.
I think this is largely the last piece of the RFC to implement.
Unfortunately I believe this is still not stabilizable as-is because
activating the feature gate changes the behavior of the existing `extern
"C"` ABI in a way that has no replacement. My thinking for how to enable
this is that we add support for the `C-unwind` ABI on stable Rust first,
and then after it hits stable we change the behavior of the `C` ABI.
That way anyone straddling stable/beta/nightly can switch to `C-unwind`
safely.
Remove the aarch64 `crypto` target_feature
The subfeatures `aes` or `sha2` should be used instead.
This can't yet be done for ARM targets as some LLVM intrinsics still require `crypto`.
Also update the runtime feature detection tests in `library/std` to mirror the updates in `stdarch`. This also helps https://github.com/rust-lang/rust/issues/86941
r? ``@Amanieu``
This commit intends to fill out some of the remaining pieces of the
C-unwind ABI. This has a number of other changes with it though to move
this design space forward a bit. Notably contained within here is:
* On `panic=unwind`, the `extern "C"` ABI is now considered as "may
unwind". This fixes a longstanding soundness issue where if you
`panic!()` in an `extern "C"` function defined in Rust that's actually
UB because the LLVM representation for the function has the `nounwind`
attribute, but then you unwind.
* Whether or not a function unwinds now mainly considers the ABI of the
function instead of first checking the panic strategy. This fixes a
miscompile of `extern "C-unwind"` with `panic=abort` because that ABI
can still unwind.
* The aborting stub for non-unwinding ABIs with `panic=unwind` has been
reimplemented. Previously this was done as a small tweak during MIR
generation, but this has been moved to a separate and dedicated MIR
pass. This new pass will, for appropriate functions and function
calls, insert a `cleanup` landing pad for any function call that may
unwind within a function that is itself not allowed to unwind. Note
that this subtly changes some behavior from before where previously on
an unwind which was caught-to-abort it would run active destructors in
the function, and now it simply immediately aborts the process.
* The `#[unwind]` attribute has been removed and all users in tests and
such are now using `C-unwind` and `#![feature(c_unwind)]`.
I think this is largely the last piece of the RFC to implement.
Unfortunately I believe this is still not stabilizable as-is because
activating the feature gate changes the behavior of the existing `extern
"C"` ABI in a way that has no replacement. My thinking for how to enable
this is that we add support for the `C-unwind` ABI on stable Rust first,
and then after it hits stable we change the behavior of the `C` ABI.
That way anyone straddling stable/beta/nightly can switch to `C-unwind`
safely.
Add convenience method for handling ipv4-mapped addresses by canonicalizing them
This simplifies checking common properties in an address-family-agnostic
way since #86335 commits to not checking IPv4 semantics
of IPv4-mapped addresses in the `Ipv6Addr` property methods.
Commit to not supporting IPv4-in-IPv6 addresses
Stabilization of the `ip` feature has for a long time been blocked on the question of whether Rust should support handling "IPv4-in-IPv6" addresses: should the various `Ipv6Address` property methods take IPv4-mapped or IPv4-compatible addresses into account. See also the IPv4-in-IPv6 Address Support issue #85609 and #69772 which originally asked the question.
# Overview
In the recent PR #85655 I proposed changing `is_loopback` to take IPv4-mapped addresses into account, so `::ffff:127.0.0.1` would be recognized as a looback address. However, due to the points that came up in that PR, I alternatively propose the following: Keeping the current behaviour and commit to not assigning any special meaning for IPv4-in-IPv6 addresses, other than what the standards prescribe. This would apply to the stable method `is_loopback`, but also to currently unstable methods like `is_global` and `is_documentation` and any future methods. This is implemented in this PR as a change in documentation, specifically the following section:
> Both types of addresses are not assigned any special meaning by this implementation, other than what the relevant standards prescribe. This means that an address like `::ffff:127.0.0.1`, while representing an IPv4 loopback address, is not itself an IPv6 loopback address; only `::1` is. To handle these so called "IPv4-in-IPv6" addresses, they have to first be converted to their canonical IPv4 address.
# Discussion
In the discussion for or against supporting IPv4-in-IPv6 addresses the question what would be least surprising for users of other languages has come up several times. At first it seemed most big other languages supported IPv4-in-IPv6 addresses (or at least considered `::ffff:127.0.0.1` a loopback address). However after further investigation it appears that supporting IPv4-in-IPv6 addresses comes down to how a language represents addresses. .Net and Go do not have a separate type for IPv4 or IPv6 addresses, and do consider `::ffff:127.0.0.1` a loopback address. Java and Python, which do have separate types, do not consider `::ffff:127.0.0.1` a loopback address. Seeing as Rust has the separate `Ipv6Addr` type, it would make sense to also not support IPv4-in-IPv6 addresses. Note that this focuses on IPv4-mapped addresses, no other language handles IPv4-compatible addresses.
Another issue that was raised is how useful supporting these IPv4-in-IPv6 addresses would be in practice. Again with the example of `::ffff:127.0.0.1`, considering it a loopback address isn't too useful as to use it with most of the socket APIs it has to be converted to an IPv4 address anyway. From that perspective it would be better to instead provide better ways for doing this conversion like stabilizing `to_ipv4_mapped` or introducing a `to_canonical` method.
A point in favour of not supporting IPv4-in-IPv6 addresses is that that is the behaviour Rust has always had, and that supporting it would require changing already stable functions like `is_loopback`. This also keeps the documentation of these functions simpler, as we only have to refer to the relevant definitions in the IPv6 specification.
# Decision
To make progress on the `ip` feature, a decision needs to be made on whether or not to support IPv4-in-IPv6 addresses.
There are several options:
- Keep the current implementation and commit to never supporting IPv4-in-IPv6 addresses (accept this PR).
- Support IPv4-in-IPv6 addresses in some/all `IPv6Addr` methods (accept PR #85655).
- Keep the current implementation and but not commit to anything yet (reject both this PR and PR #85655), this entire issue will however come up again in the stabilization of several methods under the `ip` feature.
There are more options, like supporting IPv4-in-IPv6 addresses in `IpAddr` methods instead, but to my knowledge those haven't been seriously argued for by anyone.
There is currently an FCP ongoing on PR #85655. I would ask the libs team for an alternative FCP on this PR as well, which if completed means the rejection of PR #85655, and the decision to commit to not supporting IPv4-in-IPv6 addresses.
If anyone feels there is not enough evidence yet to make the decision for or against supporting IPv4-in-IPv6 addresses, let me know and I'll do whatever I can to resolve it.
This simplifies checking common properties in an address-family-agnostic
way since since #86335 commits to not checking IPv4 semantics
of IPv4-mapped addresses in the `Ipv6Addr` property methods.
rfc3052 followup: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information for contributors, we may as well
remove it from crates in this repo.
Add documentation for `Ipv6MulticastScope`
Adds basic documentation to the unstable `Ipv6MulticastScope`, as well as marking it `#[non_exhaustive]` because future IETF RFCs may introduce additional scopes. The documentation mentions this in a section "Stability Guarantees":
> /// Not all possible values for a multicast scope have been assigned.
/// Future RFCs may introduce new scopes, which will be added as variants to this enum;
/// because of this the enum is marked as `#[non_exhaustive]`.
Move `os_str_bytes` to `sys::unix`
Followup to #84967, with `OsStrExt` and `OsStringExt` moved out of `sys_common`, there is no reason anymore for `os_str_bytes` to live in `sys_common` and not in sys. This pr moves it to the location `sys::unix::os_str` and reuses the code on other platforms via `#[path]` (as is common in `sys`) instead of importing.
Remove `Ipv4Addr::is_ietf_protocol_assignment`
This PR removes the unstable method `Ipv4Addr::is_ietf_protocol_assignment`, as I suggested in https://github.com/rust-lang/rust/issues/85612#issuecomment-847863404. The method was added in #60145, as far as I can tell primarily for the implementation of `Ipv4Addr::is_global` (addresses reserved for IETF protocol assignment are not globally reachable unless otherwise specified).
The method was added in 2019, but I haven't been able to find any open-source code using this method so far. I'm also having a hard time coming up with a usecase for specifically this method; knowing that an address is reserved for future protocols doesn't allow you to do much with it, especially since now some of those addresses are indeed assigned to a protocol and have their own behaviour (and might even be defined to be globally reachable, so if that is what you care about it is always more accurate to call `!is_global()`, instead of `is_ietf_protocol_assignment()`).
Because of these reasons, I propose removing the method (or alternatively make it a private helper for `is_global`) and also not introduce `Ipv6Addr::is_ietf_protocol_assignment` and `IpAddr::is_ietf_protocol_assignment` in the future.
Change environment variable getters to error recoverably
This PR changes the standard library environment variable getter functions to error recoverably (i.e. not panic) when given an invalid value.
On some platforms, it is invalid for environment variable names to contain `'\0'` or `'='`, or for their values to contain `'\0'`. Currently, the standard library panics when manipulating environment variables with names or values that violate these invariants. However, this behavior doesn't make a lot of sense, at least in the case of getters. If the environment variable is missing, the standard library just returns an error value, rather than panicking. It doesn't make sense to treat the case where the variable is invalid any differently from that. See the [internals thread](https://internals.rust-lang.org/t/why-should-std-var-panic/14847) for discussion. Thus, this PR changes the functions to error recoverably in this case as well.
If desired, I could change the functions that manipulate environment variables in other ways as well. I didn't do that here because it wasn't entirely clear what to change them to. Should they error silently or do something else? If someone tells me how to change them, I'm happy to implement the changes.
This fixes#86082, an ICE that arises from the current behavior. It also adds a regression test to make sure the ICE does not occur again in the future.
`@rustbot` label +T-libs
r? `@joshtriplett`
Bump bootstrap compiler to 1.55
Changing the cfgs for stdarch is missing, but my understanding is that we don't need to do it as part of this PR?
r? `@Mark-Simulacrum`
Add Linux-specific pidfd process extensions (take 2)
Continuation of #77168.
I addressed the following concerns from the original PR:
- make `CommandExt` and `ChildExt` sealed traits
- wrap file descriptors in `PidFd` struct representing ownership over the fd
- add `take_pidfd` to take the fd out of `Child`
- close fd when dropped
Tracking Issue: #82971
Move UnwindSafe, RefUnwindSafe, AssertUnwindSafe to core
They were previously only available in std::panic, not core::panic.
- https://doc.rust-lang.org/1.51.0/std/panic/trait.UnwindSafe.html
- https://doc.rust-lang.org/1.51.0/std/panic/trait.RefUnwindSafe.html
- https://doc.rust-lang.org/1.51.0/std/panic/struct.AssertUnwindSafe.html
Where this is relevant: trait objects! Inside a `#![no_std]` library it's otherwise impossible to have a struct holding a trait object, and at the same time can be used from downstream std crates in a way that doesn't interfere with catch_unwind.
```rust
// common library
#![no_std]
pub struct Thing {
pub(crate) x: &'static (dyn SomeTrait + Send + Sync),
}
pub(crate) trait SomeTrait {...}
```
```rust
// downstream application
fn main() {
let thing: library::Thing = ...;
let _ = std::panic::catch_unwind(|| { let _ = thing; }); // does not work :(
}
```
See a4131708e2/src/gradient.rs (L7-L15) for a real life example of needing to work around this problem. In particular that workaround would not even be viable if implementors of the trait were provided externally by a caller, as the `feature = "std"` would become non-additive in that case.
What happens without the UnwindSafe constraints:
```rust
fn main() {
let gradient = colorous::VIRIDIS;
let _ = std::panic::catch_unwind(|| { let _ = gradient; });
}
```
```console
error[E0277]: the type `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
--> src/main.rs:3:13
|
3 | let _ = std::panic::catch_unwind(|| { let _ = gradient; });
| ^^^^^^^^^^^^^^^^^^^^^^^^ `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)` may contain interior mutability and a reference may not be safely transferrable across a catch_unwind boundary
|
::: .rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:430:40
|
430 | pub fn catch_unwind<F: FnOnce() -> R + UnwindSafe, R>(f: F) -> Result<R> {
| ---------- required by this bound in `catch_unwind`
|
= help: within `Gradient`, the trait `RefUnwindSafe` is not implemented for `(dyn colorous::gradient::EvalGradient + Send + Sync + 'static)`
= note: required because it appears within the type `&'static (dyn colorous::gradient::EvalGradient + Send + Sync + 'static)`
= note: required because it appears within the type `Gradient`
= note: required because of the requirements on the impl of `UnwindSafe` for `&Gradient`
= note: required because it appears within the type `[closure@src/main.rs:3:38: 3:62]`
```
Make `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` warn by default
This PR makes the `SEMICOLON_IN_EXPRESSIONS_FROM_MACROS` lint warn by default.
To avoid showing a large number of un-actionable warnings to users, we only enable the lint for macros defined in the same crate. This ensures that users will be able to fix the warning by simply removing a semicolon.
In the future, I'd like to enable this lint unconditionally, and eventually make it into a hard error in a future edition. This PR is a step towards that goal.
[backtraces]: look for the `begin` symbol only after seeing `end`
On `x86_64-pc-windows-msvc`, we often get backtraces which look like
this:
```
10: 0x7ff77e0e9be5 - std::panicking::rust_panic_with_hook
11: 0x7ff77e0e11b4 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5769736bdb11136c
12: 0x7ff77e0e116f - std::sys_common::backtrace::__rust_end_short_backtrace::h61c7ecb1b55338ae
13: 0x7ff77e0f89dd - std::panicking::begin_panic::h8e60ef9f82a41805
14: 0x7ff77e0e108c - d
15: 0x7ff77e0e1069 - c
16: 0x7ff77e0e1059 - b
17: 0x7ff77e0e1049 - a
18: 0x7ff77e0e1039 - core::ptr::drop_in_place<std::rt::lang_start<()>::{{closure}}>::h1bfcd14d5e15ba81
19: 0x7ff77e0e1186 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5769736bdb11136c
20: 0x7ff77e0e100c - std::rt::lang_start::{{closure}}::ha054184bbf9921e3
```
Notice that `__rust_begin_short_backtrace` appears on frame 11 before
`__rust_end_short_backtrace` on frame 12. This is because in typical
release binaries without debug symbols, dbghelp.dll, which we use to walk
and symbolize the stack, does not know where CGU internal functions
start or end and so the closure invoked by `__rust_end_short_backtrace`
is incorrectly described as `__rust_begin_short_backtrace` because it
happens to be near that symbol.
While that can obviously change, this has been happening quite
consistently since #75048. Since this is a very small change to the std
and the change makes sense by itself, I think this is worth doing.
This doesn't completely resolve the situation for release binaries on
Windows, since without debug symbols, the stack printed can still show
incorrect symbol names (this is why the test uses `#[no_mangle]`) but it
does slightly improve the situation in that you see the same backtrace
you would see with `RUST_BACKTRACE=full` or in a debugger (without the
uninteresting bits at the top and bottom).
Fixes part of #87481
Fix may not to appropriate might not or must not
I went through and changed occurrences of `may not` to be more explicit with `might not` and `must not`.
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
On `x86_64-pc-windows-msvc`, we often get backtraces which look like
this:
```
10: 0x7ff77e0e9be5 - std::panicking::rust_panic_with_hook
11: 0x7ff77e0e11b4 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5769736bdb11136c
12: 0x7ff77e0e116f - std::sys_common::backtrace::__rust_end_short_backtrace::h61c7ecb1b55338ae
13: 0x7ff77e0f89dd - std::panicking::begin_panic::h8e60ef9f82a41805
14: 0x7ff77e0e108c - d
15: 0x7ff77e0e1069 - c
16: 0x7ff77e0e1059 - b
17: 0x7ff77e0e1049 - a
18: 0x7ff77e0e1039 - core::ptr::drop_in_place<std::rt::lang_start<()>::{{closure}}>::h1bfcd14d5e15ba81
19: 0x7ff77e0e1186 - std::sys_common::backtrace::__rust_begin_short_backtrace::h5769736bdb11136c
20: 0x7ff77e0e100c - std::rt::lang_start::{{closure}}::ha054184bbf9921e3
```
Notice that `__rust_begin_short_backtrace` appears on frame 11 before
`__rust_end_short_backtrace` on frame 12. This is because in typical
release binaries without debug symbols, dbghelp.dll, which we use to walk
and symbolize the stack, does not know where CGU internal functions
start or end and so the closure invoked by `__rust_end_short_backtrace`
is incorrectly described as `__rust_begin_short_backtrace` because it
happens to be near that symbol.
While that can obviously change, this has been happening quite
consistently since #75048. Since this is a very small change to the std
and the change makes sense by itself, I think this is worth doing.
This doesn't completely resolve the situation for release binaries on
Windows, since without debug symbols, the stack printed can still show
incorrect symbol names (this is why the test uses `#[no_mangle]`) but it
does slightly improve the situation in that you see the same backtrace
you would see with `RUST_BACKTRACE=full` or in a debugger (without the
uninteresting bits at the top and bottom).
I looked in stdlib and as @BurntSushi thought, `raw` is generally
used for raw pointers, or other hazardous kinds of thing. stdlib does
not have `into_parts` apart from the one I added to `IntoInnerError`.
I did an ad-hoc search of the rustdocs for my current game project
Otter, which includes quite a large number of dependencies.
`into_parts` seems heavily used for things quite like this.
So change this name.
Suggested-by: Andrew Gallant <jamslam@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
I didn't notice the submodule, which means I failed to re-export this
to make it actually-public.
Reported-by: Andrew Gallant <jamslam@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Make const panic!("..") work in Rust 2021.
During const eval, this replaces calls to core::panicking::panic_fmt and std::panicking::being_panic_fmt with a call to a new const fn: core::panicking::const_panic_fmt. That function uses fmt::Arguments::as_str() to get the str and calls panic_str with that instead.
panic!() invocations with formatting arguments are still not accepted, as the creation of such a fmt::Arguments cannot be done in constant functions right now.
r? `@RalfJung`
Stabilize core::task::ready!
_Tracking issue: https://github.com/rust-lang/rust/issues/70922_
This PR stabilizes the `task::ready!` macro. Similar to https://github.com/rust-lang/rust/pull/80886, this PR was waiting on https://github.com/rust-lang/rust/issues/74355 to be fixed.
The `task::ready!` API has existed in the futures ecosystem for several years, and was added on nightly last year in https://github.com/rust-lang/rust/pull/70817. The motivation for this macro is the same as it was back then: virtually every single manual future implementation makes use of this; so much so that it's one of the few things included in the [futures-core](https://docs.rs/futures-core/0.3.12/futures_core) library.
r? ``@tmandry``
cc/ ``@rust-lang/wg-async-foundations`` ``@rust-lang/libs``
## Example
```rust
use core::task::{Context, Poll};
use core::future::Future;
use core::pin::Pin;
async fn get_num() -> usize {
42
}
pub fn do_poll(cx: &mut Context<'_>) -> Poll<()> {
let mut f = get_num();
let f = unsafe { Pin::new_unchecked(&mut f) };
let num = ready!(f.poll(cx));
// ... use num
Poll::Ready(())
}
```
During const eval, this replaces calls to core::panicking::panic_fmt and
std::panicking::being_panic_fmt with a call to a new const fn:
core::panicking::const_panic_fmt. That function uses
fmt::Arguments::as_str() to get the str and calls panic_str with that
instead.
panic!() invocations with formatting arguments are still not accepted,
as the creation of such a fmt::Arguments cannot be done in constant
functions right now.
Use hashbrown's `extend_reserve()` in `HashMap`
When we added `extend_reserve()` to our implementation of `Extend` for `HashMap`, hashbrown didn't have a version we could use. Now that hashbrown has added it, we should use its version instead of implementing it ourself.
Update VxWork's UNIX support
1. VxWorks does not provide glibc
2. VxWorks does provide `sigemptyset` and `sigaddset`
Note: these changes are concurrent to [this PR](https://github.com/rust-lang/libc/pull/2295) in libc.
Stabilize `impl From<[(K, V); N]> for HashMap` (and friends)
In addition to allowing HashMap to participate in Into/From conversion, this adds the long-requested ability to use constructor-like syntax for initializing a HashMap:
```rust
let map = HashMap::from([
(1, 2),
(3, 4),
(5, 6)
]);
```
This addition is highly motivated by existing precedence, e.g. it is already possible to similarly construct a Vec from a fixed-size array:
```rust
let vec = Vec::from([1, 2, 3]);
```
...and it is already possible to collect a Vec of tuples into a HashMap (and vice-versa):
```rust
let vec = Vec::from([(1, 2)]);
let map: HashMap<_, _> = vec.into_iter().collect();
let vec: Vec<(_, _)> = map.into_iter().collect();
```
...and of course it is likewise possible to collect a fixed-size array of tuples into a HashMap ([but not vice-versa just yet](https://github.com/rust-lang/rust/issues/81615)):
```rust
let arr = [(1, 2)];
let map: HashMap<_, _> = std::array::IntoIter::new(arr).collect();
```
Therefore this addition seems like a no-brainer.
As for any impl, this would be insta-stable.
Stabilize `into_parts()` and `into_error()`
This stabilizes `IntoInnerError`'s `into_parts()` and `into_error()` methods, currently gated behind the `io_into_inner_error_parts` feature. The FCP has [already completed.](https://github.com/rust-lang/rust/issues/79704#issuecomment-880652967)
Closes#79704.
Document iteration order of `retain` functions
For `HashSet` and `HashMap`, this simply copies the comment from
`BinaryHeap::retain`.
For `BTreeSet` and `BTreeMap`, this adds an additional guarantee that
wasn't previously documented. I think that because these data structures
are inherently ordered and other functions guarantee ordered iteration,
it makes sense to provide this guarantee for `retain` as well.
Add comments explaining the unix command-line argument support.
Following up on #87236, add comments to the unix command-line argument
support explaining that the code doesn't mutate the system-provided
argc/argv, and that this is why the code doesn't need a lock or special
memory ordering.
r? ```@RalfJung```
Background:
Over the last year, pidfd support was added to the Linux kernel. This
allows interacting with other processes. In particular, this allows
waiting on a child process with a timeout in a race-free way, bypassing
all of the awful signal-handler tricks that are usually required.
Pidfds can be obtained for a child process (as well as any other
process) via the `pidfd_open` syscall. Unfortunately, this requires
several conditions to hold in order to be race-free (i.e. the pid is not
reused).
Per `man pidfd_open`:
```
· the disposition of SIGCHLD has not been explicitly set to SIG_IGN
(see sigaction(2));
· the SA_NOCLDWAIT flag was not specified while establishing a han‐
dler for SIGCHLD or while setting the disposition of that signal to
SIG_DFL (see sigaction(2)); and
· the zombie process was not reaped elsewhere in the program (e.g.,
either by an asynchronously executed signal handler or by wait(2)
or similar in another thread).
If any of these conditions does not hold, then the child process
(along with a PID file descriptor that refers to it) should instead
be created using clone(2) with the CLONE_PIDFD flag.
```
Sadly, these conditions are impossible to guarantee once any libraries
are used. For example, C code runnng in a different thread could call
`wait()`, which is impossible to detect from Rust code trying to open a
pidfd.
While pid reuse issues should (hopefully) be rare in practice, we can do
better. By passing the `CLONE_PIDFD` flag to `clone()` or `clone3()`, we
can obtain a pidfd for the child process in a guaranteed race-free
manner.
This PR:
This PR adds Linux-specific process extension methods to allow obtaining
pidfds for processes spawned via the standard `Command` API. Other than
being made available to user code, the standard library does not make
use of these pidfds in any way. In particular, the implementation of
`Child::wait` is completely unchanged.
Two Linux-specific helper methods are added: `CommandExt::create_pidfd`
and `ChildExt::pidfd`. These methods are intended to serve as a building
block for libraries to build higher-level abstractions - in particular,
waiting on a process with a timeout.
I've included a basic test, which verifies that pidfds are created iff
the `create_pidfd` method is used. This test is somewhat special - it
should always succeed on systems with the `clone3` system call
available, and always fail on systems without `clone3` available. I'm
not sure how to best ensure this programatically.
This PR relies on the newer `clone3` system call to pass the `CLONE_FD`,
rather than the older `clone` system call. `clone3` was added to Linux
in the same release as pidfds, so this shouldn't unnecessarily limit the
kernel versions that this code supports.
Unresolved questions:
* What should the name of the feature gate be for these newly added
methods?
* Should the `pidfd` method distinguish between an error occurring
and `create_pidfd` not being called?
add `Stdin::lines`, `Stdin::split` forwarder methods
Add forwarder methods `Stdin::lines` and `Stdin::split`, which consume
and lock a `Stdin` handle, and forward on to the corresponding `BufRead`
methods. This should make it easier for beginners to use those iterator
constructors without explicitly dealing with locks or lifetimes.
Replaces #86412.
~~Based on #86846 to get the tracking issue number for the `stdio_locked` feature.~~ Rebased after merge, so it's only one commit now.
r? `@joshtriplett`
`@rustbot` label +A-io +C-enhancement +D-newcomer-roadblock +T-libs-api
Following up on #87236, add comments to the unix command-line argument
support explaining that the code doesn't mutate the system-provided
argc/argv, and that this is why the code doesn't need a lock or special
memory ordering.
Simplify command-line argument initialization on unix
Simplify Rust's command-line argument initialization code on unix:
- The cleanup code isn't needed, because it was just zeroing out non-owning variables at runtime cleanup time. After 91c3eee173, Rust's command-line initialization code on unix no longer allocates `CString`s and a `Vec` at startup time.
- The `Mutex` isn't needed; if there's somehow a call to `args()` before argument initialization has happened, the code returns return an empty list, which we can do with a null check.
With these changes, a simple cdylib that doesn't use threads avoids getting `pthread_mutex_lock`/`pthread_mutex_unlock` in its symbol table.
Move asm! and global_asm! to core::arch
Follow-up to https://github.com/rust-lang/stdarch/pull/1183 .
Implements the libs-api team decision from rust-lang/rust#84019 (comment) .
In order to not break nightly users, this PR also adds the newly-moved items to the prelude. However, a decision will need to be made before stabilization as to whether these items should remain in the prelude. I will file an issue for this separately.
Fixes#84019 .
r? `@Amanieu`
Add diagnostic items for Clippy
This adds a bunch of diagnostic items to `std`/`core`/`alloc` functions, structs and traits used in Clippy. The actual refactorings in Clippy to use these items will be done in a different PR in Clippy after the next sync.
This PR doesn't include all paths Clippy uses, I've only gone through the first 85 lines of Clippy's [`paths.rs`](ecf85f4bdc/clippy_utils/src/paths.rs) (after rust-lang/rust-clippy#7466) to get some feedback early on. I've also decided against adding diagnostic items to methods, as it would be nicer and more scalable to access them in a nicer fashion, like adding a `is_diagnostic_assoc_item(did, sym::Iterator, sym::map)` function or something similar (Suggested by `@camsteffen` [on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/147480-t-compiler.2Fwg-diagnostics/topic/Diagnostic.20Item.20Naming.20Convention.3F/near/225024603))
There seems to be some different naming conventions when it comes to diagnostic items, some use UpperCamelCase (`BinaryHeap`) and some snake_case (`hashmap_type`). This PR uses UpperCamelCase for structs and traits and snake_case with the module name as a prefix for functions. Any feedback on is this welcome.
cc: rust-lang/rust-clippy#5393
r? `@Manishearth`
In the command-line argument initialization code, remove the Mutex
around the `ARGV` and `ARGC` variables, and simply check whether
ARGV is non-null before dereferencing it. This way, if either of
ARGV or ARGC is not initialized, we'll get an empty argument list.
This allows simple cdylibs to avoid having
`pthread_mutex_lock`/`pthread_mutex_unlock` appear in their symbol
tables if they don't otherwise use threads.
Add forwarder methods `Stdin::lines` and `Stdin::split`, which consume
and lock a `Stdin` handle, and forward on to the corresponding `BufRead`
methods. This should make it easier for beginners to use those iterator
constructors without explicitly dealing with locks or lifetimes.
stdio_locked: add tracking issue
Add the tracking issue number #86845 to the stability attributes for the implementation in #86799.
r? `@joshtriplett`
`@rustbot` label +A-io +C-cleanup +T-libs-api
Remove unstable `io::Cursor::remaining`
Adding `io::Cursor::remaining` in #86037 caused a conflict with the implementation of `bytes::Buf` for `io::Cursor`, leading to an error in nightly, see https://github.com/rust-lang/rust/issues/86369#issuecomment-867723485.
This fixes the error by temporarily removing the `remaining` function.
r? `@yaahc`
[docs] Clarify behaviour of f64 and f32::sqrt when argument is negative zero
From IEEE 754 section 6.3:
> Except that squareRoot(−0) shall be −0, every numeric squareRoot result shall have a positive sign.
Fix linker error
Currently, `fs::hard_link` determines whether platforms have `linkat` based on the OS, and uses `link` if they don't. However, this heuristic does not work well if a platform provides `linkat` on newer versions but not on older ones. On old MacOS, this currently causes a linking error.
This commit fixes `fs::hard_link` by telling it to use `weak!` on macOS. This means that, on that operating system, we now check for `linkat` at runtime and use `link` if it is not available.
Fixes#80804.
`@rustbot` label T-libs-impl
On old macos systems, `fs::hard_link()` will follow symlinks.
This changes the test `symlink_hard_link` to exit early on
these systems, so that tests can pass.
`weak!` is needed in a test in another module. With macros
1.0, importing `weak!` would require reordering module
declarations in `std/src/lib.rs`, which is a bit too
evil.
The way octal literals are written in IP addresses differs from the way
they are written in Rust code, so the way that octal/hex literals in IPs
are written is explictly mentioned.
Use diagnostic items instead of lang items for rfc2229 migrations
This PR removes the `Send`, `UnwindSafe` and `RefUnwindSafe` lang items introduced in https://github.com/rust-lang/rust/pull/84730, and uses diagnostic items instead to check for `Send`, `UnwindSafe` and `RefUnwindSafe` traits for RFC2229 migrations.
r? ```@nikomatsakis```
rewrote documentation for thread::yield_now()
The old documentation suggested the use of yield_now for repeated
polling instead of discouraging it; it also made the false claim that
channels are implemented using yield_now. (They are not, except for
a corner case).
Add Integer::log variants
_This is another attempt at landing https://github.com/rust-lang/rust/pull/70835, which was approved by the libs team but failed on Android tests through Bors. The text copied here is from the original issue. The only change made so far is the addition of non-`checked_` variants of the log methods._
_Tracking issue: #70887_
---
This implements `{log,log2,log10}` methods for all integer types. The implementation was provided by `@substack` for use in the stdlib.
_Note: I'm not big on math, so this PR is a best effort written with limited knowledge. It's likely I'll be getting things wrong, but happy to learn and correct. Please bare with me._
## Motivation
Calculating the logarithm of a number is a generally useful operation. Currently the stdlib only provides implementations for floats, which means that if we want to calculate the logarithm for an integer we have to cast it to a float and then back to an int.
> would be nice if there was an integer log2 instead of having to either use the f32 version or leading_zeros() which i have to verify the results of every time to be sure
_— [`@substack,` 2020-03-08](https://twitter.com/substack/status/1236445105197727744)_
At higher numbers converting from an integer to a float we also risk overflows. This means that Rust currently only provides log operations for a limited set of integers.
The process of doing log operations by converting between floats and integers is also prone to rounding errors. In the following example we're trying to calculate `base10` for an integer. We might try and calculate the `base2` for the values, and attempt [a base swap](https://www.rapidtables.com/math/algebra/Logarithm.html#log-rules) to arrive at `base10`. However because we're performing intermediate rounding we arrive at the wrong result:
```rust
// log10(900) = ~2.95 = 2
dbg!(900f32.log10() as u64);
// log base change rule: logb(x) = logc(x) / logc(b)
// log2(900) / log2(10) = 9/3 = 3
dbg!((900f32.log2() as u64) / (10f32.log2() as u64));
```
_[playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=6bd6c68b3539e400f9ca4fdc6fc2eed0)_
This is somewhat nuanced as a lot of the time it'll work well, but in real world code this could lead to some hard to track bugs. By providing correct log implementations directly on integers we can help prevent errors around this.
## Implementation notes
I checked whether LLVM intrinsics existed before implementing this, and none exist yet. ~~Also I couldn't really find a better way to write the `ilog` function. One option would be to make it a private method on the number, but I didn't see any precedent for that. I also didn't know where to best place the tests, so I added them to the bottom of the file. Even though they might seem like quite a lot they take no time to execute.~~
## References
- [Log rules](https://www.rapidtables.com/math/algebra/Logarithm.html#log-rules)
- [Rounding error playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=6bd6c68b3539e400f9ca4fdc6fc2eed0)
- [substack's tweet asking about integer log2 in the stdlib](https://twitter.com/substack/status/1236445105197727744)
- [Integer Logarithm, A. Jaffer 2008](https://people.csail.mit.edu/jaffer/III/ilog.pdf)
The old documentation suggested the use of yield_now for repeated
polling instead of discouraging it; it also made the false claim that
channels are implementing using yield_now. (They are not, except for
a corner case).
Before this, `std`'s env var getter functions would panic on
receiving certain invalid inputs. This commit makes them
return a `None` or `Err` instead.
Remove some doc aliases
As per the new doc alias policy in https://github.com/rust-lang/std-dev-guide/pull/25, this removes some controversial doc aliases:
- `malloc`, `alloc`, `realloc`, etc.
- `length` (alias for `len`)
- `delete` (alias for `remove` in collections and also file/directory deletion)
r? `@joshtriplett`
Stabilize `Seek::rewind()`
This stabilizes `Seek::rewind`. It seemed to fit into one of the existing tests, so I extended that test rather than adding a new one.
Closes#85149.
aborts: Clarify documentation and comments
In the docs for intrinsics::abort():
* Strengthen the recommendation by to use process::abort instead.
* Document the fact that it sometimes (ab)uses an LLVM debug trap and what the likely consequences are.
* State that the precise behaviour is unstable.
In the docs for process::abort():
* Promise that we have the same behaviour as C `abort()`.
* Document the likely consequences, including, specifically, the consequences on Unix.
In the internal comment for unix::abort_internal:
* Refer to the public docs for the public API functions.
* Correct and expand the description of libc::abort. Specifically:
* Do not claim that abort() unregisters signal handlers. It doesn't; it honours the SIGABRT handler.
* Discuss, extensively, the issue with abort() flushing stdio buffers.
* Describe the glibc behaviour in some detail.
Co-authored-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Fixes#40230
Add std::os::unix::fs::DirEntryExt2::file_name_ref(&self) -> &OsStr
Greetings!
This is my first PR here, so please forgive me if I've missed an important step or otherwise done something wrong. I'm very open to suggestions/fixes/corrections.
This PR adds a function that allows `std::fs::DirEntry` to vend a borrow of its filename on Unix platforms, which is especially useful for sorting. (Windows has (as I understand it) encoding differences that require an allocation.) This new function sits alongside the cross-platform [`file_name(&self) -> OsString`](https://doc.rust-lang.org/std/fs/struct.DirEntry.html#method.file_name) function.
I pitched this idea in an [internals thread](https://internals.rust-lang.org/t/allow-std-direntry-to-vend-borrows-of-its-filename/14328/4), and no one objected vehemently, so here we are.
I understand features in general, I believe, but I'm not at all confident that my whole-cloth invention of a new feature string (as required by the compiler) was correct (or that the name is appropriate). Further, there doesn't appear to be a test for the sibling `ino` function, so I didn't add one for this similarly trivial function either. If it's desirable that I should do so, I'd be happy to [figure out how to] do that.
The following is a trivial sample of a use-case for this function, in which directory entries are sorted without any additional allocations:
```rust
use std::os::unix::fs::DirEntryExt;
use std::{fs, io};
fn main() -> io::Result<()> {
let mut entries = fs::read_dir(".")?.collect::<Result<Vec<_>, io::Error>>()?;
entries.sort_unstable_by(|a, b| a.file_name_ref().cmp(b.file_name_ref()));
for p in entries {
println!("{:?}", p);
}
Ok(())
}
```
In the docs for intrinsics::abort():
* Strengthen the recommendation by to use process::abort instead.
* Document the fact that it (ab)uses an LLVM debug trap and what the
likely consequences are.
* State that the precise behaviour is unstable.
In the docs for process::abort():
* Promise that we have the same behaviour as C `abort()`.
* Document the likely consequences, including, specifically, the
consequences on Unix.
In the internal comment for unix::abort_internal:
* Refer to the public docs for the public API functions.
* Correct and expand the description of libc::abort. Specifically:
* Do not claim that abort() unregisters signal handlers. It doesn't;
it honours the SIGABRT handler.
* Discuss, extensively, the issue with abort() flushing stdio buffers.
* Describe the glibc behaviour in some detail.
Co-authored-by: Mark Wooding <mdw@distorted.org.uk>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Remove the deprecated `core::raw` and `std::raw` module.
A few months has passed since #84207. I think now it's time for the final removal.
Closes#27751.
r? `@m-ou-se`
When using `process::Command` on Windows, environment variable names must be case-preserving but case-insensitive
When using `Command` to set the environment variables, the key should be compared as uppercase Unicode but when set it should preserve the original case.
Fixes#85242
add owned locked stdio handles
Add stderr_locked, stdin_locked, and stdout_locked free functions
to obtain owned locked stdio handles in a single step. Also add
into_lock methods to consume a stdio handle and return an owned
lock. These methods will make it easier to use locked stdio
handles without having to deal with lifetime problems or keeping
bindings to the unlocked handles around.
Fixes#85383; enables #86412.
r? `@joshtriplett`
`@rustbot` label +A-io +C-enhancement +D-newcomer-roadblock +T-libs-api
More ErrorKinds for common errnos
From the commit message of the main commit here (as revised):
```
There are a number of IO error situations which it would be very
useful for Rust code to be able to recognise without having to resort
to OS-specific code. Taking some Unix examples, `ENOTEMPTY` and
`EXDEV` have obvious recovery strategies. Recently I was surprised to
discover that `ENOSPC` came out as `ErrorKind::Other`.
Since I am familiar with Unix I reviwed the list of errno values in
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/errno.h.html
Here, I add those that most clearly seem to be needed.
`@CraftSpider` provided information about Windows, and references, which
I have tried to take into account.
This has to be insta-stable because we can't sensibly have a different
set of ErrorKinds depending on a std feature flag.
I have *not* added these to the mapping tables for any operating
systems other than Unix and Windows. I hope that it is OK to add them
now for Unix and Windows now, and maybe add them to other OS's mapping
tables as and when someone on that OS is able to consider the
situation.
I adopted the general principle that it was usually a bad idea to map
two distinct error values to the same Rust error code. I notice that
this principle is already violated in the case of `EACCES` and
`EPERM`, which both map to `PermissionDenied`. I think this was
probably a mistake but it would be quite hard to change now, so I
don't propose to do anything about that.
However, for Windows, there are sometimes different error codes for
identical situations. Eg there are WSA* versions of some error
codes as well as ERROR_* ones. Also Windows seems to have a great
many more erorr codes. I don't know precisely what best practice
would be for Windows.
```
<strike>
```
Errno values I wasn't sure about so *haven't* included:
EMFILE ENFILE ENOBUFS ENOLCK:
These are all fairly Unix-specific resource exhaustion situations.
In practice it seemed not very likely to me that anyone would want
to handle these differently to `Other`.
ENOMEM ERANGE EDOM EOVERFLOW
Normally these don't get exposed to the Rust callers I hope. They
don't tend to come out of filesystem APIs.
EILSEQ
Hopefully Rust libraries open files in binary mode and do the
converstion in Rust. So Rust code ought not to be exposed to
EILSEQ.
EIO
The range of things that could cause this is troublesome. I found
it difficult to describe. I do think it would be useful to add this
at some point, because EIO on a filesystem operation is much more
serious than most other errors.
ENETDOWN
I wasn't sure if this was useful or, indeed, if any modern systems
use it.
ENOEXEC
It is not clear to me how a Rust program could respond to this. It
seems rather niche.
EPROTO ENETRESET ENODATA ENOMSG ENOPROTOOPT ENOSR ENOSTR ETIME
ENOTRECOVERABLE EOWNERDEAD EBADMSG EPROTONOSUPPORT EPROTOTYPE EIDRM
These are network or STREAMS related errors which I have never in
my own Unix programming found the need to do anything with. I think
someone who understands these better should be the one to try to
find good Rust names and descriptions for them.
ENOTTY ENXIO ENODEV EOPNOTSUPP ESRCH EALREADY ECANCELED ECHILD
EINPROGRESS
These are very hard to get unless you're already doing something
very Unix-specific, in which case the raw_os_error interface is
probably more suitable than relying on the Rust ErrorKind mapping.
EFAULT EBADF
These would seem to be the result of application UB.
```
</strike>
<i>(omitted errnos are discussed below, especially in https://github.com/rust-lang/rust/pull/79965#issuecomment-810468334)
Fix double import in wasm thread
The `unsupported` type is imported two times, as `super::unsupported` and as `crate::sys::unsupported`, throwing an error. Remove `super::unsupported` in favor of the other.
As reported in #86802.
Fix#86802
Remove & from Command::args calls in documentation
Now that arrays implement `IntoIterator`, using `&` is no longer necessary. This makes examples easier to understand.
Merge `sys_common::bytestring` back into `os_str_bytes`
`bytestring` contains code for correctly debug formatting a byte slice (`[u8]`). This functionality is and has historically only been used to provide the debug formatting of byte-based os-strings (on unix etc.).
Having this functionality in the separate `bytestring` module was useful in the past to reduce duplication, as [when it was added](https://github.com/rust-lang/rust/pull/46798) `os_str_bytes` was still split into `sys::{unix, redox, wasi, etc.}::os_str`. However, now that is no longer the case, there is not much reason for the `bytestring` functionality to be separate from `os_str_bytes`; I don't think it is very likely that another part of std will need to handle formatting byte strings that are not os-strings in the future (everything should be `utf8`). This is why this PR merges the functionality of `bytestring` directly into the debug implementation in `os_str_bytes`.
The `unsupported` type is imported two times, as `super::unsupported` and as `crate::sys::unsupported`, throwing an error. Remove `super::unsupported` in favor of the other.
Redefine `ErrorKind::Other` and stop using it in std.
This implements the idea I shared yesterday in the libs meeting when we were discussing how to handle adding new `ErrorKind`s to the standard library: This redefines `Other` to be for *user defined errors only*, and changes all uses of `Other` in the standard library to a `#[doc(hidden)]` and permanently `#[unstable]` `ErrorKind` that users can not match on. This ensures that adding `ErrorKind`s at a later point in time is not a breaking change, since the user couldn't match on these errors anyway. This way, we use the `#[non_exhaustive]` property of the enum in a more effective way.
Open questions:
- How do we check this change doesn't cause too much breakage? Will a crate run help and be enough?
- How do we ensure we don't accidentally start using `Other` again in the standard library? We don't have a `pub(not crate)` or `#[deprecated(in this crate only)]`.
cc https://github.com/rust-lang/rust/pull/79965
cc `@rust-lang/libs` `@ijackson`
r? `@dtolnay`
Add stderr_locked, stdin_locked, and stdout_locked free functions
to obtain owned locked stdio handles in a single step. Also add
into_lock methods to consume a stdio handle and return an owned
lock. These methods will make it easier to use locked stdio
handles without having to deal with lifetime problems or keeping
bindings to the unlocked handles around.
For `HashSet` and `HashMap`, this simply copies the comment from
`BinaryHeap::retain`.
For `BTreeSet` and `BTreeMap`, this adds an additional guarantee that
wasn't previously documented. I think that because these data structures
are inherently ordered and other functions guarantee ordered iteration,
it makes sense to provide this guarantee for `retain` as well.
The omission of Sink: Default is causing me a slight inconvenience in
a test harness. There seems little reason for this and Empty not to
be Clone and Copy too.
I have made all three of these insta-stable, because:
AIUI Copycan only be derived, and I was not able to find any
examples of how to unstably derive it. I think it is probably not
possible.
I hunted through the git history for precedent and found
79b8ad84c8
Implement `Copy` for `IoSlice`
https://github.com/rust-lang/rust/pull/69403
which was also insta-stable.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Use HTTPS links where possible
While looking at #86583, I wondered how many other (insecure) HTTP links were in `rustc`. This changes most other `http` links to `https`. While most of the links are in comments or documentation, there are a few other HTTP links that are used by CI that are changed to HTTPS.
Notes:
- I didn't change any to or in licences
- Some links don't support HTTPS :(
- Some `http` links were dead, in those cases I upgraded them to their new places (all of which used HTTPS)
Use `#[non_exhaustive]` where appropriate
Due to the std/alloc split, it is not possible to make `alloc::collections::TryReserveError::AllocError` non-exhaustive without having an unstable, doc-hidden method to construct (which negates the benefits from `#[non_exhaustive]`).
`@rustbot` label +C-cleanup +T-libs +S-waiting-on-review
Due to the std/alloc split, it is not possible to make
`alloc::collections::TryReserveError::AllocError` non-exhaustive without
having an unstable, doc-hidden method to construct (which negates the
benefits from `#[non_exhaustive]`.
Rollup of 11 pull requests
Successful merges:
- #85054 (Revert SGX inline asm syntax)
- #85182 (Move `available_concurrency` implementation to `sys`)
- #86037 (Add `io::Cursor::{remaining, remaining_slice, is_empty}`)
- #86114 (Reopen#79692 (Format symbols under shared frames))
- #86297 (Allow to pass arguments to rustdoc-gui tool)
- #86334 (Resolve type aliases to the type they point to in intra-doc links)
- #86367 (Fix comment about rustc_inherit_overflow_checks in abs().)
- #86381 (Add regression test for issue #39161)
- #86387 (Remove `#[allow(unused_lifetimes)]` which is now unnecessary)
- #86398 (Add regression test for issue #54685)
- #86493 (Say "this enum variant takes"/"this struct takes" instead of "this function takes")
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Add `io::Cursor::{remaining, remaining_slice, is_empty}`
Tracking issue: #86369
I came across an inconvenience when answering the following [Stack Overflow](https://stackoverflow.com/questions/67831170) question.
To get the remaining slice you have to call `buff.fill_buf().unwrap()`. Which in my opinion doesn't really tell you what is returned (in the context of Cursor). To improve readability and convenience when using Cursor i propose adding the method `remaining`.
The next thing i found inconvenient (unnecessary long) was detecting if the cursor reached the end. There are a few ways this can be achieved right now:
- `buff.fill_buf().unwrap().is_empty()`
- `buff.position() >= buff.get_ref().len()`
- `buff.bytes().next().is_none()`
Which all seem a bit unintuitive, hidden in trait documentations or just a bit long for such a simple task.
Therefor i propose another method called `is_empty`, maybe with another name, since this one may leave room for interpretation on what really is empty (the underlying slice, the remaining slice or maybe the position).
Since it seemed easier to create this PR instead of an RFC i did that, if an RFC is wanted, i can close this PR and write an RFC first.
Move `available_concurrency` implementation to `sys`
This splits out the platform-specific implementation of `available_concurrency` to the corresponding platforms under `sys`. No changes are made to the implementation.
Tidy didn't lint against this code being originally added outside of `sys` because of a bug (see #84677), this PR also reverts the exclusion that was introduced in that bugfix.
Tracking issue of `available_concurrency`: #74479
Add MIR pass to lower call to `core::slice::len` into `Len` operand
During some larger experiment with range analysis I've found that code like `let l = slice.len()` produces different MIR then one found in bound checks. This optimization pass replaces terminators that are calls to `core::slice::len` with just a MIR operand and Goto terminator.
It uses some heuristics to remove the outer borrow that is made to call `core::slice::len`, but I assume it can be eliminated, just didn't find how.
Would like to express my gratitude to `@oli-obk` who helped me a lot on Zullip
Move `OsStringExt` and `OsStrExt` to `std::os`
Moves the `OsStringExt` and `OsStrExt` traits and implementations from `sys_common` to `os`. `sys_common` is for abstractions over `sys` and shouldn't really contain publicly exported items.
This does introduce some duplication: the traits and implementations are now duplicated in `unix`, `wasi`, `hermit`, and `sgx`. However, I would argue that this duplication is no different to how something like `MetadataExt` is duplicated in `linux`, `vxworkx`, `redox`, `solaris` etc. The duplication also matches the fact that the traits on different platforms are technically distinct types: any platform is free to add it's own extra methods to the extension trait.
Change entry point to 🛡️ against 💥💥-payloads
Guard against panic payloads panicking within entrypoints, where it is
UB to do so.
Note that there are a number of tradeoffs to consider. For instance, I
considered guarding against accidental panics inside the `rt::init` and
`rt::cleanup` code as well, as it is not all that obvious these may not
panic, but doing so would mean that we initialize certain thread-local
slots unconditionally, which has its own problems.
Fixes#86030
r? `@m-ou-se`
Guard against panic payloads panicking within entrypoints, where it is
UB to do so.
Note that there are a number of implementation approaches to consider.
Some simpler, some more complicated. This particular solution is nice in
that it also guards against accidental implementation issues in
various pieces of runtime code, something we cannot prevent statically
right now.
Fixes#86030
Add has_data_left() to BufRead
This is a continuation of #40747 and also addresses #40745. The problem with the previous PR was that it had "eof" in its method name. This PR uses a more descriptive method name, but I'm open to changing it.
Dump mingw-64's error codes into our source tree.
I have verified with these runes:
$ f=library/std/src/sys/windows/c/errors.rs
$ diff -ub <(git-cat-file blob HEAD~:$f | sort) <(cat $f | perl -pe 's/WSABASEERR \+ (\d+)/10000 + $1/e' |sort) |grep ^- |less
that this does not change any existing values.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
We're going to add many more of these.
This commit is pure code motion, plus the necessary administrivia, as
I have veried with the following runes:
$ git-diff HEAD~ | grep '^+' |sort >plus
$ git-diff HEAD~ | grep '^-' | perl -pe 's/^-/+/' |sort >min
$ diff -ub min plus |less
The output is precisely the expected `mod` and `use` directives.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
DWORD is a type alias for u32, so this makes no difference.
But this entry is anomalous and in my forthcoming commits I am going
to import many errors wholesale, and I spotted that my wholesale
import didn't match what was here.
CC: Chris Denton <christophersdenton@gmail.com>
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
use ErrorKind::*;
I don't feel confident enough about Windows things to reorder this
alphabetically
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Rely on libc for correct integer types in os/unix/net/ancillary.rs.
This PR is a small maintainability improvement. It simplifies `unix/net/ancillary.rs` in `std` by removing the `cfg_ifs` for casting to the correct integer type, and just rely on libc to define the struct correctly.
Specialize `io::Bytes::size_hint` for more types
Improve the result of `<io::Bytes as Iterator>::size_hint` for some readers. I did not manage to specialize `SizeHint` for `io::Cursor`
Side question: would it be interesting for `io::Read` to have an optional `size_hint` method ?
Linear interpolation
#71016 is a previous attempt at implementation that was closed by the author. I decided to reuse the feature request issue (#71015) as a tracking issue. A member of the rust-lang org will have to edit the original post to be formatted correctly as I am not the issue's original author.
The common name `lerp` is used because it is the term used by most code in a wide variety of contexts; it also happens to be the recently chosen name of the function that was added to C++20.
To ensure symmetry as a method, this breaks the usual ordering of the method from `lerp(a, b, t)` to `t.lerp(a, b)`. This makes the most sense to me personally, and there will definitely be discussion before stabilisation anyway.
Implementing lerp "correctly" is very dififcult even though it's a very common building-block used in all sorts of applications. A good prior reading is [this proposal](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0811r2.html#linear-interpolation) for the C++20 lerp which talks about the various guarantees, which I've simplified down to:
1. Exactness: `(0.0).lerp(start, end) == start` and `(1.0).lerp(start, end) == end`
2. Consistency: `anything.lerp(x, x) == x`
3. Monotonicity: once you go up don't go down
Fun story: the version provided in that proposal, from what I understand, isn't actually monotonic.
I messed around with a *lot* of different lerp implementations because I kind of got a bit obsessed and I ultimately landed on one that uses the fused `mul_add` instruction. Floating-point lerp lore is hard to come by, so, just trust me when I say that this ticks all the boxes. I'm only 90% certain that it's monotonic, but I'm sure that people who care deeply about this will be there to discuss before stabilisation.
The main reason for using `mul_add` is that, in general, it ticks more boxes with fewer branches to be "correct." Although it will be slower on architectures without the fused `mul_add`, that's becoming more and more rare and I have a feeling that most people who will find themselves needing `lerp` will also have an efficient `mul_add` instruction available.
Rename IoSlice(Mut)::advance to advance_slice and add IoSlice(Mut)::advance
Also changes the signature of `advance_slice` to accept a `&mut &mut [IoSlice]`, not returning anything. This will better match the `IoSlice::advance` function.
Updates https://github.com/rust-lang/rust/issues/62726.
Link reference in `dyn` keyword documentation
The "read more" sentence formatted "object safety" as inline code
instead of providing a link to more information. This PR adds a link
to the Reference about this matter, as well as the page regarding trait
objects.
---
We could also put these links in the very first line (instead of the link to the
Book) and in the first paragraph which mentions the "object safe" requirement.
Personally, I think it's good to keep the link to the Book up-front as it's more
accessible than the Reference.
optimize Eq implementation for paths
Filesystems generally have a tree-ish structure which means paths are more likely to share a prefix than a suffix. Absolute paths are especially prone to share long prefixes.
quick benchmark consisting of a search through through a vec containing the absolute paths of all (1850) files in `compiler/`:
```
# old
test path::tests::bench_path_cmp ... bench: 227,407 ns/iter (+/- 2,162)
# new
test path::tests::bench_path_cmp ... bench: 64,976 ns/iter (+/- 1,142)
```
Remove `Ipv6Addr::is_unicast_site_local`
Removes the unstable method `Ipv6Addr::is_unicast_site_local`, see also #85604 where I have tried to summarize related discussion so far.
Unicast site-local addresses (`fec0::/10`) were deprecated in [IETF RFC #3879](https://datatracker.ietf.org/doc/html/rfc3879), see also [RFC #4291 Section 2.5.7](https://datatracker.ietf.org/doc/html/rfc4291#section-2.5.7). Any new implementation must no longer support the special behaviour of site-local addresses. This is mentioned in the docs of `is_unicast_site_local` and already implemented in `is_unicast_global`, which considers addresses in `fec0::/10` to have global scope, thus overlapping with `is_unicast_site_local`.
Given that RFC #3879 was published in 2004, long before Rust existed, and it is specified that any new implementation must no longer support the special behaviour of site-local addresses, I don't see how a user would ever have a need for `is_unicast_site_local`. It is also confusing that currently both `is_unicast_site_local` and `is_unicast_global` can be `true` for an address, but an address can actually only have a single scope. The deprecating RFC mentions that Site-Local scope was confusing to work with and that the classification of an address as either Link-Local or Global better matches the mental model of users.
There has been earlier discussion of removing `is_unicast_site_local` (https://github.com/rust-lang/rust/pull/60145#issuecomment-485970669) which decided against it, but that had the incorrect assumption that the method was already stable; it is not. (This confusion arose from the placement of the unstable attribute on the entire module, instead of on individual methods, resolved in #85672)
r? `@joshtriplett` as reviewer of all the related PRs
Stabilize {std, core}::prelude::rust_*.
This stabilizes the `{core, std}::prelude::{rust_2015, rust_2018, rust_2021}` modules.
The usage of these modules as the prelude in those editions was already stabilized. This just stabilizes the modules themselves, making it possible for a user to explicitly refer to them.
Tracking issue: https://github.com/rust-lang/rust/issues/85684
FCP on the RFC that included this finished here: https://github.com/rust-lang/rfcs/pull/3114#issuecomment-840577395
Add functions `Duration::try_from_secs_{f32, f64}`
These functions allow constructing a Duration from a floating point value that could be out of range without panicking.
Tracking issue: #83400
Explain non-dropped sender recv in docs
Original senders that are still hanging around could cause
Receiver::recv to not block since this is a potential footgun
for beginners, clarify more on this in the docs for readers to
be aware about it.
Maybe it would be better to show an example of the pattern where `drop(tx)` is used when it is being cloned multiple times? Although I have seen it in quite a few articles but I am surprised that this part is not very clear with the current words without careful reading.
> If the corresponding Sender has disconnected, or it disconnects while this call is blocking, this call will wake up and return Err to indicate that no more messages can ever be received on this channel. However, since channels are buffered, messages sent before the disconnect will still be properly received.
Some words there may seemed similar if I carefully read and relate it but if I am new, I probably does not know "drop" makes it "disconnected". So I mention the words "drop" and "alive" to make it more relatable to lifetime.
Original senders that are still hanging around could cause
Receiver::recv to not block since this is a potential footgun
for beginners, clarify more on this in the docs for readers to
be aware about it.
Fix minor tidbits in sender recv doc
Co-authored-by: Dylan DPC <dylan.dpc@gmail.com>
Add example for unbounded receive loops in doc
Show the drop(tx) pattern, based on tokio docs
https://tokio-rs.github.io/tokio/doc/tokio/sync/index.html
Fix example code for drop sender recv
Fix wording in sender docs
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
Updated code examples and wording in move keyword documentation
Had a conversation with someone on the Rust Discord who was confused by the move keyword documentation. Some of the wording is odd sounding ("owned by value" - what else can something be owned by?). Also, some of the examples used Copy types when demonstrating move, leading to variables still being accessible in the outer scope after the move, contradicting the examples' comments.
I changed the move keyword documentation a bit, removing that odd wording and changing all the examples to use non-Copy types
Multiple improvements to RwLocks
This PR replicates #77147, #77380 and #84650 on RWLocks :
- Split `sys_common::RWLock` in `StaticRWLock` and `MovableRWLock`
- Unbox rwlocks on some platforms (Windows, Wasm and unsupported)
- Simplify `RwLock::into_inner`
Notes to reviewers :
- For each target, I copied `MovableMutex` to guess if `MovableRWLock` should be boxed.
- ~A comment says that `StaticMutex` is not re-entrant, I don't understand why and I don't know whether it applies to `StaticRWLock`.~
r? `@m-ou-se`
Filesystems generally have a tree-ish structure which means
paths are more likely to share a prefix than a suffix. Absolute paths
are especially prone to share long prefixes.
Forwarding implementation for Seek trait's stream_position method
Forwarding implementations for `Seek` trait's `stream_position` were missed when it was stabilized in `1.51.0`
Add `Ipv6Addr::is_unicast`
Adds an unstable utility method `Ipv6Addr::is_unicast` under the feature flag `ip` (tracking issue: #27709).
Added for completeness with the other unicast methods (see also https://github.com/rust-lang/rust/issues/85604#issuecomment-848220455) and opposite of `is_multicast`.
Fix documentation style inconsistencies for IP addresses
Pulled out of #85655 as it is unrelated. Fixes some inconsistencies in the docs for IP addresses:
- Currently some addresses are backticked, some are not, this PR backticks everything consistently. (looks better imo)
- Lowercase hex-literals are used when writing addresses.
The "read more" sentence formatted "object safety" as inline code
instead of providing a link to more information. This PR adds a link
to the Reference about this matter, as well as the page regarding trait
objects.
Prior to this patch, the default panic message (resulting from calling
`panic_any(42);` for example), would print the following error message:
```
thread 'main' panicked at 'Box<Any>', ...
```
However, this should be `Box<dyn Any>` instead.
Possible errors when accessing file metadata are platform specific
In particular the `is_dir`, `is_file` and `exists` functions suggests that querying a file requires querying the directory. On Windows this is not normally true.
r? `@m-ou-se`
rustdoc: link to stable/beta docs consistently in documentation
This is an alternative to https://github.com/rust-lang/rust/pull/84941 which fixes the problem consistently by linking to stable/beta for *all* items, not just for primitives.
## User-facing changes
- Intra-doc links to primitives that currently go to rust-lang.org/nightly/std/primitive.x.html will start going to channel that rustdoc was built with. Nightly will continue going to /nightly; Beta will link to /beta; stable compilers will link to /1.52.1 (or whatever version they were built as).
- Cross-crate links from std to core currently go to /nightly unconditionally. They will start going to /1.52.0 on stable channels (but remain the same on nightly channels).
- Intra-crate links from std to std (or core to core) currently go to the same URL they are hosted at; they will continue to do so. Notably, this is different from everything else because it can preserve the distinction between /stable and /1.52.0 by using relative links.
Note that "links" includes both intra-doc links and rustdoc's own
automatically generated hyperlinks.
## Implementation changes
- Update the testsuite to allow linking to /beta and /1.52.1 in docs
- Use an html_root_url for the standard library that's dependent on the channel
This avoids linking to nightly docs on stable.
- Update rustdoc to use channel-dependent links for primitives from an
unknown crate
- Set DOC_RUST_LANG_ORG_CHANNEL from bootstrap to ensure it's in sync
- Include doc.rust-lang.org in the channel
cc Mark-Simulacrum - I know [you were dubious about this in the past](https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Rustdoc.20unconditionally.20links.20to.20nightly.20libstd.20docs/near/231223124), but I'm not quite sure why? I see this as "just a bugfix", I don't know why rustdoc should unconditionally link to nightly.
cc dtolnay who commented in https://github.com/rust-lang/rust/issues/30693:
> I would welcome a PR to solve this permanently if anyone has ideas for how. I don't believe we need an RFC.
Fixes https://github.com/rust-lang/rust/issues/30693 (note that issue is marked as feature-accepted, although I don't see where it was discussed).
## User-facing changes
- Intra-doc links to primitives that currently go to rust-lang.org/nightly/std/primitive.x.html will start going to channel that rustdoc was built with. Nightly will continue going to /nightly; Beta will link to /beta; stable compilers will link to /1.52.1 (or whatever version they were built as).
- Cross-crate links from std to core currently go to /nightly unconditionally. They will start going to /1.52.0 on stable channels (but remain the same on nightly channels).
- Intra-crate links from std to std (or core to core) currently go to the same URL they are hosted at; they will continue to do so. Notably, this is different from everything else because it can preserve the distinction between /stable and /1.52.0 by using relative links.
Note that "links" includes both intra-doc links and rustdoc's own
automatically generated hyperlinks.
## Implementation changes
- Update the testsuite to allow linking to /beta and /1.52.1 in docs
- Use an html_root_url for the standard library that's dependent on the channel
This avoids linking to nightly docs on stable.
- Update rustdoc to use channel-dependent links for primitives from an
unknown crate
- Set DOC_RUST_LANG_ORG_CHANNEL from bootstrap to ensure it's in sync
- Include doc.rust-lang.org in the channel
Support Android ndk versions `r23-beta3` and up
Since android ndk version `r23-beta3`, `libgcc` has been replaced with `libunwind`. This moves the linking of `libgcc`/`libunwind` into the `unwind` crate where we check if the system compiler can find `libunwind` and fall back to `libgcc` if needed.
Since android ndk version `r23-beta3`, `libgcc` has been replaced with
`libunwind`. This moves the linking of `libgcc`/`libunwind` into the
`unwind` crate where we check if the system compiler can find
`libunwind` and fall back to `libgcc` if needed.
- Split `sys_common::RWLock` between `StaticRWLock` and `MovableRWLock`
- Unbox `RwLock` on some platforms (Windows, Wasm and unsupported)
- Simplify `RwLock::into_inner`
To make way for a new IoSlice(Mut)::advance function that advances a
single slice.
Also changes the signature to accept a `&mut &mut [IoSlice]`, not
returning anything. This will better match the future IoSlice::advance
function.
Add #[track_caller] to panic_any
Report the panic location from the user code.
```rust
use std::panic;
use std::panic::panic_any;
fn main() {
panic::set_hook(Box::new(|panic_info| {
if let Some(location) = panic_info.location() {
println!(
"panic occurred in file '{}' at line {}",
location.file(),
location.line(),
);
} else {
println!("panic occurred but can't get location information...");
}
}));
panic_any(42);
}
````
Before:
`panic occurred in file '/rustc/ff2c947c00f867b9f012e28ba88cecfbe556f904/library/std/src/panic.rs' at line 59`
After:
`panic occurred in file 'src/main.rs' at line 17`
In particular the `is_dir`, `is_file` and `exists` functions says that querying a file requires querying the directory. On Windows this is not normally true.
Add inline attr to CString::into_inner so it can optimize out NonNull checks
It seems that currently if you convert any of the standard library's container to a pointer and then to a NonNull pointer, all will optimize out the NULL check except `CString`(https://godbolt.org/z/YPKW9G5xn),
because for some reason `CString::into_inner` isn't inlined even though it's a private function that should compile into a simple `mov` instruction.
Adding a simple `#[inline]` attribute solves this, code example:
```rust
use std::ffi::CString;
use std::ptr::NonNull;
pub fn cstring_nonull(mut n: CString) -> NonNull<i8> {
NonNull::new(CString::into_raw(n)).unwrap()
}
```
assembly before:
```asm
__ZN3wat14cstring_nonull17h371c755bcad76294E:
.cfi_startproc
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
callq __ZN3std3ffi5c_str7CString10into_inner17h28ece07b276e2878E
testq %rax, %rax
je LBB0_2
popq %rbp
retq
LBB0_2:
leaq l___unnamed_1(%rip), %rdi
leaq l___unnamed_2(%rip), %rdx
movl $43, %esi
callq __ZN4core9panicking5panic17h92a83fa9085a8f73E
.cfi_endproc
.section __TEXT,__const
l___unnamed_1:
.ascii "called `Option::unwrap()` on a `None` value"
l___unnamed_3:
.ascii "wat.rs"
.section __DATA,__const
.p2align 3
l___unnamed_2:
.quad l___unnamed_3
.asciz "\006\000\000\000\000\000\000\000\006\000\000\000(\000\000"
```
Assembly after:
```asm
__ZN3wat14cstring_nonull17h9645eb9341fb25d7E:
.cfi_startproc
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset %rbp, -16
movq %rsp, %rbp
.cfi_def_cfa_register %rbp
movq %rdi, %rax
popq %rbp
retq
.cfi_endproc
```
(Related discussion on zulip: https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/NonNull.20From.3CBox.3CT.3E.3E)
doc: clarify Mutex::try_lock, etc. errors
Clarify error returns from Mutex::try_lock, RwLock::try_read,
RwLock::try_write to make it more obvious that both poisoning
and the lock being already locked are possible errors.
Bump bootstrap compiler to beta 1.53.0
This PR bumps the bootstrap compiler to version 1.53.0 beta, as part of our usual release process (this was supposed to be Wednesday's step, but creating the beta release took longer than expected).
The PR also includes the "Bootstrap: skip rustdoc fingerprint for building docs" commit, see the reasoning [on Zulip](https://zulip-archive.rust-lang.org/241545trelease/88450153betabootstrap.html).
r? `@Mark-Simulacrum`
Fix `vxworks`
Some PRs made the `vxworks` target not build anymore. This PR fixes that:
- #82973: copy `ExitStatusError` implementation from `unix`.
- #84716: no `libc::chroot` available on `vxworks`, so for now don't implement `os::unix::fs::chroot`.
add an example to explain std::io::Read::read returning 0 in some cases
I have always found the explanation about `Read::read` returning 0 to indicate EOF but not indefinitely, so here's more info using Linux as example. I can also add example code if necessary
MSVC: Avoid using jmp stubs for dll function imports
Windows import libraries contain two symbols for every function: `__imp_FunctionName` and `FunctionName` (where `FunctionName` is the name of the function to be imported).
`__imp_FunctionName` contains the address of the imported function. This will be filled in by the Windows executable loader at runtime. `FunctionName` contains a jmp stub that simply jumps to the address given by `__imp_FunctionName`. E.g. it's a function that solely contains a single jmp instruction:
```asm
jmp __imp_FunctionName
```
When using an external DLL function in Rust, by default the linker will link to FunctionName, causing a bit of indirection at runtime. In Microsoft's C++ it's possible to instead tell it to insert calls to the address in `__imp_FunctionName` by using the `__declspec(dllimport)` attribute. In Rust it's possible to get effectively the same behaviour using the `#[link]` attribute on `extern` blocks.
----
The second commit also merges multiple `extern` blocks into one block. This is because otherwise Rust will currently create duplicate linker arguments for each block. In this case having duplicates shouldn't matter much other than the noise when displaying the linker command.
Windows implementation of feature `path_try_exists`
Draft of a Windows implementation of `try_exists` (#83186).
The first commit reorganizes the code so I would be interested to get some feedback on if this is a good idea or not. It moves the `Path::try_exists` function to `fs::exists`. leaving the former as a wrapper for the latter. This makes it easier to provide platform specific implementations and matches the `fs::metadata` function.
The other commit implements a Windows specific variant of `exists`. I'm still figuring out my approach so this is very much a first draft. Eventually this will need some more eyes from knowledgable Windows people.
Clarify error returns from Mutex::try_lock, RwLock::try_read,
RwLock::try_write to make it more obvious that both poisoning
and the lock being already locked are possible errors.
std: Don't inline TLS accessor on MinGW
This is causing [issues] on Cargo's own CI for MinGW and given the
original investigation there's no reason that MinGW should work when
MSVC doesn't, this this tweaks the MSVC exception to being a Windows exception.
[issues]: https://github.com/rust-lang/cargo/runs/2626676503?check_suite_focus=true#step:9:2453
Move `std::memchr` to `sys_common`
`std::memchr` is a thin abstraction over the different `memchr` implementations in `sys`, along with documentation and tests. The module is only used internally by `std`, nothing is exported externally. Code like this is exactly what the `sys_common` module is for, so this PR moves it there.
Update list of allowed aarch64 features
I recently added these features to std_detect for aarch64 linux, pending [review](https://github.com/rust-lang/stdarch/pull/1146).
I have commented any features not supported by LLVM 9, the current minimum version for Rust. Some (PAuth at least) were renamed between 9 & 12 and I've left them disabled. TME, however, is not in LLVM 9 but I've left it enabled.
See https://github.com/rust-lang/stdarch/issues/993
Rollup of 8 pull requests
Successful merges:
- #84717 (impl FromStr for proc_macro::Literal)
- #85169 (Add method-toggle to <details> for methods)
- #85287 (Expose `Concurrent` (private type in public i'face))
- #85315 (adding time complexity for partition_in_place iter method)
- #85439 (Add diagnostic item to `CStr`)
- #85464 (Fix UB in documented example for `ptr::swap`)
- #85470 (Fix invalid CSS rules for a:hover)
- #85472 (CTFE Machine: do not expose Allocation)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Introduce `sys_common::rt::rtprintpanic!` to replace `sys_common::util` functionality
This PR introduces a new macro `rtprintpanic!`, similar to `sys_common::util::dumb_print` and uses that macro to replace all `sys_common::util` functionality.
std: Attempt again to inline thread-local-init across crates
Issue #25088 has been part of `thread_local!` for quite some time now.
Historical attempts have been made to add `#[inline]` to `__getit`
in #43931, #50252, and #59720, but these attempts ended up not landing
at the time due to segfaults on Windows.
In the interim though with `const`-initialized thread locals AFAIK this
is the only remaining bug which is why you might want to use
`#[thread_local]` over `thread_local!`. As a result I figured it was
time to resubmit this and see how it fares on CI and if I can help
debugging any issues that crop up.
Closes#25088
Override `clone_from` for some types
Override `clone_from` method of the `Clone` trait for:
- `cell::RefCell`
- `cmp::Reverse`
- `io::Cursor`
- `mem::ManuallyDrop`
This can bring performance improvements.
Issue #25088 has been part of `thread_local!` for quite some time now.
Historical attempts have been made to add `#[inline]` to `__getit`
in #43931, #50252, and #59720, but these attempts ended up not landing
at the time due to segfaults on Windows.
In the interim though with `const`-initialized thread locals AFAIK this
is the only remaining bug which is why you might want to use
`#[thread_local]` over `thread_local!`. As a result I figured it was
time to resubmit this and see how it fares on CI and if I can help
debugging any issues that crop up.
Closes#25088
Provide ExitStatusError
Closes#73125
In MR #81452 "Add #[must_use] to [...] process::ExitStatus" we concluded that the existing arrangements in are too awkward so adding that `#[must_use]` is blocked on improving the ergonomics.
I wrote a mini-RFC-style discusion of the approach in https://github.com/rust-lang/rust/issues/73125#issuecomment-771092741
# Stabilization report
## Summary
This stabilizes using macro expansion in key-value attributes, like so:
```rust
#[doc = include_str!("my_doc.md")]
struct S;
#[path = concat!(env!("OUT_DIR"), "/generated.rs")]
mod m;
```
See the changes to the reference for details on what macros are allowed;
see Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455)
for alternatives that were considered and rejected ("why accept no more
and no less?")
This has been available on nightly since 1.50 with no major issues.
## Notes
### Accepted syntax
The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`). Note that decorators and the like may be able to observe other expression forms.
### Expansion ordering
Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input.
There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work).
## Test cases
- https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs
- https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs
The feature has also been dogfooded extensively in the compiler and
standard library:
- https://github.com/rust-lang/rust/pull/83329
- https://github.com/rust-lang/rust/pull/83230
- https://github.com/rust-lang/rust/pull/82641
- https://github.com/rust-lang/rust/pull/80534
## Implementation history
- Initial proposal: https://github.com/rust-lang/rust/issues/55414#issuecomment-554005412
- Experiment to see how much code it would break: https://github.com/rust-lang/rust/pull/67121
- Preliminary work to restrict expansion that would conflict with this
feature: https://github.com/rust-lang/rust/pull/77271
- Initial implementation: https://github.com/rust-lang/rust/pull/78837
- Fix for an ICE: https://github.com/rust-lang/rust/pull/80563
## Unresolved Questions
~~https://github.com/rust-lang/rust/pull/83366#issuecomment-805180738 listed some concerns, but they have been resolved as of this final report.~~
## Additional Information
There are two workarounds that have a similar effect for `#[doc]`
attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons:
```rust
macro_rules! forward_inner_docs {
($e:expr => $i:item) => {
#[doc = $e]
$i
};
}
forward_inner_docs!(include_str!("lib.rs") => struct S {});
```
This also works for other attributes (like `#[path = concat!(...)]`).
The other is to use `doc(include)`:
```rust
#![feature(external_doc)]
#[doc(include = "lib.rs")]
struct S {}
```
The first works, but is non-trivial for people to discover, and
difficult to read and maintain. The second is a strange special-case for
a particular use of the macro. This generalizes it to work for any use
case, not just including files.
I plan to remove `doc(include)` when this is stabilized. The
`forward_inner_docs` workaround will still compile without warnings, but
I expect it to be used less once it's no longer necessary.
Simplify `cfg(any(unix, target_os="redox"))` in example to just `cfg(unix)`
Update example for `OsString` that handled `redox` seperately from `unix`: Redox has been completely integrated under `target_family="unix"`, so `cfg(unix)` implies `target_os="redox"`
35dbef2350/compiler/rustc_target/src/spec/redox_base.rs (L26)
Expand WASI abbreviation in docs
I was pretty sure this was related to something for WebAssembly but wasn't 100% sure so I checked but even on these top-level docs I couldn't find the abbreviation expanded. I'm normally used to Rust docs being detailed and explanatory and writing abbreviations like this out in full at least once so I thought it was worth the change. Feel free to close this if it's too much.
Do not allocate or unwind after fork
### Objective scenarios
* Make (simple) panics safe in `Command::pre_exec_hook`, including most `panic!` calls, `Option::unwrap`, and array bounds check failures.
* Make it possible to `libc::fork` and then safely panic in the child (needed for the above, but this requirement means exposing the new raw hook API which the `Command` implementation needs).
* In singlethreaded programs, where panic in `pre_exec_hook` is already memory-safe, prevent the double-unwinding malfunction #79740.
I think we want to make panic after fork safe even though the post-fork child environment is only experienced by users of `unsafe`, beause the subset of Rust in which any panic is UB is really far too hazardous and unnatural.
#### Approach
* Provide a way for a program to, at runtime, switch to having panics abort. This makes it possible to panic without making *any* heap allocations, which is needed because on some platforms malloc is UB in a child forked from a multithreaded program (see https://github.com/rust-lang/rust/pull/80263#issuecomment-774272370, and maybe also the SuS [spec](https://pubs.opengroup.org/onlinepubs/9699919799/functions/fork.html)).
* Make that change in the child spawned by `Command`.
* Document the rules comprehensively enough that a programmer has a fighting chance of writing correct code.
* Test that this all works as expected (and in particular, that there aren't any heap allocations we missed)
Fixes#79740
#### Rejected (or previously attempted) approaches
* Change the panic machinery to be able to unwind without allocating, at least when the payload and message are both `'static`. This seems like it would be even more subtle. Also that is a potentially-hot path which I don't want to mess with.
* Change the existing panic hook mechanism to not convert the message to a `String` before calling the hook. This would be a surprising change for existing code and would not be detected by the type system.
* Provide a `raw_panic_hook` function to intercept panics in a way that doesn't allocate. (That was an earlier version of this MR.)
### History
This MR could be considered a v2 of #80263. Thanks to everyone who commented there. In particular, thanks to `@m-ou-se,` `@Mark-Simulacrum` and `@hyd-dev.` (Tagging you since I think you might be interested in this new MR.) Compared to #80263, this MR has very substantial changes and additions.
Additionally, I have recently (2021-04-20) completely revised this series following very helpful comments from `@m-ou-se.`
r? `@m-ou-se`
Some platforma (eg ARM64) apparently generate SIGTRAP for panic abort!
See eg
https://github.com/rust-lang/rust/pull/81858#issuecomment-840702765
This is probably a bug, but we don't want to entangle this MR with it.
When it's fixed, this commit should be reverted.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Add auto traits and clone trait migrations for RFC2229
This PR
- renames the existent RFC2229 migration `disjoint_capture_drop_reorder` to `disjoint_capture_migration`
- add additional migrations for auto traits and clone trait
Closesrust-lang/project-rfc-2229#29Closesrust-lang/project-rfc-2229#28
r? `@nikomatsakis`
It is unergnomic to have to say things like
bad.into_status().signal()
Implementing `ExitStatusExt` for `ExitStatusError` fixes this.
Unfortunately it does mean making a previously-infallible method
capable of panicing, although of course the existing impl remains
infallible.
The alternative would be a whole new `ExitStatusErrorExt` trait.
`<ExitStatus as ExitStatusExt>::into_raw()` is not particularly
ergonomic to call because of the often-required type annotation.
See for example the code in the test case in
library/std/src/sys/unix/process/process_unix/tests.rs
Perhaps we should provide equivalent free functions for `ExitStatus`
and `ExitStatusExt` in std::os::unix::process and maybe deprecate this
trait method. But I think that is for the future.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Closes#73125
This is in pursuance of
Issue #73127 Consider adding #[must_use] to std::process::ExitStatus
In
MR #81452 Add #[must_use] to [...] process::ExitStatus
we concluded that the existing arrangements in are too awkward
so adding that #[must_use] is blocked on improving the ergonomics.
I wrote a mini-RFC-style discusion of the approach in
https://github.com/rust-lang/rust/issues/73125#issuecomment-771092741
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Change param name (k to key and v to value) in std::env module
1. When I was reading code the ide displayed `k` and `v`, so I
thought it would be better to show key and value?
2. I noticed var method already uses `key` instead of `k` so it
is more consistent to use `key` instead of `k`?
Thanks
Emit errors/warns on some wrong uses of rustdoc attributes
This PR adds a few diagnostics:
- error if conflicting `#[doc(inline)]`/`#[doc(no_inline)]` are found
- introduce the `invalid_doc_attributes` lint (warn-by-default) which triggers:
- if a crate-level attribute is used on a non-`crate` item
- if `#[doc(inline)]`/`#[doc(no_inline)]` is used on a non-`use` item
The code could probably be improved but I wanted to get feedback first. Also, some of those changes could be considered breaking changes, so I don't know what the procedure would be? ~~And finally, for the warnings, they are currently hard warnings, maybe it would be better to introduce a lint?~~ (EDIT: introduced the `invalid_doc_attributes` lint)
Closes#80275.
r? `@jyn514`
fork fails there. The failure message is confusing: so c.status()
returns an Err, the closure panics, and the test thinks the panic was
propagated from inside the child.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Co-authored-by: Mara Bos <m-ou.se@m-ou.se>
Rearrange SGX split module files
In #75979 several inlined modules were split out into multiple files.
This PR keeps the multiple files but moves a few things around to
organize things in a coherent way.
Cleanup of `wasm`
Some more cleanup of `sys`, this time `wasm`
- Reuse `unsupported::args` (functionally equivalent implementation, just an empty iterator).
- Split out `atomics` implementation of `wasm::thread`, the non-`atomics` implementation is reused from `unsupported`.
- Move all of the `atomics` code to a separate directory `wasm/atomics`.
````@rustbot```` label: +T-libs-impl
r? ````@m-ou-se````
In #75979 several inlined modules were split out into multiple files.
This PR keeps the multiple files but moves a few things around to
organize things in a coherent way.
This tests that we can indeed safely panic after fork, both
a raw libc::fork and in a Command pre_exec hook.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Co-authored-by: Mara Bos <m-ou.se@m-ou.se>
This is safe (does not involve heap allocation) but we don't yet have
a test to ensure that stays true. That will come in a moment.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Co-authored-by: Mara Bos <m-ou.se@m-ou.se>
Unwinding after fork() in the child is UB on some platforms, because
on those (including musl) malloc can be UB in the child of a
multithreaded program, and unwinding must box for the payload.
Even if it's safe, unwinding past fork() in the child causes whatever
traps the unwind to return twice. This is very strange and clearly
not desirable. With the default behaviour of the thread library, this
can even result in a panic in the child being transformed into zero
exit status (ie, success) as seen in the parent!
Fixes#79740.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
We must change the atomic read on panic entry to `Acquire`, to pick up
a possible an `always_panic` on another thread.
We add `count` to the names of panic_count::get and ::is_zaero,
because now there is another reason why panic ought to maybe abort.
Renaming these ensures that we have checked every call site to ensure
that they don't need further adjustment.
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Co-authored-by: Mara Bos <m-ou.se@m-ou.se>
Stablize {HashMap,BTreeMap}::into_{keys,values}
I would propose to stabilize `{HashMap,BTreeMap}::into_{keys,values}`( aka. `map_into_keys_values`).
Closes#75294.
Rollup of 11 pull requests
Successful merges:
- #83553 (Update `ptr` docs with regards to `ptr::addr_of!`)
- #84183 (Update RELEASES.md for 1.52.0)
- #84709 (Add doc alias for `chdir` to `std::env::set_current_dir`)
- #84803 (Reduce duplication in `impl_dep_tracking_hash` macros)
- #84808 (Account for unsatisfied bounds in E0599)
- #84843 (use else if in std library )
- #84865 (rustbuild: Pass a `threads` flag that works to windows-gnu lld)
- #84878 (Clarify documentation for `[T]::contains`)
- #84882 (platform-support: Center the contents of the `std` and `host` columns)
- #84903 (Remove `rustc_middle::mir::interpret::CheckInAllocMsg::NullPointerTest`)
- #84913 (Do not ICE on invalid const param)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Add doc alias for `chdir` to `std::env::set_current_dir`
Searching for `chdir` in the Rust documentation produces no useful
results.
I wrote some code recently that called `libc::chdir` and manually
handled errors, because I didn't realize that the safe
`std::env::set_current_dir` existed. I searched for `chdir` and
`change_dir` and `change_directory` (the latter two based on the
precedent of unabbreviating set by `create_dir`), and I also read
through `std::fs` expecting to potentially find it there. Given that
none of those led to `std::env::set_current_dir`, I think that provides
sufficient justification to add this specific alias.
Update `ptr` docs with regards to `ptr::addr_of!`
This updates the documentation since `ptr::addr_of!` and `ptr::addr_of_mut!` are now stable. One might remove the distinction between the sections `# On packed structs` and `# Examples`, as the old section on packed structs was primarily to prevent users of doing undefined behavior, which is not necessary anymore.
Technically there is now wrong/outdated documentation on stable, but I don't think this is worth a point release 😉Fixes#83509.
``````````@rustbot`````````` modify labels: T-doc
Move all `sys::ext` modules to `os`
This PR moves all `sys::ext` modules to `os`, centralizing the location of all `os` code and simplifying the dependencies between `os` and `sys`.
Because this also removes all uses `cfg_if!` on publicly exported items, where after #81969 there were still a few left, this should properly work around https://github.com/rust-analyzer/rust-analyzer/issues/6038.
`@rustbot` label: +T-libs-impl
This updates the documentation since `ptr::addr_of!` and
`ptr::addr_of_mut!` are now stable. One might remove the distinction
between the sections `# On packed structs` and `# Examples`, as the old
section on packed structs was primarily to prevent users of doing unde-
fined behavior, which is not necessary anymore.
There is also a new section in "how to obtain a pointer", which referen-
ces the `ptr::addr_of!` macros.
This commit contains squashed commits from code review.
Co-authored-by: Joshua Nelson <joshua@yottadb.com>
Co-authored-by: Mara Bos <m-ou.se@m-ou.se>
Co-authored-by: Soveu <marx.tomasz@gmail.com>
Co-authored-by: Ralf Jung <post@ralfj.de>
