Add documentation to boxed conversions
Among other changes, documents whether allocations are necessary
to complete the type conversion.
Part of #51430, supersedes #89199
Update to Unicode 14.0
The Unicode Standard [announced Version 14.0](https://home.unicode.org/announcing-the-unicode-standard-version-14-0/) on September 14, 2021, and this pull request updates the generated tables in `core` accordingly.
This did require a little prep-work in `unicode-table-generator`. First, #81358 had modified the generated file instead of the tool, so that change is now reflected in the tool as well. Next, I found that the "Alphabetic" property in version 14 was panicking when generating a bitset, "cannot pack 264 into 8 bits". We've been using the skiplist for that anyway, so I changed this to fail gracefully. Finally, I confirmed that the tool still created the exact same tables for 13 before moving to 14.
Add 'core::array::from_fn' and 'core::array::try_from_fn'
These auxiliary methods fill uninitialized arrays in a safe way and are particularly useful for elements that don't implement `Default`.
```rust
// Foo doesn't implement Default
struct Foo(usize);
let _array = core::array::from_fn::<_, _, 2>(|idx| Foo(idx));
```
Different from `FromIterator`, it is guaranteed that the array will be fully filled and no error regarding uninitialized state will be throw. In certain scenarios, however, the creation of an **element** can fail and that is why the `try_from_fn` function is also provided.
```rust
#[derive(Debug, PartialEq)]
enum SomeError {
Foo,
}
let array = core::array::try_from_fn(|i| Ok::<_, SomeError>(i));
assert_eq!(array, Ok([0, 1, 2, 3, 4]));
let another_array = core::array::try_from_fn(|_| Err(SomeError::Foo));
assert_eq!(another_array, Err(SomeError::Foo));
```
Add #[must_use] to string/char transformation methods
These methods could be misconstrued as modifying their arguments instead of returning new values.
Where possible I made the note recommend a method that does mutate in place.
Parent issue: #89692
Fix minor std::thread documentation typo
callers of spawn_unchecked() need to make sure that the thread
not outlive references in the passed closure, not the other way around.
Optimize File::read_to_end and read_to_string
Reading a file into an empty vector or string buffer can incur unnecessary `read` syscalls and memory re-allocations as the buffer "warms up" and grows to its final size. This is perhaps a necessary evil with generic readers, but files can be read in smarter by checking the file size and reserving that much capacity.
`std::fs::read` and `std::fs::read_to_string` already perform this optimization: they open the file, reads its metadata, and call `with_capacity` with the file size. This ensures that the buffer does not need to be resized and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls `file.read_to_end` or `file.read_to_string` they do not get this optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of examples of this. They're found all over in unit tests, which isn't a big deal, but there are also several real instances in the compiler and in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the documentation for both the `Read` trait and the `Read::read_to_end` method both show this exact pattern as examples of how to use readers. What this says to me is that this shouldn't be solved by simply fixing the instances of it in this codebase. If it's here it's certain to be prevalent in the wider Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and `read_to_string` directly on `File`. This way it's no longer a minor footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as `impl Read` or `dyn Read` will benefit. For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to `self.inner`'s methods. That way it can call `File`'s optimized implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked `unsafe`.
- `File::read_to_string` being more efficient means that the performance note for `io::read_to_string` can be softened. I've added `@camelid's` suggested wording from https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502.
r? `@joshtriplett`
These methods could be misconstrued as modifying their arguments instead
of returning new values.
Where possible I made the note recommend a method that does mutate in
place.
Among other changes, documents whether allocations are necessary
to complete the type conversion.
Part of #51430
Co-authored-by: Giacomo Stevanato <giaco.stevanato@gmail.com>
Co-authored-by: Joshua Nelson <github@jyn.dev>
Implement #85440 (Random test ordering)
This PR adds `--shuffle` and `--shuffle-seed` options to `libtest`. The options are similar to the [`-shuffle` option](c894b442d1/src/testing/testing.go (L1482-L1499)) that was recently added to Go.
Here are the relevant parts of the help message:
```
--shuffle Run tests in random order
--shuffle-seed SEED
Run tests in random order; seed the random number
generator with SEED
...
By default, the tests are run in alphabetical order. Use --shuffle or set
RUST_TEST_SHUFFLE to run the tests in random order. Pass the generated
"shuffle seed" to --shuffle-seed (or set RUST_TEST_SHUFFLE_SEED) to run the
tests in the same order again. Note that --shuffle and --shuffle-seed do not
affect whether the tests are run in parallel.
```
Is an RFC needed for this?
Reading a file into an empty vector or string buffer can incur
unnecessary `read` syscalls and memory re-allocations as the buffer
"warms up" and grows to its final size. This is perhaps a necessary evil
with generic readers, but files can be read in smarter by checking the
file size and reserving that much capacity.
`std::fs::read` and `read_to_string` already perform this optimization:
they open the file, reads its metadata, and call `with_capacity` with
the file size. This ensures that the buffer does not need to be resized
and an initial string of small `read` syscalls.
However, if a user opens the `File` themselves and calls
`file.read_to_end` or `file.read_to_string` they do not get this
optimization.
```rust
let mut buf = Vec::new();
file.read_to_end(&mut buf)?;
```
I searched through this project's codebase and even here are a *lot* of
examples of this. They're found all over in unit tests, which isn't a
big deal, but there are also several real instances in the compiler and
in Cargo. I've documented the ones I found in a comment here:
https://github.com/rust-lang/rust/issues/89516#issuecomment-934423999
Most telling, the `Read` trait and the `read_to_end` method both show
this exact pattern as examples of how to use readers. What this says to
me is that this shouldn't be solved by simply fixing the instances of it
in this codebase. If it's here it's certain to be prevalent in the wider
Rust ecosystem.
To that end, this commit adds specializations of `read_to_end` and
`read_to_string` directly on `File`. This way it's no longer a minor
footgun to start with an empty buffer when reading a file in.
A nice side effect of this change is that code that accesses a `File` as
a bare `Read` constraint or via a `dyn Read` trait object will benefit.
For example, this code from `compiler/rustc_serialize/src/json.rs`:
```rust
pub fn from_reader(rdr: &mut dyn Read) -> Result<Json, BuilderError> {
let mut contents = Vec::new();
match rdr.read_to_end(&mut contents) {
```
Related changes:
- I also added specializations to `BufReader` to delegate to
`self.inner`'s methods. That way it can call `File`'s optimized
implementations if the inner reader is a file.
- The private `std::io::append_to_string` function is now marked
`unsafe`.
- `File::read_to_string` being more efficient means that the performance
note for `io::read_to_string` can be softened. I've added @camelid's
suggested wording from:
https://github.com/rust-lang/rust/issues/80218#issuecomment-936806502
Make cfg imply doc(cfg)
This is a reopening of #79341, rebased and modified a bit (we made a lot of refactoring in rustdoc's types so they needed to be reflected in this PR as well):
* `hidden_cfg` is now in the `Cache` instead of `DocContext` because `cfg` information isn't stored anymore on `clean::Attributes` type but instead computed on-demand, so we need this information in later parts of rustdoc.
* I removed the `bool_to_options` feature (which makes the code a bit simpler to read for `SingleExt` trait implementation.
* I updated the version for the feature.
There is only one thing I couldn't figure out: [this comment](https://github.com/rust-lang/rust/pull/79341#discussion_r561855624)
> I think I'll likely scrap the whole `SingleExt` extension trait as the diagnostics for 0 and >1 items should be different.
How/why should they differ?
EDIT: this part has been solved, the current code was fine, just needed a little simplification.
cc `@Nemo157`
r? `@jyn514`
Original PR description:
This is only active when the `doc_cfg` feature is active.
The implicit cfg can be overridden via `#[doc(cfg(...))]`, so e.g. to hide a `#[cfg]` you can use something like:
```rust
#[cfg(unix)]
#[doc(cfg(all()))]
pub struct Unix;
```
By adding `#![doc(cfg_hide(foobar))]` to the crate attributes the cfg `#[cfg(foobar)]` (and _only_ that _exact_ cfg) will not be implicitly treated as a `doc(cfg)` to render a message in the documentation.
Rename `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`
_Tracking issue: https://github.com/rust-lang/rust/issues/74479_
This PR renames `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`.
## Rationale
The API was initially named `std:🧵:hardware_concurrency`, mirroring the [C++ API of the same name](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency). We eventually decided to omit any reference to the word "hardware" after [this comment](https://github.com/rust-lang/rust/pull/74480#issuecomment-662045841). And so we ended up with `available_concurrency` instead.
---
For a talk I was preparing this week I was reading through ["Understanding and expressing scalable concurrency" (A. Turon, 2013)](http://aturon.github.io/academic/turon-thesis.pdf), and the following passage stood out to me (emphasis mine):
> __Concurrency is a system-structuring mechanism.__ An interactive system that deals with disparate asynchronous events is naturally structured by division into concurrent threads with disparate responsibilities. Doing so creates a better fit between problem and solution, and can also decrease the average latency of the system by preventing long-running computations from obstructing quicker ones.
> __Parallelism is a resource.__ A given machine provides a certain capacity for parallelism, i.e., a bound on the number of computations it can perform simultaneously. The goal is to maximize throughput by intelligently using this resource. For interactive systems, parallelism can decrease latency as well.
_Chapter 2.1: Concurrency is not Parallelism. Page 30._
---
_"Concurrency is a system-structuring mechanism. Parallelism is a resource."_ — It feels like this accurately captures the way we should be thinking about these APIs. What this API returns is not "the amount of concurrency available to the program" which is a property of the program, and thus even with just a single thread is effectively unbounded. But instead it returns "the amount of _parallelism_ available to the program", which is a resource hard-constrained by the machine's capacity (and can be further restricted by e.g. operating systems).
That's why I'd like to propose we rename this API from `available_concurrency` to `available_parallelism`. This still meets the criteria we previously established of not attempting to define what exactly we mean by "hardware", "threads", and other such words. Instead we only talk about "concurrency" as an abstract resource available to our program.
r? `@joshtriplett`
refactor: make VecDeque's IterMut fields module-private, not just crate-private
Made the fields of VecDeque's IterMut private by creating a IterMut::new(...) function to create a new instance of IterMut and migrating usage to use IterMut::new(...).
refactor: VecDeques Drain fields to private
Made the fields of VecDeque's Drain private by creating a Drain::new(...) function to create a new instance of Drain and migrating usage to use Drain::new(...).
Expand documentation for `FpCategory`.
I intend these changes to be helpful to readers who are not yet familiar with the quirks of floating-point numbers. Additionally, I felt it was misleading to describe `Nan` as being the result of division by zero, since most divisions by zero (except for 0/0) produce `Infinite` floats, so I moved that remark to the `Infinite` variant with adjustment.
The first sentence of the `Nan` documentation is copied from `f32`; I followed the example of the `f64` documentation by referring to `f32` for general concepts, rather than duplicating the text.
----
I considered making similar changes to the documentation of the `is_*` methods of floats, but decided that that was a much larger and trickier problem; here, each of the variants' descriptions can be expected to be read in context of being mutually exclusive with the others.
Fix Lower/UpperExp formatting for integers and precision zero
Fixes the integer part of #89493 (I daren't touch the floating-point formatting code). The issue is that the "subtracted" precision essentially behaves like extra trailing zeros, but this is not currently reflected in the code properly.
for signed wrapping remainder, do not compare lhs with MIN
Since the wrapped remainder is going to be 0 for all cases when the rhs is -1, there is no need to compare the lhs with MIN.
Made the fields of VecDeque's IterMut private by creating a IterMut::new(...) function to create a new instance of IterMut and migrating usage to use IterMut::new(...).
Since the wrapped remainder is going to be 0 for all cases when the rhs is -1,
there is no need to divide in this case. Comparing the lhs with MIN is only done
for the overflow bool. In particular, this results in better code generation for
wrapping remainder, which discards the overflow bool completely.
refactor: VecDeques PairSlices fields to private
Reducing VecDeque's PairSlices fields to private, a `from(...)` method is already used to create PairSlices.
Use the 64b inner:monotonize() implementation not the 128b one for aarch64
aarch64 prior to v8.4 (FEAT_LSE2) doesn't have an instruction that guarantees
untorn 128b reads except for completing a 128b load/store exclusive pair
(ldxp/stxp) or compare-and-swap (casp) successfully. The requirement to
complete a 128b read+write atomic is actually more expensive and more unfair
than the previous implementation of monotonize() which used a Mutex on aarch64,
especially at large core counts. For aarch64 switch to the 64b atomic
implementation which is about 13x faster for a benchmark that involves many
calls to Instant::now().
Correctly handle supertraits for min_specialization
Supertraits of specialization markers could circumvent checks for
min_specialization. Elaborating predicates prevents this.
r? ````@nikomatsakis````
path.push() should work as expected on windows verbatim paths
On Windows, std::fs::canonicalize() returns an so-called UNC path. UNC paths differ with regular paths because:
- This type of path can much longer than a non-UNC path (32k vs 260 characters).
- The prefix for a UNC path is ``Component::Prefix(Prefix::DiskVerbatim(..)))``
- No `/` is allowed
- No `.` is allowed
- No `..` is allowed
Rust has poor handling of such paths. If you join a UNC path with a path with any of the above, then this will not work.
I've implemented a new method `fn join_fold()` which joins paths and also removes any `.` and `..` from it, and replaces `/` with `\` on Windows. Using this function it is possible to use UNC paths without issue. In addition, this function is useful on Linux too; paths can be appended without having to call `canonicalize()` to remove the `.` and `..`.
This PR needs test cases, which can I add. I hope this will a start of a discussion.
Stabilize `const_panic`
Closes#51999
FCP completed in #89006
```@rustbot``` label +A-const-eval +A-const-fn +T-lang
cc ```@oli-obk``` for review (not `r?`'ing as not on lang team)
Improve wording of `map_or_else` docs
Changes doc text to refer to the "default" parameter as the "default"
function.
Previously, the doc text referred to the "f" parameter as the "default" function; and the "default" parameter as the "fallback" function.
implement advance_(back_)_by on more iterators
Add more efficient, non-default implementations for `feature(iter_advance_by)` (#77404) on more iterators and adapters.
This PR only contains implementations where skipping over items doesn't elide any observable side-effects such as user-provided closures or `clone()` functions. I'll put those in a separate PR.
Add truncate note to Vec::resize
A very minor addition to the `Vec::resize` documentation to point out the `truncate` method.
When I was searching for something matching `truncate` I managed to miss it, along with some colleagues. We later found it by chance. We did find `resize` however, so I was hoping to point it out in the documentation.
Partially stabilize `array_methods`
This stabilizes `<[T; N]>::as_slice` and `<[T; N]>::as_mut_slice`, which is forms part of the `array_methods` feature: #76118.
This also makes `<[T; N]>::as_slice` const due to its trivial nature.
Manual Debug for Unix ExitCode ExitStatus ExitStatusError
These structs have misleading names. An ExitStatus[Error] is actually a Unix wait status; an ExitCode is actually an exit status. These misleading names appear in the `Debug` output.
The `Display` impls on Unix have been improved, but the `Debug` impls are still misleading, as reported in #74832.
Fix this by pretending that these internal structs are called `unix_exit_status` and `unix_wait_status` as applicable. (We can't actually rename the structs because of the way that the cross-platform machinery works: the names are cross-platform.)
After this change, this program
```
#![feature(exit_status_error)]
fn main(){
let x = std::process::Command::new("false").status().unwrap();
dbg!(x.exit_ok());
eprintln!("x={:?}",x);
}
```
produces this output
```
[src/main.rs:4] x.exit_ok() = Err(
ExitStatusError(
unix_wait_status(
256,
),
),
)
x=ExitStatus(unix_wait_status(256))
```
Closes#74832
Remove unnecessary unsafe block in `process_unix`
Because it's nested under this unsafe fn!
This block isn't detected as unnecessary because of a bug in the compiler: #88260.
Mark unsafe methods NonZero*::unchecked_(add|mul) as const.
Now that https://github.com/rust-lang/rfcs/pull/3016 has landed, these two unstable `std` function can be marked `const`, according to this detail of #84186.
const fn for option copied, take & replace
Tracking issue: [#67441](https://github.com/rust-lang/rust/issues/67441)
Adding const fn for the copied, take and replace method of Option. Also adding necessary unit test.
It's my first contribution so I am pretty sure I don't know what I'm doing but there's a first for everything!
Add `Ipv6Addr::is_benchmarking`
This PR adds the unstable method `Ipv6Addr::is_benchmarking`. This method is added for parity with `Ipv4Addr::is_benchmarking`, and I intend to use it in a future rework of `Ipv6Addr::is_global` (edit: #86634) to more accurately follow the [IANA Special Address Registry](https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml) (like is done in `Ipv4Addr::is_global`).
With `Ipv6Addr::is_benchmarking` and `Ipv4Addr::is_benchmarking` now both existing, `IpAddr::is_benchmarking` is also added.
Fix read_to_end to not grow an exact size buffer
If you know how much data to expect and use `Vec::with_capacity` to pre-allocate a buffer of that capacity, `Read::read_to_end` will still double its capacity. It needs some space to perform a read, even though that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading into a small "probe" buffer instead. If that read returns `0` then it's confirmed that the buffer was the perfect size. If it doesn't, the probe buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in `initial_buffer_size`.
Curiously, there was a unit test that specifically checked that `Read::read_to_end` *does* over-allocate. I removed that test, too.
Make `<[T]>::split_at_unchecked` and `<[T]>::split_at_mut_unchecked` public
The methods were originally added in https://github.com/rust-lang/rust/pull/75936 (30dc32b10e), but for some reason as private. Nevertheless, the methods have documentation and even a [tracking issue](https://github.com/rust-lang/rust/issues/76014).
It's very weird to have a tracking issue for private methods and these methods may be useful outside of the standard library. As such, this PR makes the methods public.
Use bitand when checking for signed integer division overflow
For `self == Self::MIN && rhs == -1`, LLVM does not realize that this is the same check made by `self / rhs`, so the code generated may have some unnecessary duplication. For `(self == Self::MIN) & (rhs == -1)`, LLVM realizes it is the same check.
Optimize unnecessary check in Vec::retain
The function `vec::Vec::retain` only have two stages:
1. Nothing was deleted.
2. Some elements were deleted.
Here is an unnecessary check `if g.deleted_cnt > 0` in the loop, and it's difficult for compiler to optimize it. I split the loop into two stages manully and keep the code clean using const generics.
I write a special but common bench case for this optimization. I call retain on vec but keep all elements.
Before and after this optimization:
```
test vec::bench_retain_whole_100000 ... bench: 84,803 ns/iter (+/- 17,314)
```
```
test vec::bench_retain_whole_100000 ... bench: 42,638 ns/iter (+/- 16,910)
```
The result is expected, there are two `if`s before the optimization and one `if` after.
On MinGW toolchains the various features (such as function sections)
necessary to eliminate dead function references are disabled due to
various bugs. This means that the windows sockets library will most
likely remain linked to any mingw toolchain built program that also
utilizes libstd.
That said, I made an attempt to also enable `function-sections` and
`--gc-sections` during my experiments, but the symbol references
remained, sadly.
For `self == Self::MIN && rhs == -1`, LLVM does not realize that this is the
same check made by `self / rhs`, so the code generated may have some unnecessary
duplication. For `(self == Self::MIN) & (rhs == -1)`, LLVM realizes it is the
same check.
Constify ?-operator for Result and Option
Try to make `?`-operator usable in `const fn` with `Result` and `Option`, see #74935 . Note that the try-operator itself was constified in #87237.
TODO
* [x] Add tests for const T -> T conversions
* [x] cleanup commits
* [x] Remove `#![allow(incomplete_features)]`
* [?] Await decision in #86808 - I'm not sure
* [x] Await support for parsing `~const` in bootstrapping compiler
* [x] Tracking issue(s)? - #88674
Remove ignore-tidy-undocumented-unsafe from core::slice::sort
Write down the missing safety arguments to be able to remove `ignore-tidy-undocumented-unsafe` from `core::slice::sort`.
Helps with #66219
``@rustbot`` label C-cleanup T-libs
Restructure std::rt
These changes should reduce binary size slightly while at the same slightly improving performance of startup, thread spawning and `std:🧵:current()`. I haven't verified if the compiler is able to optimize some of these cases already, but at least for some others the compiler is unable to do these optimizations as they slightly change behavior in cases where program startup would crash anyway by omitting a backtrace and panic location.
I can remove 6f6bb16 if preferred.
The reference automatically coerces to a pointer. Writing an explicit
cast here is slightly misleading because that's most commonly used when
a pointer needs to be converted from one pointer type to another, e.g.
`*const c_void` to `*const sigaction` or vice versa.
The `Step` trait guarantees that `Range<impl Step>` yields items in
sorted order. We can override the `Iterator::is_sorted` method based on
this guarantee, as we already do for `Iterator::min` and `max`.
