Restructure std::rt
These changes should reduce binary size slightly while at the same slightly improving performance of startup, thread spawning and `std:🧵:current()`. I haven't verified if the compiler is able to optimize some of these cases already, but at least for some others the compiler is unable to do these optimizations as they slightly change behavior in cases where program startup would crash anyway by omitting a backtrace and panic location.
I can remove 6f6bb16 if preferred.
The reference automatically coerces to a pointer. Writing an explicit
cast here is slightly misleading because that's most commonly used when
a pointer needs to be converted from one pointer type to another, e.g.
`*const c_void` to `*const sigaction` or vice versa.
Add SOLID targets
This PR introduces new tier 3 targets for [SOLID](https://www.kmckk.co.jp/eng/SOLID/) embedded development platform by Kyoto Microcomputer Co., Ltd.
| Target name | `target_arch` | `target_vendor` | `target_os` |
|--------------------------------|---------------|-----------------|--------------|
| `aarch64-kmc-solid_asp3` | `aarch64` | `kmc` | `solid_asp3` |
| `armv7a-kmc-solid_asp3-eabi` | `arm` | `kmc` | `solid_asp3` |
| `armv7a-kmc-solid_asp3-eabihf` | `arm` | `kmc` | `solid_asp3` |
## Related PRs
- [ ] `libc`: https://github.com/rust-lang/libc/pull/2227
- [ ] `cc`: https://github.com/alexcrichton/cc-rs/pull/609
## Non-blocking Issues
- [ ] The target kernel can support `Thread::unpark` directly, but this property is not utilized because the underlying kernel feature is used to implement `Condvar` and it's unclear whether `std` should guarantee that parking tokens are not clobbered by other synchronization primitives.
- [ ] The rustc book: The page title "\*-kmc-solid-\*" shows up as "-kmc-solid-" in TOC
## Tier 3 Target Policy
As tier 3 targets, the new targets are required to adhere to [the tier 3 target policy](https://doc.rust-lang.org/nightly/rustc/target-tier-policy.html#tier-3-target-policy) requirements. This section quotes each requirement in entirety and describes how they are met.
> - A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)
See [`src/doc/rustc/src/platform-support/kmc-solid.md`](https://github.com/kawadakk/rust/blob/release-add-solid-support/src/doc/rustc/src/platform-support/kmc-solid.md).
> - Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.
> - Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.
The new target names follow this format: `$ARCH-$VENDOR-$OS-$ABI`, which is already adopted by most existing targets. `$ARCH` and `$ABI` follow the convention: `aarch64-*` for AArch64, `armv7a-*-eabi` for Armv7-A with EABI. `$OS` is used to distinguish multiple variations of the platform in a somewhat similar way to the Apple targets, though we are only adding one variation in this PR. `$VENDOR` denotes the platform vendor name similarly to the Apple, Solaris, SGX, and VxWorks targets.
`$OS` corresponds to the value of `target_os` and takes the format `solid-$KERNEL`. The inclusion of a hyphen prevents unique decomposition of target names, though the mapping between target names and target attributes isn't trivial in the first place, e.g., because of the Android targets.
More targets may be added later, as we support other base kernels (there are at least three at the point of writing) and are interested in supporting other processor architectures in the future.
> - Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.
> - The target must not introduce license incompatibilities.
> - Anything added to the Rust repository must be under the standard Rust license (`MIT OR Apache-2.0`).
> - The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the `tidy` tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
> - If the target supports building host tools (such as `rustc` or `cargo`), those host tools must not depend on proprietary (non-FOSS) libraries, other than ordinary runtime libraries supplied by the platform and commonly used by other binaries built for the target. For instance, `rustc` built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
> - Targets should not require proprietary (non-FOSS) components to link a functional binary or library.
> - "onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are *not* limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.
We intend to make the contribution fully available under the standard Rust license with no additional legal restrictions whatsoever. This PR does not introduce any new dependency less permissive than the Rust license policy, and we are willing to ensure this doesn't happen for future contributions regarding the new targets.
The new targets don't support building host tools.
Although the new targets use a platform-provided C compiler toolchain, it can be substituted by [GNU Arm Embedded Toolchain](https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-rm) for testing purposes.
> - Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (`core` for most targets, `alloc` for targets that can support dynamic memory allocation, `std` for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.
Most features are implemented. The following features are not implemented due to the lack of native support:
- `fs::File::{file_attr, truncate, duplicate, set_permissions}`
- `fs::{symlink, link, canonicalize}`
- Process creation
- Command-line arguments
~~Networking is not implemented yet, and we intend to add it as soon as it's ready.~~
Edit (2021-07-07): Networking is now implemented.
Backtrace generation is not really a good fit for embedded targets, so it's intentionally left unimplemented. Unwinding is functional, however.
> - The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running tests (even if they do not pass), the documentation must explain how to run tests for the target, using emulation if possible or dedicated hardware if necessary.
See [`src/doc/rustc/src/platform-support/kmc-solid.md`](https://github.com/kawadakk/rust/blob/release-add-solid-support/src/doc/rustc/src/platform-support/kmc-solid.md). Running tests is not supported.
> - Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
> - This requirement does not prevent part or all of this policy from being cited in an explicit contract or work agreement (e.g. to implement or maintain support for a target). This requirement exists to ensure that a developer or team responsible for reviewing and approving a target does not face any legal threats or obligations that would prevent them from freely exercising their judgment in such approval, even if such judgment involves subjective matters or goes beyond the letter of these requirements.
> - Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via ``@`)` to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
> - Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.
> - Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
> - In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.
We acknowledge these requirements and intend to ensure they are met.
There are no closely related targets at the moment.
SOLID[1] is an embedded development platform provided by Kyoto
Microcomputer Co., Ltd. This commit introduces a basic Tier 3 support
for SOLID.
# New Targets
The following targets are added:
- `aarch64-kmc-solid_asp3`
- `armv7a-kmc-solid_asp3-eabi`
- `armv7a-kmc-solid_asp3-eabihf`
SOLID's target software system can be divided into two parts: an
RTOS kernel, which is responsible for threading and synchronization,
and Core Services, which provides filesystems, networking, and other
things. The RTOS kernel is a μITRON4.0[2][3]-derived kernel based on
the open-source TOPPERS RTOS kernels[4]. For uniprocessor systems
(more precisely, systems where only one processor core is allocated for
SOLID), this will be the TOPPERS/ASP3 kernel. As μITRON is
traditionally only specified at the source-code level, the ABI is
unique to each implementation, which is why `asp3` is included in the
target names.
More targets could be added later, as we support other base kernels
(there are at least three at the point of writing) and are interested
in supporting other processor architectures in the future.
# C Compiler
Although SOLID provides its own supported C/C++ build toolchain, GNU Arm
Embedded Toolchain seems to work for the purpose of building Rust.
# Unresolved Questions
A μITRON4 kernel can support `Thread::unpark` natively, but it's not
used by this commit's implementation because the underlying kernel
feature is also used to implement `Condvar`, and it's unclear whether
`std` should guarantee that parking tokens are not clobbered by other
synchronization primitives.
# Unsupported or Unimplemented Features
Most features are implemented. The following features are not
implemented due to the lack of native support:
- `fs::File::{file_attr, truncate, duplicate, set_permissions}`
- `fs::{symlink, link, canonicalize}`
- Process creation
- Command-line arguments
Backtrace generation is not really a good fit for embedded targets, so
it's intentionally left unimplemented. Unwinding is functional, however.
## Dynamic Linking
Dynamic linking is not supported. The target platform supports dynamic
linking, but enabling this in Rust causes several problems.
- The linker invocation used to build the shared object of `std` is
too long for the platform-provided linker to handle.
- A linker script with specific requirements is required for the
compiled shared object to be actually loadable.
As such, we decided to disable dynamic linking for now. Regardless, the
users can try to create shared objects by manually invoking the linker.
## Executable
Building an executable is not supported as the notion of "executable
files" isn't well-defined for these targets.
[1] https://solid.kmckk.com/SOLID/
[2] http://ertl.jp/ITRON/SPEC/mitron4-e.html
[3] https://en.wikipedia.org/wiki/ITRON_project
[4] https://toppers.jp/
Fix spacing of links in inline code.
Similar to #80733, but the focus is different. This PR eliminates all occurrences of pieced-together inline code blocks like [`Box`]`<`[`Option`]`<T>>` and replaces them with good-looking ones (using HTML-syntax), like <code>[Box]<[Option]\<T>></code>. As far as I can tell, I should’ve found all of these in the standard library (regex search with `` r"`\]`|`\[`" ``) \[except for in `core::convert` where I’ve noticed other things in the docs that I want to fix in a separate PR]. In particular, unlike #80733, I’ve added almost no new instance of inline code that’s broken up into multiple links (or some link and some link-free part). I also added tooltips (the stuff in quotes for the markdown link listings) in places that caught my eye, but that’s by no means systematic, just opportunistic.
[Box]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[`Box`]: https://doc.rust-lang.org/std/boxed/struct.Box.html "Box"
[Option]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
[`Option`]: https://doc.rust-lang.org/std/option/enum.Option.html "Option"
Context: I got annoyed by repeatedly running into new misformatted inline code while reading the standard library docs. I know that once issue #83997 (and/or related ones) are resolved, these changes become somewhat obsolete, but I fail to notice much progress on that end right now.
r? `@jyn514`
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::fmt
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::{rc, sync}
----------
Fix spacing for links inside code blocks, and improve link tooltips in alloc::string
----------
Fix spacing for links inside code blocks in alloc::vec
----------
Fix spacing for links inside code blocks in core::option
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in core::result
----------
Fix spacing for links inside code blocks in core::{iter::{self, iterator}, stream::stream, poll}
----------
Fix spacing for links inside code blocks, and improve a few link tooltips in std::{fs, path}
----------
Fix spacing for links inside code blocks in std::{collections, time}
----------
Fix spacing for links inside code blocks in and make formatting of `&str`-like types consistent in std::ffi::{c_str, os_str}
----------
Fix spacing for links inside code blocks, and improve link tooltips in std::ffi
----------
Fix spacing for links inside code blocks, and improve a few link tooltips
in std::{io::{self, buffered::{bufreader, bufwriter}, cursor, util}, net::{self, addr}}
----------
Fix typo in link to `into` for `OsString` docs
----------
Remove tooltips that will probably become redundant in the future
----------
Apply suggestions from code review
Replacing `…std/primitive.reference.html` paths with just `reference`
Co-authored-by: Joshua Nelson <github@jyn.dev>
----------
Also replace `…std/primitive.reference.html` paths with just `reference` in `core::pin`
Add a better error message for #39364
There is a known bug in the implementation of mpsc channels in rust.
This adds a clearer error message when the bug occurs, so that developers don't lose too much time looking for the origin of the bug.
See https://github.com/rust-lang/rust/issues/39364
Fix WinUWP std compilation errors due to I/O safety
I/O safety for Windows has landed in #87329. However, it does not cover UWP specific parts and prevents all UWP targets from building. See https://github.com/YtFlow/Maple/issues/18. This PR fixes these compile errors when building std for UWP targets.
If you know how much data to expect and use `Vec::with_capacity` to
pre-allocate a buffer of that capacity, `Read::read_to_end` will still
double its capacity. It needs some space to perform a read, even though
that read ends up returning `0`.
It's a bummer to carefully pre-allocate 1GB to read a 1GB file into
memory and end up using 2GB.
This fixes that behavior by special casing a full buffer and reading
into a small "probe" buffer instead. If that read returns `0` then it's
confirmed that the buffer was the perfect size. If it doesn't, the probe
buffer is appended to the normal buffer and the read loop continues.
Fixing this allows several workarounds in the standard library to be
removed:
- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the
previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in
`initial_buffer_size`.
Curiously, there was a unit test that specifically checked that
`Read::read_to_end` *does* over-allocate. I removed that test, too.
Impl `Error` for `FromSecsError` without foreign type
Using it through the crate-local path in `std` means that it shouldn't make an "Implementations on Foreign Types" section in the `std::error::Error` docs.
Fixes a technicality regarding the size of C's `char` type
Specifically, ISO/IEC 9899:2018 — better known as "C18" — (and at least
C11, C99 and C89) do not specify the size of `byte` in bits.
Section 3.6 defines "byte" as "addressable unit of data storage" while
section 6.2.5 ("Types") only defines "char" as "large enough to store
any member of the basic execution set" giving it a lower bound of 7 bit
(since there are 96 characters in the basic execution set).
With section 6.5.3.4 paragraph 4 "When sizeof is applied to an operant
that has type char […] the result is 1" you could read this as the size
of `char` in bits being defined as exactly the same as the number of
bits in a byte but it's also valid to read that as an exception.
In general implementations take `char` as the smallest unit of
addressable memory, which for modern byte-addressed architectures is
overwhelmingly 8 bits to the point of this convention being completely
cemented into just about all of our software.
So is any of this actually relevant at all? I hope not. I sincerely hope
that this never, ever comes up.
But if for some reason a poor rustacean is having to interface with C
code running on a Cray X1 that in 2003 is still doing word-addressed
memory with 64-bit chars and they trust the docs here blindly it will
blow up in her face. And I'll be truly sorry for her to have to deal
with … all of that.
Using it through the crate-local path in `std` means that it shouldn't make an "Implementations on Foreign Types" section in the `std::error::Error` docs.
Add initial support for m68k
This patch series adds initial support for m68k making use of the new M68k
backend introduced with LLVM-13. Additional changes will be needed to be
able to actually use the backend for this target.
Specifically, ISO/IEC 9899:2018 — better known as "C18" — (and at least
C11, C99 and C89) do not specify the size of `byte` in bits.
Section 3.6 defines "byte" as "addressable unit of data storage" while
section 6.2.5 ("Types") only defines "char" as "large enough to store
any member of the basic execution set" giving it a lower bound of 7 bit
(since there are 96 characters in the basic execution set).
With section 6.5.3.4 paragraph 4 "When sizeof is applied to an operant
that has type char […] the result is 1" you could read this as the size
of `char` in bits being defined as exactly the same as the number of
bits in a byte but it's also valid to read that as an exception.
In general implementations take `char` as the smallest unit of
addressable memory, which for modern byte-addressed architectures is
overwhelmingly 8 bits to the point of this convention being completely
cemented into just about all of our software.
So is any of this actually relevant at all? I hope not. I sincerely hope
that this never, ever comes up.
But if for some reason a poor rustacean is having to interface with C
code running on a Cray X1 that in 2003 is still doing word-addressed
memory with 64-bit words and they trust the docs here blindly it will
blow up in her face. And I'll be truly sorry for her to have to deal
with … all of that.
Add intra-doc links and small changes to `std::os` to be more consistent
I believe that a few items in `std::os` should be linked. I've also added a basic example in `std::os::windows`.
fix potential race in AtomicU64 time monotonizer
The AtomicU64-based monotonizer introduced in #83093 is incorrect because several threads could try to update the value concurrently and a thread which doesn't have the newest value among all the updates could win.
That bug probably has little real world impact since it doesn't make observed time worse than hardware clocks. The worst case would probably be a thread which has a clock that is behind by several cycles observing several inconsistent fixups, which should be similar to observing the unfiltered backslide in the first place.
New benchmarks, they don't look as good as the original PR but still an improvement compared to the mutex.
I don't know why the contended mutex case is faster now than in the previous benchmarks.
```
actually_monotonic() == true:
test time::tests::instant_contention_01_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 45 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads ... bench: 45 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads ... bench: 45 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads ... bench: 46 ns/iter (+/- 0)
atomic u64:
test time::tests::instant_contention_01_threads ... bench: 66 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 287 ns/iter (+/- 14)
test time::tests::instant_contention_04_threads ... bench: 296 ns/iter (+/- 43)
test time::tests::instant_contention_08_threads ... bench: 604 ns/iter (+/- 163)
test time::tests::instant_contention_16_threads ... bench: 1,147 ns/iter (+/- 29)
mutex:
test time::tests::instant_contention_01_threads ... bench: 78 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 652 ns/iter (+/- 275)
test time::tests::instant_contention_04_threads ... bench: 900 ns/iter (+/- 32)
test time::tests::instant_contention_08_threads ... bench: 1,927 ns/iter (+/- 62)
test time::tests::instant_contention_16_threads ... bench: 3,748 ns/iter (+/- 146)
```
Avoid codegen for Result::into_ok in lang_start
This extra codegen seems to be the cause for the regressions in max-rss on #86034. While LLVM will certainly optimize the dead code away, avoiding it's generation in the first place seems good, particularly when it is so simple.
#86034 produced this [diff](https://gist.github.com/Mark-Simulacrum/95c7599883093af3b960c35ffadf4dab#file-86034-diff) for a simple `fn main() {}`. With this PR, that diff [becomes limited to just a few extra IR instructions](https://gist.github.com/Mark-Simulacrum/95c7599883093af3b960c35ffadf4dab#file-88988-from-pre-diff) -- no extra functions.
Note that these are pre-optimization; LLVM surely will eliminate this during optimization. However, that optimization can end up generating more work and bump memory usage, and this eliminates that.
Allow `panic!("{}", computed_str)` in const fn.
Special-case `panic!("{}", arg)` and translate it to `panic_display(&arg)`. `panic_display` will behave like `panic_any` in cosnt eval and behave like `panic!(format_args!("{}", arg))` in runtime.
This should bring Rust 2015 and 2021 to feature parity in terms of `const_panic`; and hopefully would unblock the stabilisation of #51999.
`@rustbot` modify labels: +T-compiler +T-libs +A-const-eval +A-const-fn
r? `@oli-obk`
Add chown functions to std::os::unix::fs to change the owner and group of files
This is a straightforward wrapper that uses the existing helpers for C
string handling and errno handling.
Having this available is convenient for UNIX utility programs written in
Rust, and avoids having to call unsafe functions like `libc::chown`
directly and handle errors manually, in a program that may otherwise be
entirely safe code.
In addition, these functions provide a more Rustic interface by
accepting appropriate traits and using `None` rather than `-1`.
Add TcpListener::into_incoming and IntoIncoming
The `incoming` method is really useful, however for some use cases the borrow
this introduces is needlessly restricting. Thus, an owned variant is added.
r? ``@joshtriplett``
The RefCell is now borrowed exactly once. In addition a code sequence
that contains an unwrap that is guaranteed to never panic at runtime is
replaced with get_or_insert_with, which makes the intended behavior
clearer and will not emit code to panic even without optimizations.
Previously the thread name would first be heap allocated and then
re-allocated to add a nul terminator. Now it will be heap allocated only
once with nul terminator added form the start.
This is a straightforward wrapper that uses the existing helpers for C
string handling and errno handling.
Having this available is convenient for UNIX utility programs written in
Rust, and avoids having to call unsafe functions like `libc::chown`
directly and handle errors manually, in a program that may otherwise be
entirely safe code.
In addition, these functions provide a more Rustic interface by
accepting appropriate traits and using `None` rather than `-1`.
This works by doing two things:
- Adding links that are specific to the crate. Since not all primitive
items are defined in `core` (due to lang_items), these need to use
relative links and not intra-doc links.
- Duplicating `primitive_docs` in both core and std. This allows not needing CARGO_PKG_NAME to build the standard library. It also adds a tidy check to make sure they stay the same.
More symbolic doc aliases
A bunch of small changes, mostly adding `#[doc(alias = "…")]` entries for symbolic `"…"`.
Also a small change in documentation of `const` keywords.
Document when to use Windows' `symlink_dir` vs. `symlink_file`
It was previously unclear why there are two functions and when they should be used.
Fixes: #88635
Add links in docs for some primitive types
This pull request adds additional links in existing documentation of some of the primitive types.
Where items are linked only once, I have used the `[link](destination)` format. For items in `std`, I have linked directly to the HTML, since although the primitives are in `core`, they are not displayed on `core` documentation. I was unsure of what length I should keep lines of documentation to, so I tried to keep them within reason.
Additionally, I have avoided excessively linking to keywords like `self` when they are not relevant to the documentation. I can add these links if it would be an improvement.
I hope this can improve Rust. Please let me know if there's anything I did wrong!
While issues have been seen on arm64 platforms the Arm architecture requires
that the counter monotonically increases and that it must provide a uniform
view of system time (e.g. it must not be possible for a core to receive a
message from another core with a time stamp and observe time going backwards
(ARM DDI 0487G.b D11.1.2). While there have been a few 64bit SoCs that have
bugs (#49281, #56940) which cause time to not monotonically increase, these have
been fixed in the Linux kernel and we shouldn't penalize all Arm SoCs for those
who refuse to update their kernels:
SUN50I_ERRATUM_UNKNOWN1 - Allwinner A64 / Pine A64 - fixed in 5.1
FSL_ERRATUM_A008585 - Freescale LS2080A/LS1043A - fixed in 4.10
HISILICON_ERRATUM_161010101 - Hisilicon 1610 - fixed in 4.11
ARM64_ERRATUM_858921 - Cortex A73 - fixed in 4.12
255a3f3e18 std: Force `Instant::now()` to be monotonic added a mutex to work around
this problem and a small test program using glommio shows the majority of time spent
acquiring and releasing this Mutex. 3914a7b0da tries to improve this, but actually
makes it worse on big systems as for 128b atomics a ldxp/stxp pair (and
successful loop) is required which is expensive as a lock and because of how
the load/store-exclusives scale on large Arm systems is both unfair to threads
and tends to go backwards in performance.
aarch64 prior to v8.4 (FEAT_LSE2) doesn't have an instruction that guarantees
untorn 128b reads except for completing a 128b load/store exclusive pair
(ldxp/stxp) or compare-and-swap (casp) successfully. The requirement to
complete a 128b read+write atomic is actually more expensive and more unfair
than the previous implementation of monotonize() which used a Mutex on aarch64,
especially at large core counts. For aarch64 switch to the 64b atomic
implementation which is about 13x faster for a benchmark that involves many
calls to Instant::now().
There is a known bug in the implementation of mpsc channels in rust.
This adds a clearer error message when the bug occurs, so that developers don't lose too much time looking for the origin of the bug.
See https://github.com/rust-lang/rust/issues/39364
Stabilize std::os::unix::fs::chroot
I've verified that this works as documented, and I've tested it in (a nightly
build of) production software as a replacement for an unsafe call to
`libc::chroot`. It's been available in nightly for a few releases. I think it's
ready to stabilize.
---
Tracking issue: https://github.com/rust-lang/rust/issues/84715
Allow writing of incomplete UTF-8 sequences to the Windows console via stdout/stderr
# Problem
Writes of just an incomplete UTF-8 byte sequence (e.g. `b"\xC3"` or `b"\xF0\x9F"`) to stdout/stderr with a Windows console attached error with `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` even though further writes could complete the codepoint. This is currently a rare occurence since the [linewritershim](2c56ea38b0/library/std/src/io/buffered/linewritershim.rs) implementation flushes complete lines immediately and buffers up to 1024 bytes for incomplete lines. It can still happen as described in #83258.
The problem will become more pronounced once the developer can switch stdout/stderr from line-buffered to block-buffered or immediate when the changes in the "Switchable buffering for Stdout" pull request (#78515) get merged.
# Patch description
If there is at least one valid UTF-8 codepoint all valid UTF-8 is passed through to the extracted `write_valid_utf8_to_console()` fn. The new code only comes into play if `write()` is being passed a short byte slice comprising an incomplete UTF-8 codepoint. In this case up to three bytes are buffered in the `IncompleteUtf8` struct associated with `Stdout` / `Stderr`. The bytes are accepted one at a time. As soon as an error can be detected `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` is returned. Once a complete UTF-8 codepoint is received it is passed to the `write_valid_utf8_to_console()` and the buffer length is set to zero.
Calling `flush()` will neither error nor write anything if an incomplete codepoint is present in the buffer.
# Tests
Currently there are no Windows-specific tests for console writing code at all. Writing (regression) tests for this problem is a bit challenging since unit tests and UI tests don't run in a console and suddenly popping up another console window might be surprising to developers running the testsuite and it might not work at all in CI builds. To just test the new functionality in unit tests the code would need to be refactored. Some guidance on how to proceed would be appreciated.
# Public API changes
* `std::str::verifications::utf8_char_width()` would be exposed as `std::str::utf8_char_width()` behind the "str_internals" feature gate.
# Related issues
* Fixes#83258.
* PR #78515 will exacerbate the problem.
# Open questions
* Add tests?
* Squash into one commit with better commit message?
Stabilize `UnsafeCell::raw_get()`
This PR stabilizes the associated function `UnsafeCell::raw_get()`. The FCP has [already completed](https://github.com/rust-lang/rust/issues/66358#issuecomment-899095068). While there was some discussion about the naming after the close of the FCP, it looks like people have agreed on this name. Still, it would probably be best if a `libs-api` member had a look at this and stated whether more discussion is needed.
While I was at it, I added some tests for `UnsafeCell`, because there were barely any.
Closes#66358.
Use the return value of readdir_r() instead of errno
POSIX says:
> If successful, the readdir_r() function shall return zero; otherwise,
> an error number shall be returned to indicate the error.
But we were previously using errno instead of the return value. This
led to issue #86649.
POSIX says:
> If successful, the readdir_r() function shall return zero; otherwise,
> an error number shall be returned to indicate the error.
But we were previously using errno instead of the return value. This
led to issue #86649.
Add `TcpStream::set_linger` and `TcpStream::linger`
Adds methods for getting/setting the `SO_LINGER` option on TCP sockets. Behavior is consistent across Unix and Windows.
r? `@joshtriplett` (I noticed you've been reviewing net related PRs)
Document `std::env::current_exe` possible rename behaviour
It might not be obvious that the "path of the current running executable" may (or may not) imply "at the time it was loaded".
This came up recently in chat so I thought it might be worth documenting.
Add Saturating type (based on Wrapping type)
Tracking #87920
### Unresolved Questions
<!--
Include any open questions that need to be answered before the feature can be
stabilised.
-->
- [x] ~`impl Div for Saturating<T>` falls back on inner integer division - which seems alright?~
- [x] add `saturating_div`? (to respect division by `-1`)
- [x] There is no `::saturating_shl` and `::saturating_shr`. (How to) implement `Shl`, `ShlAssign`, `Shr` and `ShrAssign`?
- [naively](3f7d2ce28f)
- [x] ~`saturating_neg` is only implemented on [signed integer types](https://doc.rust-lang.org/std/?search=saturating_n)~
- [x] Is the implementation copied over from the `Wrapping`-type correct for `Saturating`?
- [x] `Saturating::rotate_left`
- [x] `Saturating::rotate_right`
- [x] `Not`
- [x] `BitXorOr` and `BitXorOrAssign`
- [x] `BitOr` and `BitOrAssign`
- [x] `BitAnd` and `BitAndAssign`
- [x] `Saturating::swap_bytes`
- [x] `Saturating::reverse_bits`
The `incoming` method is really useful, however for some use cases the borrow
this introduces is needlessly restricting. Thus, an owned variant is added.
Add `c_size_t` and `c_ssize_t` to `std::os::raw`.
Apparently these aren't guaranteed to be the same, and are merely "always the same in practice" (see https://rust-lang.zulipchat.com/#narrow/stream/136281-t-lang.2Fwg-unsafe-code-guidelines/topic/.60usize.60.20vs.20.60size_t.60).
This is a big footgun, but I suspect it can be alleviated if we expose this and start migrating people to it in advance of any platforms that ever have this as different.
I'll file a tracking issue after this gets some traction.
Stabilise BufWriter::into_parts
The FCP for this has already completed, in #80690.
This was just blocked on #85901 (which changed the name), which is now merged. The original stabilisation MR was #84770 but that has a lot of noise in it, and I also accidentally deleted the branch while trying to tidy up. So here is a new MR. Sorry for the noise.
Closes#80690
Errorkind reorder
I was doing a bit more work in this area and the untidiness of these two orderings bothered me.
The commit messages have the detailed rationale. For your convenience, I c&p them here:
```
io::ErrorKind: rationalise ordering in main enum
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
```
```
io::Error: alphabeticise the match in as_str()
There was no rationale for the previous ordering.
```
r? kennytm since that's who rust-highfive picked before, in #88294 which I accidentally closed.
These structs have misleading names. An ExitStatus[Error] is actually
a Unix wait status; an ExitCode is actually an exit status.
The Display impls are fixed, but the Debug impls are still misleading,
as reported in #74832.
Fix this by pretending that these internal structs are called
`unix_exit_status` and `unix_wait_status` as applicable. (We can't
actually rename the structs because of the way that the cross-platform
machinery works: the names are cross-platform.)
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
It is useful to keep some coherent structure to this ordering. In
particular, Other and Uncategorized should be next to each other, at
the end.
Also it seems to make sense to treat UnexpectedEof and OutOfMemory
specially, since they are not like the other errors (despite
OutOfMemory also being generatable by some OS errors).
So:
* Move Other to the end, just before Uncategorized
* Move Unsupported to between Interrupted and UnexpectedEof
* Add some comments documenting where to add things
Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
Implement `AsFd` etc. for `UnixListener`.
Implement `AsFd`, `From<OwnedFd>`, and `Into<OwnedFd>` for
`UnixListener`. This is a follow-up to #87329.
r? `@joshtriplett`
add file_prefix method to std::path
This is an initial implementation of `std::path::Path::file_prefix`. It is effectively a "left" variant of the existing [`file_stem`](https://doc.rust-lang.org/std/path/struct.Path.html#method.file_stem) method. An illustration of the difference is
```rust
use std::path::Path;
let path = Path::new("foo.tar.gz");
assert_eq!(path.file_stem(), Some("foo.tar"));
assert_eq!(path.file_prefix(), Some("foo"));
```
In my own development, I generally find I almost always want the prefix, rather than the stem, so I thought it might be best to suggest it's addition to libstd.
Of course, as this is my first contribution, I expect there is probably more work that needs to be done. Additionally, if the libstd team feel this isn't appropriate then so be it.
There has been some [discussion about this on Zulip](https://rust-lang.zulipchat.com/#narrow/stream/219381-t-libs/topic/file_lstem/near/238076313) and a user there suggested I open a PR to see whether someone in the libstd team thinks it is worth pursuing.
where available use AtomicU{64,128} instead of mutex for Instant backsliding protection
This decreases the overhead of backsliding protection on x86 systems with unreliable TSC, e.g. windows. And on aarch64 systems where 128bit atomics are available.
The following benchmarks were taken on x86_64 linux though by overriding `actually_monotonic()`, the numbers may look different on other platforms
```
# actually_monotonic() == true
test time::tests::instant_contention_01_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_04_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_08_threads ... bench: 44 ns/iter (+/- 0)
test time::tests::instant_contention_16_threads ... bench: 44 ns/iter (+/- 0)
# 1x AtomicU64
test time::tests::instant_contention_01_threads ... bench: 65 ns/iter (+/- 0)
test time::tests::instant_contention_02_threads ... bench: 157 ns/iter (+/- 20)
test time::tests::instant_contention_04_threads ... bench: 281 ns/iter (+/- 53)
test time::tests::instant_contention_08_threads ... bench: 555 ns/iter (+/- 77)
test time::tests::instant_contention_16_threads ... bench: 883 ns/iter (+/- 107)
# mutex
test time::tests::instant_contention_01_threads ... bench: 60 ns/iter (+/- 2)
test time::tests::instant_contention_02_threads ... bench: 770 ns/iter (+/- 231)
test time::tests::instant_contention_04_threads ... bench: 1,347 ns/iter (+/- 45)
test time::tests::instant_contention_08_threads ... bench: 2,693 ns/iter (+/- 114)
test time::tests::instant_contention_16_threads ... bench: 5,244 ns/iter (+/- 487)
```
Since I don't have an arm machine with 128bit atomics I wasn't able to benchmark the AtomicU128 implementation.
I/O safety.
Introduce `OwnedFd` and `BorrowedFd`, and the `AsFd` trait, and
implementations of `AsFd`, `From<OwnedFd>` and `From<T> for OwnedFd`
for relevant types, along with Windows counterparts for handles and
sockets.
Tracking issue: <https://github.com/rust-lang/rust/issues/87074>
RFC: <https://github.com/rust-lang/rfcs/blob/master/text/3128-io-safety.md>
Highlights:
- The doc comments at the top of library/std/src/os/unix/io/mod.rs and library/std/src/os/windows/io/mod.rs
- The new types and traits in library/std/src/os/unix/io/fd.rs and library/std/src/os/windows/io/handle.rs
- The removal of the `RawHandle` struct the Windows impl, which had the same name as the `RawHandle` type alias, and its functionality is now folded into `Handle`.
Managing five levels of wrapping (File wraps sys::fs::File wraps sys::fs::FileDesc wraps OwnedFd wraps RawFd, etc.) made for a fair amount of churn and verbose as/into/from sequences in some places. I've managed to simplify some of them, but I'm open to ideas here.
r? `@joshtriplett`
Add fast path for Path::cmp that skips over long shared prefixes
```
# before
test path::tests::bench_path_cmp_fast_path_buf_sort ... bench: 60,811 ns/iter (+/- 865)
test path::tests::bench_path_cmp_fast_path_long ... bench: 6,459 ns/iter (+/- 275)
test path::tests::bench_path_cmp_fast_path_short ... bench: 1,777 ns/iter (+/- 34)
# after
test path::tests::bench_path_cmp_fast_path_buf_sort ... bench: 38,140 ns/iter (+/- 211)
test path::tests::bench_path_cmp_fast_path_long ... bench: 1,471 ns/iter (+/- 24)
test path::tests::bench_path_cmp_fast_path_short ... bench: 1,106 ns/iter (+/- 9)
```
The name (and updated documentation) make the FFI-only usage clearer, and wrapping Option<OwnedHandle> avoids the need to write a separate Drop or Debug impl.
Co-authored-by: Josh Triplett <josh@joshtriplett.org>
Add TcpStream type to TcpListener::incoming docs
## Context
While going through the "The Rust Programming Language" book (Klabnik & Nichols), the TCP server example directs us to use TcpListener::incoming. I was curious how I could pass this value to a function (before reading ahead in the book), so I looked up the docs to determine the signature.
When I opened the docs, I found https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming, which didn't mention TcpStream anywhere in the example.
Eventually, I clicked on https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.accept in the docs (after clicking a few other locations first), and was able to surmise that the value contained TcpStream.
## Opportunity
While this type is mentioned several times in this doc, I feel that someone should be able to fully use the results of the TcpListner::incoming iterator based solely on the docs of just this method.
## Implementation
I took the code from the top-level TcpListener https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming and blended it with the existing docs for TcpListener::incoming https://doc.rust-lang.org/std/net/struct.TcpListener.html#method.incoming.
It does make the example a little longer, and it also introduces a little duplication. It also gives the reader the type signatures they need to move on to the next step.
## Additional considerations
I noticed that in this doc, `handle_connection` and `handle_client` are both used to accept a TcpStream in the docs on this page. I want to standardize on one function name convention, so readers don't accidentally think two different concepts are being referenced. I didn't want to cram do too much in one PR, I can update this PR to make that change, or I could send another PR (if you would like).
First attempted contribution to Rust (and I'm also still very new, hence reading through the rust book for the first time)! Would you please let me know what you think?
Update the backtrace crate in libstd
This commit updates the backtrace crate in libstd now that dependencies
have been updated to use `memchr` from the standard library as well.
This is mostly just making sure deps are up-to-date and have all the
latest-and-greatest fixes and such.
Closesrust-lang/backtrace-rs#432
This commit updates the backtrace crate in libstd now that dependencies
have been updated to use `memchr` from the standard library as well.
This is mostly just making sure deps are up-to-date and have all the
latest-and-greatest fixes and such.
Closesrust-lang/backtrace-rs#432
Unbox mutexes, condvars and rwlocks on hermit
[RustyHermit](https://github.com/hermitcore/rusty-hermit) provides now movable synchronization primitives and we are able to unbox mutexes and condvars.
Fix environment variable getter docs
`@RalfJung` pointed out a number of errors and suboptimal choices I made in my documentation for #86183. This PR should (hopefully) fix the problems they've identified.
Change WASI's `RawFd` from `u32` to `c_int` (`i32`).
WASI previously used `u32` as its `RawFd` type, since its "file descriptors"
are unsigned table indices, and there's no fundamental reason why WASI can't
have more than 2^31 handles.
However, this creates myriad little incompability problems with code
that also supports Unix platforms, where `RawFd` is `c_int`. While WASI
isn't a Unix, it often shares code with Unix, and this difference made
such shared code inconvenient. #87329 is the most recent example of such
code.
So, switch WASI to use `c_int`, which is `i32`. This will mean that code
intending to support WASI should ideally avoid assuming that negative file
descriptors are invalid, even though POSIX itself says that file descriptors
are never negative.
This is a breaking change, but `RawFd` is considerd an experimental
feature in [the documentation].
[the documentation]: https://doc.rust-lang.org/stable/std/os/wasi/io/type.RawFd.html
r? `@alexcrichton`
The libs-api team agrees to allow const_trait_impl to appear in the
standard library as long as stable code cannot be broken (they are
properly gated) this means if the compiler teams thinks it's okay, then
it's okay.
My priority on constifying would be:
1. Non-generic impls (e.g. Default) or generic impls with no
bounds
2. Generic functions with bounds (that use const impls)
3. Generic impls with bounds
4. Impls for traits with associated types
For people opening constification PRs: please cc me and/or oli-obk.
WASI previously used `u32` as its `RawFd` type, since its "file descriptors"
are unsigned table indices, and there's no fundamental reason why WASI can't
have more than 2^31 handles.
However, this creates myriad little incompability problems with code
that also supports Unix platforms, where `RawFd` is `c_int`. While WASI
isn't a Unix, it often shares code with Unix, and this difference made
such shared code inconvenient. #87329 is the most recent example of such
code.
So, switch WASI to use `c_int`, which is `i32`. This will mean that code
intending to support WASI should ideally avoid assuming that negative file
descriptors are invalid, even though POSIX itself says that file descriptors
are never negative.
This is a breaking change, but `RawFd` is considerd an experimental
feature in [the documentation].
[the documentation]: https://doc.rust-lang.org/stable/std/os/wasi/io/type.RawFd.html
