Inline trivial (noop) flush calls
At work I noticed that `writer.flush()?` didn't get optimized away in cases where the flush is obviously a no-op, which I had expected (well, desired).
I went through and added `#[inline]` to a bunch of cases that were obviously noops, or delegated to ones that were obviously noops. I omitted platforms I don't have access to (some tier3). I didn't do this very scientifically, in cases where it was non-obvious I left `#[inline]` off.
Make sure the implementation of TcpStream::as_raw_fd is fully inlined
Currently the following function:
```rust
use std::os::fd::{AsRawFd, RawFd};
use std::net::TcpStream;
pub fn as_raw_fd(socket: &TcpStream) -> RawFd {
socket.as_raw_fd()
}
```
Is optimized to the following:
```asm
example::as_raw_fd:
push rax
call qword ptr [rip + <std::net::tcp::TcpStream as std::sys_common::AsInner<std::sys_common::net::TcpStream>>::as_inner@GOTPCREL]
mov rdi, rax
call qword ptr [rip + std::sys_common::net::TcpStream::socket@GOTPCREL]
mov rdi, rax
pop rax
jmp qword ptr [rip + _ZN73_$LT$std..sys..unix..net..Socket$u20$as$u20$std..os..fd..raw..AsRawFd$GT$9as_raw_fd17h633bcf7e481df8bbE@GOTPCREL]
```
I think it would make more sense to inline trivial functions used within `TcpStream::AsRawFd`.
Add 64-bit `time_t` support on 32-bit glibc Linux to `set_times`
Add support to `set_times` for 64-bit `time_t` on 32-bit glibc Linux platforms which have a 32-bit `time_t`. Split from #109773.
Tracking issue: #98245
linkat() not available in the system headers of Solaris 10
I've installed rustup on x86_64-unknown-linux-gnu and would like to use the target sparcv9-sun-solaris. For this, I have built a gcc from the source code for cross-compiling to sparcv9-sun-solaris2.10 with system headers of Solaris 10.
With the following hello word example:
main.rs:
```rust
fn main() {
println!("Hello, world!");
}
```
I had a compilation error:
```
$ rustc -v --target sparcv9-sun-solaris -C linker=/opt/cross-solaris/gcc730/bin/sparcv9-sun-solaris2.10-gcc main.rs
error: linking with `/opt/cross-solaris/gcc730/bin/sparcv9-sun-solaris2.10-gcc` failed: exit status: 1
|
= note: "/opt/cross-solaris/gcc730/bin/sparcv9-sun-solaris2.10-gcc" "-m64" "/tmp/rustcgebYgj/symbols.o" "main.main.89363361-cgu.0.rcgu.o" "main.main.89363361-cgu.1.rcgu.o" "main.main.89363361-cgu.2.rcgu.o" "main.main.89363361-cgu.3.rcgu.o" "main.main.89363361-cgu.4.rcgu.o" "main.main.89363361-cgu.5.rcgu.o" "main.csypsau9u2r8348.rcgu.o" "-Wl,-z,ignore" "-L" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib" "-Wl,-Bstatic" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libstd-fa47c8247d587714.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libpanic_unwind-5c87bbe223e6c2a3.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libobject-d484934062ff9fbb.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libmemchr-e8dbd5835abcbf43.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libaddr2line-909ad09329bde2f9.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libgimli-4d74a3be929697ac.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/librustc_demangle-47cbe1d7f7271ae1.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libstd_detect-239fd2d25fb32a00.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libhashbrown-c4a7ce45fb9dec19.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libminiz_oxide-fa6bc3d9bfb4e402.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libadler-419f5a82ddd339a3.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/librustc_std_workspace_alloc-7672b378962c11be.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libunwind-0f9e07f0a032c000.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libcfg_if-ede7757c356dfb28.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/liblibc-808d56fbc668148a.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/liballoc-784767fe059ad3fe.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/librustc_std_workspace_core-aa31d7ef0556bbe1.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libcore-81d07df07db18847.rlib" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libcompiler_builtins-313a510e63006db2.rlib" "-Wl,-Bdynamic" "-lsocket" "-lposix4" "-lpthread" "-lresolv" "-lgcc_s" "-lc" "-lm" "-lrt" "-lpthread" "-lsendfile" "-llgrp" "-L" "/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib" "-o" "main" "-nodefaultlibs"
= note: /opt/cross-solaris/gcc730/lib/gcc/sparcv9-sun-solaris2.10/7.3.0/../../../../sparcv9-sun-solaris2.10/bin/ld: warning: -z ignore ignored.
/home/dlaugt/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/sparcv9-sun-solaris/lib/libstd-fa47c8247d587714.rlib(std-fa47c8247d587714.std.5c42d2c1-cgu.0.rcgu.o): In function `std::sys::unix::fs:🔗:h3683dfbfbb4995cb':
/rustc/897e37553bba8b42751c67658967889d11ecd120/library/std/src/sys/unix/fs.rs:1407: undefined reference to `linkat'
collect2: error: ld returned 1 exit status
= help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
```
linkat() is not available in the system headers of Solaris 10. The hello word example works fine when I build/use rust with this PR change.
Add vectored positioned I/O on Unix
Add methods for vectored I/O with an offset on `File` for `unix` under `#![feature(unix_file_vectored_at)]`.
The new methods are wrappers around `preadv` and `pwritev`.
Tracking issue: #89517
Remove various double spaces in the libraries.
I was just pretty bothered by this when reading the source for a function, and was suggested to check if this happened elsewhere.
As is the current toy program:
fn main() -> std::io::Result<()> {
use std::fs;
let metadata = fs::metadata("foo.txt")?;
assert!(!metadata.is_dir());
Ok(())
}
... observed under strace will issue:
[snip]
statx(0, NULL, AT_STATX_SYNC_AS_STAT, STATX_ALL, NULL) = -1 EFAULT (Bad address)
statx(AT_FDCWD, "foo.txt", AT_STATX_SYNC_AS_STAT, STATX_ALL, {stx_mask=STATX_ALL|STATX_MNT_ID, stx_attributes=0, stx_mode=S_IFREG|0644, stx_size=0, ...}) = 0
While statx is not necessarily always present, checking for it can be
delayed to the first error condition. Said condition may very well never
happen, in which case the check got avoided altogether.
Note this is still suboptimal as there still will be programs issuing
it, but bulk of the problem is removed.
Tested by forbidding the syscall for the binary and observing it
correctly falls back to newfstatat.
While here tidy up the commentary, in particular by denoting some
problems with the current approach.
fs: Fix#50619 (again) and add a regression test
Bug #50619 was fixed by adding an end_of_stream flag in #50630.
Unfortunately, that fix only applied to the readdir_r() path. When I
switched Linux to use readdir() in #92778, I inadvertently reintroduced
the bug on that platform. Other platforms that had always used
readdir() were presumably never fixed.
This patch enables end_of_stream for all platforms, and adds a
Linux-specific regression test that should hopefully prevent the bug
from being reintroduced again.
Use more LFS functions.
On Linux, use mmap64, open64, openat64, and sendfile64 in place of their non-LFS counterparts.
This is relevant to #94173.
With these changes (together with rust-lang/backtrace-rs#501), the simple binaries I produce with rustc seem to have no non-LFS functions, so maybe #94173 is fixed. But I can't be sure if I've missed something and maybe some non-LFS functions could sneak in somehow.
Bug #50619 was fixed by adding an end_of_stream flag in #50630.
Unfortunately, that fix only applied to the readdir_r() path. When I
switched Linux to use readdir() in #92778, I inadvertently reintroduced
the bug on that platform. Other platforms that had always used
readdir() were presumably never fixed.
This patch enables end_of_stream for all platforms, and adds a
Linux-specific regression test that should hopefully prevent the bug
from being reintroduced again.
On Linux, use mmap64, open64, openat64, and sendfile64 in place of their
non-LFS counterparts.
This is relevant to #94173.
With these changes (together with rust-lang/backtrace-rs#501), the
simple binaries I produce with rustc seem to have no non-LFS functions,
so maybe #94173 is fixed. But I can't be sure if I've missed something
and maybe some non-LFS functions could sneak in somehow.
Commit 77005950f0 implemented masking of
FileType to fix an issue[^1] in the semantic of FileType comparison.
This commit introduces masking to Hash to maintain the invariant that
x == y => hash(x) == hash(y).
[^1]: https://github.com/rust-lang/rust/issues/104900
Eliminate 280-byte memset from ReadDir iterator
This guy:
1536ab1b38/library/std/src/sys/unix/fs.rs (L589)
It turns out `libc::dirent64` is quite big—https://docs.rs/libc/0.2.135/libc/struct.dirent64.html. In #103135 this memset accounted for 0.9% of the runtime of iterating a big directory.
Almost none of the big zeroed value is ever used. We memcpy a tiny prefix (19 bytes) into it, and then read just 9 bytes (`d_ino` and `d_type`) back out. We can read exactly those 9 bytes we need directly from the original entry_ptr instead.
## History
This code got added in #93459 and tweaked in #94272 and #94750.
Prior to #93459, there was no memset but a full 280 bytes were being copied from the entry_ptr.
<table><tr><td>copy 280 bytes</td></tr></table>
This was not legal because not all of those bytes might be initialized, or even allocated, depending on the length of the directory entry's name, leading to a segfault. That PR fixed the segfault by creating a new zeroed dirent64 and copying just the guaranteed initialized prefix into it.
<table><tr><td>memset 280 bytes</td><td>copy 19 bytes</td></tr></table>
However this was still buggy because it used `addr_of!((*entry_ptr).d_name)`, which is considered UB by Miri in the case that the full extent of entry_ptr is not in bounds of the same allocation. (Arguably this shouldn't be a requirement, but here we are.)
The UB got fixed by #94272 by replacing `addr_of` with some pointer manipulation based on `offset_from`, but still fundamentally the same operation.
<table><tr><td>memset 280 bytes</td><td>copy 19 bytes</td></tr></table>
Then #94750 noticed that only 9 of those 19 bytes were even being used, so we could pick out only those 9 to put in the ReadDir value.
<table><tr><td>memset 280 bytes</td><td>copy 19 bytes</td><td>copy 9 bytes</td></tr></table>
After my PR we just grab the 9 needed bytes directly from entry_ptr.
<table><tr><td>copy 9 bytes</td></tr></table>
The resulting code is more complex but I believe still worthwhile to land for the following reason. This is an extremely straightforward thing to accomplish in C and clearly libc assumes that; literally just `entry_ptr->d_name`. The extra work in comparison to accomplish it in Rust is not an example of any actual safety being provided by Rust. I believe it's useful to have uncovered that and think about what could be done in the standard library or language to support this obvious operation better.
## References
- https://man7.org/linux/man-pages/man3/readdir.3.html
fs::get_path solarish version.
similar to linux, albeit there is no /proc/self notion on solaris
based system thus flattening the difference for simplification sake.
Rollup of 8 pull requests
Successful merges:
- #101118 (fs::get_mode enable getting the data via fcntl/F_GETFL on major BSD)
- #102072 (Add `ptr::Alignment` type)
- #102799 (rustdoc: remove hover gap in file picker)
- #102820 (Show let-else suggestion on stable.)
- #102829 (rename `ImplItemKind::TyAlias` to `ImplItemKind::Type`)
- #102831 (Don't use unnormalized type in `Ty::fn_sig` call in rustdoc `clean_middle_ty`)
- #102834 (Remove unnecessary `lift`/`lift_to_tcx` calls from rustdoc)
- #102838 (remove cfg(bootstrap) from Miri)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
std::io: migrate ReadBuf to BorrowBuf/BorrowCursor
This PR replaces `ReadBuf` (used by the `Read::read_buf` family of methods) with `BorrowBuf` and `BorrowCursor`.
The general idea is to split `ReadBuf` because its API is large and confusing. `BorrowBuf` represents a borrowed buffer which is mostly read-only and (other than for construction) deals only with filled vs unfilled segments. a `BorrowCursor` is a mostly write-only view of the unfilled part of a `BorrowBuf` which distinguishes between initialized and uninitialized segments. For `Read::read_buf`, the caller would create a `BorrowBuf`, then pass a `BorrowCursor` to `read_buf`.
In addition to the major API split, I've made the following smaller changes:
* Removed some methods entirely from the API (mostly the functionality can be replicated with two calls rather than a single one)
* Unified naming, e.g., by replacing initialized with init and assume_init with set_init
* Added an easy way to get the number of bytes written to a cursor (`written` method)
As well as simplifying the API (IMO), this approach has the following advantages:
* Since we pass the cursor by value, we remove the 'unsoundness footgun' where a malicious `read_buf` could swap out the `ReadBuf`.
* Since `read_buf` cannot write into the filled part of the buffer, we prevent the filled part shrinking or changing which could cause underflow for the caller or unexpected behaviour.
## Outline
```rust
pub struct BorrowBuf<'a>
impl Debug for BorrowBuf<'_>
impl<'a> From<&'a mut [u8]> for BorrowBuf<'a>
impl<'a> From<&'a mut [MaybeUninit<u8>]> for BorrowBuf<'a>
impl<'a> BorrowBuf<'a> {
pub fn capacity(&self) -> usize
pub fn len(&self) -> usize
pub fn init_len(&self) -> usize
pub fn filled(&self) -> &[u8]
pub fn unfilled<'this>(&'this mut self) -> BorrowCursor<'this, 'a>
pub fn clear(&mut self) -> &mut Self
pub unsafe fn set_init(&mut self, n: usize) -> &mut Self
}
pub struct BorrowCursor<'buf, 'data>
impl<'buf, 'data> BorrowCursor<'buf, 'data> {
pub fn clone<'this>(&'this mut self) -> BorrowCursor<'this, 'data>
pub fn capacity(&self) -> usize
pub fn written(&self) -> usize
pub fn init_ref(&self) -> &[u8]
pub fn init_mut(&mut self) -> &mut [u8]
pub fn uninit_mut(&mut self) -> &mut [MaybeUninit<u8>]
pub unsafe fn as_mut(&mut self) -> &mut [MaybeUninit<u8>]
pub unsafe fn advance(&mut self, n: usize) -> &mut Self
pub fn ensure_init(&mut self) -> &mut Self
pub unsafe fn set_init(&mut self, n: usize) -> &mut Self
pub fn append(&mut self, buf: &[u8])
}
```
## TODO
* ~~Migrate non-unix libs and tests~~
* ~~Naming~~
* ~~`BorrowBuf` or `BorrowedBuf` or `SliceBuf`? (We might want an owned equivalent for the async IO traits)~~
* ~~Should we rename the `readbuf` module? We might keep the name indicate it includes both the buf and cursor variations and someday the owned version too. Or we could change it. It is not publicly exposed, so it is not that important~~.
* ~~`read_buf` method: we read into the cursor now, so the `_buf` suffix is a bit weird.~~
* ~~Documentation~~
* Tests are incomplete (I adjusted existing tests, but did not add new ones).
cc https://github.com/rust-lang/rust/issues/78485, https://github.com/rust-lang/rust/issues/94741
supersedes: https://github.com/rust-lang/rust/pull/95770, https://github.com/rust-lang/rust/pull/93359fixes#93305
