std::io: migrate ReadBuf to BorrowBuf/BorrowCursor
This PR replaces `ReadBuf` (used by the `Read::read_buf` family of methods) with `BorrowBuf` and `BorrowCursor`.
The general idea is to split `ReadBuf` because its API is large and confusing. `BorrowBuf` represents a borrowed buffer which is mostly read-only and (other than for construction) deals only with filled vs unfilled segments. a `BorrowCursor` is a mostly write-only view of the unfilled part of a `BorrowBuf` which distinguishes between initialized and uninitialized segments. For `Read::read_buf`, the caller would create a `BorrowBuf`, then pass a `BorrowCursor` to `read_buf`.
In addition to the major API split, I've made the following smaller changes:
* Removed some methods entirely from the API (mostly the functionality can be replicated with two calls rather than a single one)
* Unified naming, e.g., by replacing initialized with init and assume_init with set_init
* Added an easy way to get the number of bytes written to a cursor (`written` method)
As well as simplifying the API (IMO), this approach has the following advantages:
* Since we pass the cursor by value, we remove the 'unsoundness footgun' where a malicious `read_buf` could swap out the `ReadBuf`.
* Since `read_buf` cannot write into the filled part of the buffer, we prevent the filled part shrinking or changing which could cause underflow for the caller or unexpected behaviour.
## Outline
```rust
pub struct BorrowBuf<'a>
impl Debug for BorrowBuf<'_>
impl<'a> From<&'a mut [u8]> for BorrowBuf<'a>
impl<'a> From<&'a mut [MaybeUninit<u8>]> for BorrowBuf<'a>
impl<'a> BorrowBuf<'a> {
pub fn capacity(&self) -> usize
pub fn len(&self) -> usize
pub fn init_len(&self) -> usize
pub fn filled(&self) -> &[u8]
pub fn unfilled<'this>(&'this mut self) -> BorrowCursor<'this, 'a>
pub fn clear(&mut self) -> &mut Self
pub unsafe fn set_init(&mut self, n: usize) -> &mut Self
}
pub struct BorrowCursor<'buf, 'data>
impl<'buf, 'data> BorrowCursor<'buf, 'data> {
pub fn clone<'this>(&'this mut self) -> BorrowCursor<'this, 'data>
pub fn capacity(&self) -> usize
pub fn written(&self) -> usize
pub fn init_ref(&self) -> &[u8]
pub fn init_mut(&mut self) -> &mut [u8]
pub fn uninit_mut(&mut self) -> &mut [MaybeUninit<u8>]
pub unsafe fn as_mut(&mut self) -> &mut [MaybeUninit<u8>]
pub unsafe fn advance(&mut self, n: usize) -> &mut Self
pub fn ensure_init(&mut self) -> &mut Self
pub unsafe fn set_init(&mut self, n: usize) -> &mut Self
pub fn append(&mut self, buf: &[u8])
}
```
## TODO
* ~~Migrate non-unix libs and tests~~
* ~~Naming~~
* ~~`BorrowBuf` or `BorrowedBuf` or `SliceBuf`? (We might want an owned equivalent for the async IO traits)~~
* ~~Should we rename the `readbuf` module? We might keep the name indicate it includes both the buf and cursor variations and someday the owned version too. Or we could change it. It is not publicly exposed, so it is not that important~~.
* ~~`read_buf` method: we read into the cursor now, so the `_buf` suffix is a bit weird.~~
* ~~Documentation~~
* Tests are incomplete (I adjusted existing tests, but did not add new ones).
cc https://github.com/rust-lang/rust/issues/78485, https://github.com/rust-lang/rust/issues/94741
supersedes: https://github.com/rust-lang/rust/pull/95770, https://github.com/rust-lang/rust/pull/93359fixes#93305
Fix HorizonOS regression in FileTimes
The changes in #98246 caused a regression for multiple Newlib-based systems. This is just a fix including HorizonOS to the list of targets which require a workaround.
``@AzureMarker`` ``@ian-h-chamberlain``
r? ``@nagisa``
Support setting file accessed/modified timestamps
Add `struct FileTimes` to contain the relevant file timestamps, since
most platforms require setting all of them at once. (This also allows
for future platform-specific extensions such as setting creation time.)
Add `File::set_file_time` to set the timestamps for a `File`.
Implement the `sys` backends for UNIX, macOS (which needs to fall back
to `futimes` before macOS 10.13 because it lacks `futimens`), Windows,
and WASI.
Add `struct FileTimes` to contain the relevant file timestamps, since
most platforms require setting all of them at once. (This also allows
for future platform-specific extensions such as setting creation time.)
Add `File::set_file_time` to set the timestamps for a `File`.
Implement the `sys` backends for UNIX, macOS (which needs to fall back
to `futimes` before macOS 10.13 because it lacks `futimens`), Windows,
and WASI.
remove_dir_all: use fallback implementation on Miri
Fixes https://github.com/rust-lang/miri/issues/1966
The new implementation requires `openat`, `unlinkat`, and `fdopendir`. These cannot easily be shimmed in Miri since libstd does not expose APIs corresponding to them. So for now it is probably easiest to just use the fallback code in Miri. Nobody should run Miri as root anyway...
unix: reduce the size of DirEntry
On platforms where we call `readdir` instead of `readdir_r`, we store
the name as an allocated `CString` for variable length. There's no point
carrying around a full `dirent64` with its fixed-length `d_name` too.
On platforms where we call `readdir` instead of `readdir_r`, we store
the name as an allocated `CString` for variable length. There's no point
carrying around a full `dirent64` with its fixed-length `d_name` too.
Each recursive call was creating an `OsString` for a `&Path`, only for
it to be turned into a `CString` right away. Instead we can directly
pass `.name_cstr()`, saving two allocations each time.
This only affects the `slow` code path, if there is no `dirent.d_type` or if
the type is `DT_UNKNOWN`.
POSIX specifies that calling `unlink()` or `unlinkat(..., 0)` on a directory can
succeed:
> "The _path_ argument shall not name a directory unless the process has
> appropriate privileges and the implementation supports using _unlink()_ on
> directories."
This however can cause orphaned directories requiring an fsck e.g. on Illumos
UFS, so we have to avoid that in the common case. We now just try to recurse
into it first and unlink() if we can't open it as a directory.
unix: Use metadata for `DirEntry::file_type` fallback
When `DirEntry::file_type` fails to match a known `d_type`, we should
fall back to `DirEntry::metadata` instead of a bare `lstat`, because
this is faster and more reliable on targets with `fstatat`.
When `DirEntry::file_type` fails to match a known `d_type`, we should
fall back to `DirEntry::metadata` instead of a bare `lstat`, because
this is faster and more reliable on targets with `fstatat`.
The dirent returned from readdir() is only guaranteed to be valid for
d_reclen bytes on common platforms. Since we copy the name separately
anyway, we can copy everything except d_name into DirEntry::entry.
Fixes#93384.
Bump libc and fix remove_dir_all on Fuchsia after CVE fix
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Depends on https://github.com/rust-lang/libc/pull/2654 since we
apparently haven't needed to reference DT_UNKNOWN before this.
With the previous `is_dir` impl, we would attempt to unlink
a directory in the None branch, but Fuchsia supports returning
ENOTEMPTY from unlinkat() without the AT_REMOVEDIR flag because
we don't currently differentiate unlinking files and directories
by default.
On the Fuchsia side I've opened https://fxbug.dev/92273 to discuss
whether this is the correct behavior, but it doesn't seem like
addressing the error code is necessary to make our tests happy.
Updates std's libc crate to include DT_UNKNOWN for Fuchsia.
readdir() is preferred over readdir_r() on Linux and many other
platforms because it more gracefully supports long file names. Both
glibc and musl (and presumably all other Linux libc implementations)
guarantee that readdir() is thread-safe as long as a single DIR* is not
accessed concurrently, which is enough to make a readdir()-based
implementation of ReadDir safe. This implementation is already used for
some other OSes including Fuchsia, Redox, and Solaris.
See #40021 for more details. Fixes#86649. Fixes#34668.
Implement most of RFC 2930, providing the ReadBuf abstraction
This replaces the `Initializer` abstraction for permitting reading into uninitialized buffers, closing #42788.
This leaves several APIs described in the RFC out of scope for the initial implementation:
* read_buf_vectored
* `ReadBufs`
Closes#42788, by removing the relevant APIs.
