This pushes the implementation detail of proxying `read_to_end` through to
`read_to_end_uninitialized` all the way down to the `FileDesc` and `Handle`
implementations on Unix/Windows. This way intermediate layers will also be able
to take advantage of this optimized implementation.
This commit also adds the optimized implementation for `ChildStdout` and
`ChildStderr`.
Squashed 10 commits:
1) The main cause of the problem is that libstd/os/mod.rs treats emscripten targets as an alias of linux targets, whereas liblibc treats emscripten targets as musl-compliant, so it gets a slightly different struct stat64 defined.
This commit adds conditional compilation checks to use the correct timestamp format on fs metadata functions in the case of compiling to emscripten targets.
2) Update previous commit to comply with rust formatting standards.
Removed tab characters, remove trailing whitespaces.
3) Move emscripten changes into their own file under libstd/os/emscripten
Put libstd/os/linux/fs back to the way it was.
4) Cannot use stat.st_ctim on emscripten to get created time.
5) Remove compile-time conditionals for target_env = musl, it looks like musl builds compile fine already.
6) Undone some formatting changes that are no longer needed,
Removed some more target_env="musl" compilation checks that I missed from my previous commit.
7) upgrade to liblibc e19309c, it fixes the differences in the musl stat and stat64 definitions.
8) Undo the compile-time checks to check for emscripten (or musl targets) in the FileAttr struct.
No longer needed after updating liblibc to e19309c.
9) Change the MetadataExt implementation of emscripten fs.rs module to match the changes in new liblibc.
10) remove a stray return statement, should have been removed in the previous commit.
Android should use 64-bit LFS symbols for `lseek` and `ftruncate`, lest
those offset parameters suffer a lossy cast down to a 32-bit `off_t`.
Unlike GNU/Linux, Android's `stat`, `dirent`, and related functions are
always 64-bit LFS compatible, and `open` already implies `O_LARGEFILE`,
so all those don't need to follow Linux. It might be nice to unify them
anyway, but those other LFS symbols aren't present in API 18 bionic.
r? @alexcrichton
Some struct members have a slighty different name on NetBSD. This has been
fixed in the libc crate, but not in libstd.
This also removes `st_spare` from MetadataExt, since it is private field
reserved for future use.
Android should use 64-bit LFS symbols for `lseek` and `ftruncate`, lest
those offset parameters suffer a lossy cast down to a 32-bit `off_t`.
Unlike GNU/Linux, Android's `stat`, `dirent`, and related functions are
always 64-bit LFS compatible, and `open` already implies `O_LARGEFILE`,
so all those don't need to follow Linux. It might be nice to unify them
anyway, but those other LFS symbols aren't present in API 18 bionic.
r? @alexcrichton
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md
Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.
To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.
The commit here incorporates two backwards-compatible components:
* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
accessors of all fields. The fields now returned widened types which are the
same across platforms (consistency across platforms is not required, however,
it's just convenient).
and two also backwards-incompatible components:
* Change the definition of all `std::os::*::raw` type aliases to
correspond to the newly widened types that are being returned on each
platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
definitions rather than the standard ones.
The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.
[audit]: https://github.com/rust-lang/rfcs/pull/1415#issuecomment-180645582
---
Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!
Closes#28978Closes#30050Closes#31549
Here's another go at adding emscripten support. This needs to wait again on new [libc definitions](https://github.com/rust-lang-nursery/libc/pull/122) landing. To get the libc definitions right I had to add support for i686-unknown-linux-musl, which are very similar to emscripten's, which are derived from arm/musl.
This branch additionally removes the makefile dependency on the `EMSCRIPTEN` environment variable by not building the unused compiler-rt.
Again, this is not sufficient for actually compiling to asmjs since it needs additional LLVM patches.
r? @alexcrichton
Backtraces, and the compilation of libbacktrace for asmjs, are disabled.
This port doesn't use jemalloc so, like pnacl, it disables jemalloc *for all targets*
in the configure file.
It disables stack protection.
On Linux we have to do this for binary compatibility with 2.6.18, but for other
OSes (e.g. OSX/BSDs/etc) they all support this flag so we don't need to pass it.
These accessors are used to get at the last modification, last access, and
creation time of the underlying file. Currently not all platforms provide the
creation time, so that currently returns `Option`.
- use `symlink_file` and `symlink_dir` instead of the old `soft_link`
- create a junction instead of a directory symlink for testing recursive_rmdir (as it causes the
same troubles, but can be created by users without `SeCreateSymbolicLinkPrivilege`)
- `remove_dir_all` was unable to remove directory symlinks and junctions
- only run tests that create symlinks if we have the right permissions.
- rename `Path2` to `Path`
- remove the global `#[allow(deprecated)]` and outdated comments
- After factoring out `create_junction()` from the test `directory_junctions_are_directories` and
removing needlessly complex code, what I was left with was:
```
#[test]
#[cfg(windows)]
fn directory_junctions_are_directories() {
use sys::fs::create_junction;
let tmpdir = tmpdir();
let foo = tmpdir.join("foo");
let bar = tmpdir.join("bar");
fs::create_dir(&foo).unwrap();
check!(create_junction(&foo, &bar));
assert!(bar.metadata().unwrap().is_dir());
}
```
It test whether a junction is a directory instead of a reparse point. But it actually test the
target of the junction (which is a directory if it exists) instead of the junction itself, which
should always be a symlink. So this test is invalid, and I expect it only exists because the
author was suprised by it. So I removed it.
Some things that do not yet work right:
- relative symlinks do not accept forward slashes
- the conversion of paths for `create_junction` is hacky
- `remove_dir_all` now messes with the internal data of `FileAttr` to be able to remove symlinks.
We should add some method like `is_symlink_dir()` to it, so code outside the standard library
can see the difference between file and directory symlinks too.
I have it set as stable right now under the rationale that it's extending an existing, stable API to another type in the "obvious" way.
r? @alexcrichton
cc @reem
This pull request adds support for [Illumos](http://illumos.org/)-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes#21000, #25845, and #25846.
Required changes in libc are already merged: https://github.com/rust-lang-nursery/libc/pull/138
Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from `make check`.
There are some changes I'm not quite sure about, e.g. macro usage in `src/libstd/num/f64.rs` and `DirEntry` structure in `src/libstd/sys/unix/fs.rs`, so any comments on how to rewrite it better would be greatly appreciated.
Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409
Thanks!
r? @brson
Tracking issue: #30014
This implements the RFC and makes a few other changes.
I have added a few extra tests, and made the Windows and
Unix code as similar as possible.
Part of the RFC mentions the unstable OpenOptionsExt trait
on Windows (see #27720). I have added a few extra methods
to future-proof it for CreateFile2.
Tracking issue: #30014
This implements the RFC and makes a few other changes.
I have added a few extra tests, and made the Windows and
Unix code as similar as possible.
Part of the RFC mentions the unstable OpenOptionsExt trait
on Windows (see #27720). I have added a few extra methods
to future-proof it for CreateFile2.
This commit is the standard API stabilization commit for the 1.6 release cycle.
The list of issues and APIs below have all been through their cycle-long FCP and
the libs team decisions are listed below
Stabilized APIs
* `Read::read_exact`
* `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`)
* libcore -- this was a bit of a nuanced stabilization, the crate itself is now
marked as `#[stable]` and the methods appearing via traits for primitives like
`char` and `str` are now also marked as stable. Note that the extension traits
themeselves are marked as unstable as they're imported via the prelude. The
`try!` macro was also moved from the standard library into libcore to have the
same interface. Otherwise the functions all have copied stability from the
standard library now.
* The `#![no_std]` attribute
* `fs::DirBuilder`
* `fs::DirBuilder::new`
* `fs::DirBuilder::recursive`
* `fs::DirBuilder::create`
* `os::unix::fs::DirBuilderExt`
* `os::unix::fs::DirBuilderExt::mode`
* `vec::Drain`
* `vec::Vec::drain`
* `string::Drain`
* `string::String::drain`
* `vec_deque::Drain`
* `vec_deque::VecDeque::drain`
* `collections::hash_map::Drain`
* `collections::hash_map::HashMap::drain`
* `collections::hash_set::Drain`
* `collections::hash_set::HashSet::drain`
* `collections::binary_heap::Drain`
* `collections::binary_heap::BinaryHeap::drain`
* `Vec::extend_from_slice` (renamed from `push_all`)
* `Mutex::get_mut`
* `Mutex::into_inner`
* `RwLock::get_mut`
* `RwLock::into_inner`
* `Iterator::min_by_key` (renamed from `min_by`)
* `Iterator::max_by_key` (renamed from `max_by`)
Deprecated APIs
* `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`)
* `OsString::from_bytes`
* `OsStr::to_cstring`
* `OsStr::to_bytes`
* `fs::walk_dir` and `fs::WalkDir`
* `path::Components::peek`
* `slice::bytes::MutableByteVector`
* `slice::bytes::copy_memory`
* `Vec::push_all` (renamed to `extend_from_slice`)
* `Duration::span`
* `IpAddr`
* `SocketAddr::ip`
* `Read::tee`
* `io::Tee`
* `Write::broadcast`
* `io::Broadcast`
* `Iterator::min_by` (renamed to `min_by_key`)
* `Iterator::max_by` (renamed to `max_by_key`)
* `net::lookup_addr`
New APIs (still unstable)
* `<[T]>::sort_by_key` (added to mirror `min_by_key`)
Closes#27585Closes#27704Closes#27707Closes#27710Closes#27711Closes#27727Closes#27740Closes#27744Closes#27799Closes#27801
cc #27801 (doesn't close as `Chars` is still unstable)
Closes#28968
* Delete `sys::unix::{c, sync}` as these are now all folded into libc itself
* Update all references to use `libc` as a result.
* Update all references to the new flat namespace.
* Moves all windows bindings into sys::c
This commit stabilizes and deprecates library APIs whose FCP has closed in the
last cycle, specifically:
Stabilized APIs:
* `fs::canonicalize`
* `Path::{metadata, symlink_metadata, canonicalize, read_link, read_dir, exists,
is_file, is_dir}` - all moved to inherent methods from the `PathExt` trait.
* `Formatter::fill`
* `Formatter::width`
* `Formatter::precision`
* `Formatter::sign_plus`
* `Formatter::sign_minus`
* `Formatter::alternate`
* `Formatter::sign_aware_zero_pad`
* `string::ParseError`
* `Utf8Error::valid_up_to`
* `Iterator::{cmp, partial_cmp, eq, ne, lt, le, gt, ge}`
* `<[T]>::split_{first,last}{,_mut}`
* `Condvar::wait_timeout` - note that `wait_timeout_ms` is not yet deprecated
but will be once 1.5 is released.
* `str::{R,}MatchIndices`
* `str::{r,}match_indices`
* `char::from_u32_unchecked`
* `VecDeque::insert`
* `VecDeque::shrink_to_fit`
* `VecDeque::as_slices`
* `VecDeque::as_mut_slices`
* `VecDeque::swap_remove_front` - (renamed from `swap_front_remove`)
* `VecDeque::swap_remove_back` - (renamed from `swap_back_remove`)
* `Vec::resize`
* `str::slice_mut_unchecked`
* `FileTypeExt`
* `FileTypeExt::{is_block_device, is_char_device, is_fifo, is_socket}`
* `BinaryHeap::from` - `from_vec` deprecated in favor of this
* `BinaryHeap::into_vec` - plus a `Into` impl
* `BinaryHeap::into_sorted_vec`
Deprecated APIs
* `slice::ref_slice`
* `slice::mut_ref_slice`
* `iter::{range_inclusive, RangeInclusive}`
* `std::dynamic_lib`
Closes#27706Closes#27725
cc #27726 (align not stabilized yet)
Closes#27734Closes#27737Closes#27742Closes#27743Closes#27772Closes#27774Closes#27777Closes#27781
cc #27788 (a few remaining methods though)
Closes#27790Closes#27793Closes#27796Closes#27810
cc #28147 (not all parts stabilized)
This commit adds #[derive(Clone)] to std::fs::Metadata, making that struct
cloneable. Although the exact contents of that struct differ between OSes,
they all have it contain only value types, meaning that the data can be re-used without repercussions.
It also adds #[derive(Clone)] to every type used by that struct across all
OSes, including the various Unix `stat` structs and Windows's
`WIN32_FILE_ATTRIBUTE_DATA`.
- Rewrite `std::sys::fs::readlink` not to rely on `PATH_MAX`
It currently has the following problems:
1. It uses `_PC_NAME_MAX` to query the maximum length of a file path in
the underlying system. However, the meaning of the constant is the
maximum length of *a path component*, not a full path. The correct
constant should be `_PC_PATH_MAX`.
2. `pathconf` *may* fail if the referred file does not exist. This can
be problematic if the file which the symbolic link points to does not
exist, but the link itself does exist. In this case, the current
implementation resorts to the hard-coded value of `1024`, which is not
ideal.
3. There may exist a platform where there is no limit on file path
lengths in general. That's the reaon why GNU Hurd doesn't define
`PATH_MAX` at all, in addition to having `pathconf` always returning
`-1`. In these platforms, the content of the symbolic link can be
silently truncated if the length exceeds the hard-coded limit mentioned
above.
4. The value obtained by `pathconf` may be outdated at the point of
actually calling `readlink`. This is inherently racy.
This commit introduces a loop that gradually increases the length of the
buffer passed to `readlink`, eliminating the need of `pathconf`.
- Remove the arbitrary memory limit of `std::sys::fs::realpath`
As per POSIX 2013, `realpath` will return a malloc'ed buffer if the
second argument is a null pointer.[1]
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/realpath.html
- Comment on functions that are still using `PATH_MAX`
There are some functions that only work in terms of `PATH_MAX`, such as
`F_GETPATH` in OS X. Comments on them for posterity.
This commit removes all unstable and deprecated functions in the standard
library. A release was recently cut (1.3) which makes this a good time for some
spring cleaning of the deprecated functions.
This commit is an implementation of [RFC 1184][rfc] which tweaks the behavior of
the `#![no_std]` attribute and adds a new `#![no_core]` attribute. The
`#![no_std]` attribute now injects `extern crate core` at the top of the crate
as well as the libcore prelude into all modules (in the same manner as the
standard library's prelude). The `#![no_core]` attribute disables both std and
core injection.
[rfc]: https://github.com/rust-lang/rfcs/pull/1184
This commit is an implementation of [RFC 1174][rfc] which adds three new traits
to the standard library:
* `IntoRawFd` - implemented on Unix for all I/O types (files, sockets, etc)
* `IntoRawHandle` - implemented on Windows for files, processes, etc
* `IntoRawSocket` - implemented on Windows for networking types
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1174-into-raw-fd-socket-handle-traits.mdCloses#27062
