Gecko recently had a bug reported [1] with a deadlock in the Rust TLS
implementation for Windows. TLS destructors are implemented in a sort of ad-hoc
fashion on Windows as it doesn't natively support destructors for TLS keys. To
work around this the runtime manages a list of TLS destructors and registers a
hook to get run whenever a thread exits. When a thread exits it takes a look at
the list and runs all destructors.
Unfortunately it turns out that there's a lock which is held when our "at thread
exit" callback is run. The callback then attempts to acquire a lock protecting
the list of TLS destructors. Elsewhere in the codebase while we hold a lock over
the TLS destructors we try to acquire the same lock held first before our
special callback is run. And as a result, deadlock!
This commit sidesteps the issue with a few small refactorings:
* Removed support for destroying a TLS key on Windows. We don't actually ever
exercise this as a public-facing API, and it's only used during `lazy_init`
during racy situations. To handle that we just synchronize `lazy_init`
globally on Windows so we never have to call `destroy`.
* With no need to support removal the global synchronized `Vec` was tranformed
to a lock-free linked list. With the removal of locks this means that
iteration no long requires a lock and as such we won't run into the deadlock
problem mentioned above.
Note that it's still a general problem that you have to be extra super careful
in TLS destructors. For example no code which runs a TLS destructor on Windows
can call back into the Windows API to do a dynamic library lookup. Unfortunately
I don't know of a great way around that, but this at least fixes the immediate
problem that Gecko was seeing which is that with "well behaved" destructors the
system would still deadlock!
[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1358151
std: Use `poll` instead of `select`
This gives us the benefit of supporting file descriptors over the limit that
select supports, which...
Closes#40894
This reverts commit 2cf686f2cd (#40842)
RawFd is a type alias for c_int, which is itself a type alias for i32.
As a result, adding AsRawFd and IntoRawFd impls for RawFd actually adds
them for i32.
As a result, the reverted commit makes this valid:
```
use std::os::unix::io::AsRawFd;
fn arf<T: AsRawFd>(_: T) {}
fn main() {
arf(32i32)
}
```
Implimenting AsRawFd and IntoRawFd for i32 breaks the promises of both
those traits that their methods return a valid RawFd.
r? @aturon
cc @Mic92 @kamalmarhubi
Implement AsRawFd/IntoRawFd for RawFd
This is useful to build os abstraction like the nix crate does.
It allows to define functions, which accepts generic arguments
of data structures convertible to RawFd, including RawFd itself.
For example:
```
fn write<FD: AsRawFd>(fd: FD, buf: &[u8]) -> Result<usize>
write(file, buf);
```
instead of:
```
fn write(fd: RawFd, buf: &[u8]) -> Result<usize>
write(file.as_raw_fd(), buf);
```
cc @kamalmarhubi
This is useful to build os abstraction like the nix crate does.
It allows to define functions, which accepts generic arguments
of data structures convertible to RawFd, including RawFd itself.
For example:
fn write<FD: AsRawFd>(fd: FD, buf: &[u8]) -> Result<usize>
instead of:
fn write(fd: RawFd, buf: &[u8]) -> Result<usize>
write(foo.as_raw_fd(), buf);
Leftovers from #39594; From<Box> impls
These are a few more impls that follow the same reasoning as those from #39594.
What's included:
* `From<Box<str>> for String`
* `From<Box<[T]>> for Vec<T>`
* `From<Box<CStr>> for CString`
* `From<Box<OsStr>> for OsString`
* `From<Box<Path>> for PathBuf`
* `Into<Box<str>> for String`
* `Into<Box<[T]>> for Vec<T>`
* `Into<Box<CStr>> for CString`
* `Into<Box<OsStr>> for OsString`
* `Into<Box<Path>> for PathBuf`
* `<Box<CStr>>::into_c_string`
* `<Box<OsStr>>::into_os_string`
* `<Box<Path>>::into_path_buf`
* Tracking issue for latter three methods + three from previous PR.
Currently, the opposite direction isn't doable with `From` (only `Into`) because of the separation between `liballoc` and `libcollections`. I'm holding off on those for a later PR.
std::process for fuchsia: updated to latest liblaunchpad
Our liblaunchpad changed a bit and so fuchsia's std::process impl needs to change a bit.
@raphlinus
Improve backtrace formating while panicking.
Fixes#37783.
Done:
- Fix alignment of file paths for better readability
- `RUST_BACKTRACE=full` prints all the informations (current behaviour)
- `RUST_BACKTRACE=(short|yes)` is the default and does:
- Skip irrelevant frames at the beginning and the end
- Remove function address
- Remove the current directory from the absolute paths
- Remove `::hfabe6541873` at the end of the symbols
- `RUST_BACKTRACE=(0|no)` disables the backtrace.
- `RUST_BACKTRACE=<everything else>` is equivalent to `short` for
backward compatibility.
- doc
- More uniform printing across platforms.
Removed, TODO in a new PR:
- Remove path prefix for libraries and libstd
Example of short backtrace:
```rust
fn fail() {
panic!();
}
fn main() {
let closure = || fail();
closure();
}
```
Short:
```
thread 'main' panicked at 'explicit panic', t.rs:2
Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
0: t::fail
at ./t.rs:2
1: t::main::{{closure}}
at ./t.rs:6
2: t::main
at ./t.rs:7
```
Full:
```
thread 'main' panicked at 'This function never returns!', t.rs:2
stack backtrace:
0: 0x558ddf666478 - std::sys:👿:backtrace::tracing:👿:unwind_backtrace::hec84c9dd8389cc5d
at /home/yamakaky/dev/rust/rust/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: 0x558ddf65d90e - std::sys_common::backtrace::_print::hfa25f8b31f4b4353
at /home/yamakaky/dev/rust/rust/src/libstd/sys_common/backtrace.rs:71
2: 0x558ddf65cb5e - std::sys_common::backtrace::print::h9b711e11ac3ba805
at /home/yamakaky/dev/rust/rust/src/libstd/sys_common/backtrace.rs:60
3: 0x558ddf66796e - std::panicking::default_hook::{{closure}}::h736d216e74748044
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:355
4: 0x558ddf66743c - std::panicking::default_hook::h16baff397e46ea10
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:371
5: 0x558ddf6682bc - std::panicking::rust_panic_with_hook::h6d5a9bb4eca42c80
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:559
6: 0x558ddf64ea93 - std::panicking::begin_panic::h17dc549df2f10b99
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:521
7: 0x558ddf64ec42 - t::diverges::he6bc43fc925905f5
at /tmp/p/t.rs:2
8: 0x558ddf64ec5a - t::main::h0ffc20356b8a69c0
at /tmp/p/t.rs:6
9: 0x558ddf6687f5 - core::ops::FnOnce::call_once::hce41f19c0db56f93
10: 0x558ddf667cde - std::panicking::try::do_call::hd4c8c97efb4291df
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:464
11: 0x558ddf698d77 - __rust_try
12: 0x558ddf698c57 - __rust_maybe_catch_panic
at /home/yamakaky/dev/rust/rust/src/libpanic_unwind/lib.rs:98
13: 0x558ddf667adb - std::panicking::try::h2c56ed2a59ec1d12
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:440
14: 0x558ddf66cc9a - std::panic::catch_unwind::h390834e0251cc9af
at /home/yamakaky/dev/rust/rust/src/libstd/panic.rs:361
15: 0x558ddf6809ee - std::rt::lang_start::hb73087428e233982
at /home/yamakaky/dev/rust/rust/src/libstd/rt.rs:57
16: 0x558ddf64ec92 - main
17: 0x7fecb869e290 - __libc_start_main
18: 0x558ddf64e8b9 - _start
19: 0x0 - <unknown>
```
Switch Fuchsia to readdir (instead of readdir_r)
The readdir_r function is deprecated on newer Posix systems because of
various problems, and not implemented at all for Fuchsia. There are
already implementations using both, and this patch switches Fuchsia
over to the readdir-based one.
Fixes#40021 for Fuchsia, but that issue also contains discussion of
what should happen for other Posix systems.
Follow rename of mx_handle_wait Magenta syscalls
The mx_handle_wait_* syscalls in Magenta were renamed to
mx_object_wait. The syscall is used in the Magenta/Fuchsia
implementation of std::process, to wait on child processes.
In addition, this patch enables the use of the system provided
libbacktrace library on Fuchsia targets. Symbolization is not yet
working, but at least it allows printing hex addresses in a backtrace
and makes building succeed when the backtrace feature is not disabled.
The readdir_r function is deprecated on newer Posix systems because of
various problems, and not implemented at all for Fuchsia. There are
already implementations using both, and this patch switches Fuchsia
over to the readdir-based one.
Fixes#40021 for Fuchsia, but that issue also contains discussion of
what should happen for other Posix systems.
The mx_handle_wait_* syscalls in Magenta were renamed to
mx_object_wait. The syscall is used in the Magenta/Fuchsia
implementation of std::process, to wait on child processes.
In addition, this patch enables the use of the system provided
libbacktrace library on Fuchsia targets. Symbolization is not yet
working, but at least it allows printing hex addresses in a backtrace
and makes building succeed when the backtrace feature is not disabled.
- `RUST_BACKTRACE=full` prints all the informations (old behaviour)
- `RUST_BACKTRACE=(0|no)` disables the backtrace.
- `RUST_BACKTRACE=<everything else>` (including `1`) shows a simplified
backtrace, without the function addresses and with cleaned filenames
and symbols. Also removes some unneded frames at the beginning and the
end.
Fixes#37783.
PR is #38165.
make Child::try_wait return io::Result<Option<ExitStatus>>
This is much nicer for callers who want to short-circuit real I/O errors
with `?`, because they can write this
if let Some(status) = foo.try_wait()? {
...
} else {
...
}
instead of this
match foo.try_wait() {
Ok(status) => {
...
}
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {
...
}
Err(err) => return Err(err),
}
The original design of `try_wait` was patterned after the `Read` and
`Write` traits, which support both blocking and non-blocking
implementations in a single API. But since `try_wait` is never blocking,
it makes sense to optimize for the non-blocking case.
Tracking issue: https://github.com/rust-lang/rust/issues/38903
Delete the makefile build system
This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild.
Rustbuild has been the default build system [since 1.15.0](https://github.com/rust-lang/rust/pull/37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year.
And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!
Add support for test suites emulated in QEMU
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
This is much nicer for callers who want to short-circuit real I/O errors
with `?`, because they can write this
if let Some(status) = foo.try_wait()? {
...
} else {
...
}
instead of this
match foo.try_wait() {
Ok(status) => {
...
}
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {
...
}
Err(err) => return Err(err),
}
The original design of `try_wait` was patterned after the `Read` and
`Write` traits, which support both blocking and non-blocking
implementations in a single API. But since `try_wait` is never blocking,
it makes sense to optimize for the non-blocking case.
Tracking issue: https://github.com/rust-lang/rust/issues/38903
Use less syscalls in `FileDesc::set_{nonblocking,cloexec}`
Only set the flags if they differ from what the OS reported, use
`FIONBIO` to atomically set the non-blocking IO flag on Linux.
Add peek APIs to std::net
Adds "peek" APIs to `std::net` sockets, including:
- `UdpSocket.peek()`
- `UdpSocket.peek_from()`
- `TcpStream.peek()`
These methods enable socket reads without side-effects. That is, repeated calls to `peek()` return identical data. This is accomplished by providing the POSIX flag `MSG_PEEK` to the underlying socket read operations.
This also moves the current implementation of `recv_from` out of the platform-independent `sys_common` and into respective `sys/windows` and `sys/unix` implementations. This allows for more platform-dependent implementations where necessary.
Fixes#38980
Expand documentation of process::exit and exec
Show a conventional way to use process::exit when destructors are considered important and also
mention that the same caveats wrt destructors apply to exec as well.
These methods enable socket reads without side-effects. That is,
repeated calls to peek() return identical data. This is accomplished
by providing the POSIX flag MSG_PEEK to the underlying socket read
operations.
This also moves the current implementation of recv_from out of the
platform-independent sys_common and into respective sys/windows and
sys/unix implementations. This allows for more platform-dependent
implementations.
Use less syscalls in `anon_pipe()`
Save a `ENOSYS` failure from `pipe2` and don't try again.
Use `cvt` instead of `cvt_r` for `pipe2` - `EINTR` is not an error
`pipe2` can return.
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
Make backtraces work on Windows GNU targets again.
This is done by adding a function that can return a filename
to pass to backtrace_create_state. The filename is obtained in
a safe way by first getting the filename, locking the file so it can't
be moved, and then getting the filename again and making sure it's the same.
See: https://github.com/rust-lang/rust/pull/37359#issuecomment-260123399
Issue: #33985
Note though that this isn't that pretty...
I had to implement a `WideCharToMultiByte` wrapper function to convert to the ANSI code page. This will work better than only allowing ASCII provided that the ANSI code page is set to the user's local language, which is often the case.
Also, please make sure that I didn't break the Unix build.
This is done by adding a function that can return a filename
to pass to backtrace_create_state. The filename is obtained in
a safe way by first getting the filename, locking the file so it can't
be moved, and then getting the filename again and making sure it's the same.
See: https://github.com/rust-lang/rust/pull/37359#issuecomment-260123399
Issue: #33985
Show a conventional way to use process::exit when destructors are considered important and also
mention that the same caveats wrt destructors apply to exec as well.
This commit adds a new method to the `Child` type in the `std::process` module
called `try_wait`. This method is the same as `wait` except that it will not
block the calling thread and instead only attempt to collect the exit status. On
Unix this means that we call `waitpid` with the `WNOHANG` flag and on Windows it
just means that we pass a 0 timeout to `WaitForSingleObject`.
Currently it's possible to build this method out of tree, but it's unfortunately
tricky to do so. Specifically on Unix you essentially lose ownership of the pid
for the process once a call to `waitpid` has succeeded. Although `Child` tracks
this state internally to be resilient to multiple calls to `wait` or a `kill`
after a successful wait, if the child is waited on externally then the state
inside of `Child` is not updated. This means that external implementations of
this method must be extra careful to essentially not use a `Child`'s methods
after a call to `waitpid` has succeeded (even in a nonblocking fashion).
By adding this functionality to the standard library it should help canonicalize
these external implementations and ensure they can continue to robustly reuse
the `Child` type from the standard library without worrying about pid ownership.
Turns out that even though all these functions take a `size_t` they don't
actually work that well with anything larger than the maximum value of
`ssize_t`, the return value. Furthermore it looks like OSX rejects any
read/write requests larger than `INT_MAX - 1`. Handle all these cases by just
clamping the maximum size of a read/write on Unix to a platform-specific value.
Closes#38590
Removes magenta build warning.
Small bug fix to remove an unused type in the magenta process code that causes build failures for magenta's rustc.
r? @alexcrichton
@tedsta @raphlinus
Fuchsia support for std::process via liblaunchpad.
Now we can launch processes on Fuchsia via the Rust standard library! ... Mostly.
Right now, ~5% of the time, reading the stdout/stderr off the pipes will fail. Some Magenta kernel people think it's probably a bug in Magenta's pipes. I wrote a unit test that demonstrates the issue in C, which I was told will expedite a fix. https://fuchsia-review.googlesource.com/#/c/15628/
Hopefully this can get merged once the issue is fixed :)
@raphlinus
On unix like systems, the underlying file corresponding to any given path may
change at any time. This function makes it possible to set the permissions of
the a file corresponding to a `File` object even if its path changes.
The mx_cprng_draw syscall has changed signature to separate the status
and size return values, rather than multiplexing them into a single
value with errors interpreted as a negative value. This patch tracks
that change.
The DirEntryExt::ino() implementation was omitted from the first
iteration of this patch, because a dependency needed to be
configured. The fix is straightforward enough.
It is good practice to implement Debug for public types, and
indicating what directory you're reading seems useful.
Signed-off-by: David Henningsson <diwic@ubuntu.com>
Implement `read_offset` and `write_offset`
These functions allow to read from and write to a file from multiple
threads without changing the per-file cursor, avoiding the race between
the seek and the read.
These functions allow to read from and write to a file in one atomic
action from multiple threads, avoiding the race between the seek and the
read.
The functions are named `{read,write}_at` on non-Windows (which don't
change the file cursor), and `seek_{read,write}` on Windows (which
change the file cursor).
When getaddrinfo returns EAI_SYSTEM retrieve actual error from errno.
Fixes issue #36546. This change also updates libc to earliest version
that includes EAI_SYSTEM constant.
Previously, in cases where `EAI_SYSTEM` has been returned from getaddrinfo, the
resulting `io::Error` would be broadly described as "System error":
Error { repr: Custom(Custom { kind: Other, error: StringError("failed to lookup address information: System error") }) }
After change a more detailed error is crated based on particular value of
errno, for example:
Error { repr: Os { code: 64, message: "Machine is not on the network" } }
The only downside is that the prefix "failed to lookup address information" is
no longer included in the error message.
This adds support for building the Rust compiler and standard
library for s390x-linux, allowing a full cross-bootstrap sequence
to complete. This includes:
- Makefile/configure changes to allow native s390x builds
- Full Rust compiler support for the s390x C ABI
(only the non-vector ABI is supported at this point)
- Port of the standard library to s390x
- Update the liblibc submodule to a version including s390x support
- Testsuite fixes to allow clean "make check" on s390x
Caveats:
- Resets base cpu to "z10" to bring support in sync with the default
behaviour of other compilers on the platforms. (Usually, upstream
supports all older processors; a distribution build may then chose
to require a more recent base version.) (Also, using zEC12 causes
failures in the valgrind tests since valgrind doesn't fully support
this CPU yet.)
- z13 vector ABI is not yet supported. To ensure compatible code
generation, the -vector feature is passed to LLVM. Note that this
means that even when compiling for z13, no vector instructions
will be used. In the future, support for the vector ABI should be
added (this will require common code support for different ABIs
that need different data_layout strings on the same platform).
- Two test cases are (temporarily) ignored on s390x to allow passing
the test suite. The underlying issues still need to be fixed:
* debuginfo/simd.rs fails because of incorrect debug information.
This seems to be a LLVM bug (also seen with C code).
* run-pass/union/union-basic.rs simply seems to be incorrect for
all big-endian platforms.
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Fix argument to FIONBIO ioctl
The FIONBIO ioctl takes as argument a pointer to an integer, which
should be either 0 or 1 to indicate whether nonblocking mode is to
be switched off or on. The type of the pointed-to variable is "int".
However, the set_nonblocking routine in libstd/sys/unix/net.rs passes
a pointer to a libc::c_ulong variable. This doesn't matter on all
32-bit platforms and on all litte-endian platforms, but it will
break on big-endian 64-bit platforms.
Found while porting Rust to s390x (a big-endian 64-bit platform).
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
The FIONBIO ioctl takes as argument a pointer to an integer, which
should be either 0 or 1 to indicate whether nonblocking mode is to
be switched off or on. The type of the pointed-to variable is "int".
However, the set_nonblocking routine in libstd/sys/unix/net.rs passes
a pointer to a libc::c_ulong variable. This doesn't matter on all
32-bit platforms and on all litte-endian platforms, but it will
break on big-endian 64-bit platforms.
Found while porting Rust to s390x (a big-endian 64-bit platform).
Signed-off-by: Ulrich Weigand <ulrich.weigand@de.ibm.com>
Use monotonic time in condition variables.
Configure condition variables to use monotonic time using
pthread_condattr_setclock on systems where this is possible.
This fixes the issue when thread waiting on condition variable is
woken up too late when system time is moved backwards.
Use arc4rand(9) on FreeBSD
From rust-lang-nursery/rand#112:
>After reading through #30691 it seems that there's general agreement that using OS-provided facilities for seeding rust userland processes is fine as long as it doesn't use too much from libc. FreeBSD's `arc4random_buf(3)` is not only a whole lot of libc code, but also not even currently exposed in the libc crate. Fortunately, the mechanism `arc4random_buf(3)` et al. use for getting entropy from the kernel ([`arc4rand(9)`](https://www.freebsd.org/cgi/man.cgi?query=arc4random&apropos=0&sektion=9&manpath=FreeBSD+10.3-RELEASE&arch=default&format=html)) is exposed via `sysctl(3)` with constants that are already in the libc crate.
>I haven't found too much documentation on `KERN_ARND`—it's missing or only briefly described in most of the places that cover sysctl mibs. But, from digging through the kernel source, it appears that the sysctl used in this PR is very close to just calling `arc4rand(9)` directly (with `reseed` set to 0 and no way to change it).
I expected [rand](/rust-lang-nursery/rand) to reply quicker, so I tried submitting it there first. It's been a few weeks with no comment, so I don't know the state of it, but maybe someone will see it here and have an opinion. This is basically the same patch. It pains me to duplicate the code but I guess it hasn't been factored out into just one place yet.
Emscripten test fixes
This picks up parts of #31623 to disable certain tests that emscripten can't run, as threads/processes are not supported.
I re-applied @tomaka's changes manually, I can rebase those commits with his credentials if he wants.
It also disables jemalloc for emscripten (at least in Rustbuild, I have to check if there is another setting for the same thing in the old makefile approach).
This should not impact anything for normal builds.
Fix build on DragonFly (unused function errno_location)
Function errno_location() is not used on DragonFly. As warnings are
errors, this breaks the build.
Handle RwLock reader count overflow
`pthread_rwlock_rdlock` may return `EAGAIN` if the maximum reader count overflows. We shouldn't return a successful lock in that case.
Configure condition variables to use monotonic time using
pthread_condattr_setclock on systems where this is possible.
This fixes the issue when thread waiting on condition variable is
woken up too late when system time is moved backwards.
rustbuild: make backtraces (RUST_BACKTRACE) optional
but keep them enabled by default to maintain the status quo.
When disabled shaves ~56KB off every x86_64-unknown-linux-gnu
binary.
To disable backtraces you have to use a config.toml (see
src/bootstrap/config.toml.example for details) when building rustc/std:
$ python bootstrap.py --config=config.toml
---
r? @alexcrichton
cc rust-lang/rfcs#1417
std: Fix usage of SOCK_CLOEXEC
This code path was intended to only get executed on Linux, but unfortunately the
`cfg!` was malformed so it actually never got executed.
DoubleEndedIterator for Args
This PR implements the DoubleEndedIterator trait for the `std::env::Args[Os]` structure, as well
as the internal implementations.
It is primarily motivated by me, as I happened to implement a simple `reversor` program many times
now, which so far had to use code like this:
```Rust
for arg in std::env::args().skip(1).collect::<Vec<_>>().iter().rev() {}
```
... even though I would have loved to do this instead:
```Rust
for arg in std::env::args().skip(1).rev() {}
```
The latter is more natural, and I did not find a reason for not implementing it.
After all, on every system, the number of arguments passed to the program are known
at runtime.
To my mind, it follows KISS, and does not try to be smart at all. Also, there are no unit-tests,
primarily as I did not find any existing tests for the `Args` struct either.
The windows implementation is basically a copy-pasted variant of the `next()` method implementation,
and I could imagine sharing most of the code instead. Actually I would be happy if the reviewer would
ask for it.
but keep them enabled by default to maintain the status quo.
When disabled shaves ~56KB off every x86_64-unknown-linux-gnu
binary.
To disable backtraces you have to use a config.toml (see
src/bootstrap/config.toml.example for details) when building rustc/std:
$ python bootstrap.py --config=config.toml
The number of arguments given to a process is always known, which
makes implementing DoubleEndedIterator possible.
That way, the Iterator::rev() method becomes usable, among others.
Signed-off-by: Sebastian Thiel <byronimo@gmail.com>
Tidy for DoubleEndedIterator
I chose to not create a new feature for it, even though
technically, this makes me lie about the original availability
of the implementation.
Verify with @alexchrichton
Setup feature flag for new std::env::Args iterators
Add test for Args reverse iterator
It's somewhat depending on the input of the test program,
but made in such a way that should be somewhat flexible to changes
to the way it is called.
Deduplicate windows ArgsOS code for DEI
DEI = DoubleEndedIterator
Move env::args().rev() test to run-pass
It must be controlling it's arguments for full isolation.
Remove superfluous feature name
Assert all arguments returned by env::args().rev()
Let's be very sure it works as we expect, why take chances.
Fix rval of os_string_from_ptr
A trait cannot be returned, but only the corresponding object.
Deref pointers to actually operate on the argument
Put unsafe to correct location
std: fix `readdir` errors for solaris
A `NULL` from `readdir` could be the end of stream or an error. The only
way to know is to check `errno`, so it must be set to a known value first,
like a 0 that POSIX will never use.
This currently only matters for solaris targets, as the other unix platforms
are using `readdir_r` with a direct error return indication. However, this is
getting deprecated (#34668) so they should all eventually switch to `readdir`.
This PR adds `set_errno`, uses it to clear the value before calling `readdir`,
then checks it again after to see the reason for a `NULL`. A few other small
fixes are included just to get solaris compiling at all.
I couldn't get cross-compilation completely going, so I don't have a good way
to test this beyond a smoke-test cargo build of std. I'd appreciate input from
someone more familiar with solaris -- cc @nbaksalyar?
A `NULL` from `readdir` could be the end of stream or an error. The
only way to know is to check `errno`, so it must be set to a known value
first, like a 0 that POSIX will never use.
This patch adds `set_errno`, uses it to clear the value before calling
`readdir`, then checks it again after to see the reason for a `NULL`.
The `use ffi::CStr` in `unix/thread.rs` was previously guarded, but now
all platforms need it for `Thread::set_name()`. Newlib and Solaris do
nothing here, as they have no way to set a thread name, but they still
define the same method signature.
Although the set of APIs being stabilized this release is relatively small, the
trains keep going! Listed below are the APIs in the standard library which have
either transitioned from unstable to stable or those from unstable to
deprecated.
Stable
* `BTreeMap::{append, split_off}`
* `BTreeSet::{append, split_off}`
* `Cell::get_mut`
* `RefCell::get_mut`
* `BinaryHeap::append`
* `{f32, f64}::{to_degrees, to_radians}` - libcore stabilizations mirroring past
libstd stabilizations
* `Iterator::sum`
* `Iterator::product`
Deprecated
* `{f32, f64}::next_after`
* `{f32, f64}::integer_decode`
* `{f32, f64}::ldexp`
* `{f32, f64}::frexp`
* `num::One`
* `num::Zero`
Added APIs (all unstable)
* `iter::Sum`
* `iter::Product`
* `iter::Step` - a few methods were added to accomodate deprecation of One/Zero
Removed APIs
* `From<Range<T>> for RangeInclusive<T>` - everything about `RangeInclusive` is
unstable
Closes#27739Closes#27752Closes#32526Closes#33444Closes#34152
cc #34529 (new tracking issue)
Don't ignore errors of syscalls in std::sys::unix::fd
If any of these syscalls fail, it indicates a programmer error that
should not be silently ignored.
std: Fix up stabilization discrepancies
* Remove the deprecated `CharRange` type which was forgotten to be removed
awhile back.
* Stabilize the `os::$platform::raw::pthread_t` type which was intended to be
stabilized as part of #32804
* Remove the deprecated `CharRange` type which was forgotten to be removed
awhile back.
* Stabilize the `os::$platform::raw::pthread_t` type which was intended to be
stabilized as part of #32804
The only applies to pthread mutexes. We solve this by creating the
mutex with the PTHREAD_MUTEX_NORMAL type, which guarantees that
re-locking from the same thread will deadlock.
rustc: Implement custom panic runtimes
This commit is an implementation of [RFC 1513] which allows applications to
alter the behavior of panics at compile time. A new compiler flag, `-C panic`,
is added and accepts the values `unwind` or `panic`, with the default being
`unwind`. This model affects how code is generated for the local crate, skipping
generation of landing pads with `-C panic=abort`.
[RFC 1513]: https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md
Panic implementations are then provided by crates tagged with
`#![panic_runtime]` and lazily required by crates with
`#![needs_panic_runtime]`. The panic strategy (`-C panic` value) of the panic
runtime must match the final product, and if the panic strategy is not `abort`
then the entire DAG must have the same panic strategy.
With the `-C panic=abort` strategy, users can expect a stable method to disable
generation of landing pads, improving optimization in niche scenarios,
decreasing compile time, and decreasing output binary size. With the `-C
panic=unwind` strategy users can expect the existing ability to isolate failure
in Rust code from the outside world.
Organizationally, this commit dismantles the `sys_common::unwind` module in
favor of some bits moving part of it to `libpanic_unwind` and the rest into the
`panicking` module in libstd. The custom panic runtime support is pretty similar
to the custom allocator support with the only major difference being how the
panic runtime is injected (takes the `-C panic` flag into account).
Closes#32837
std: Allow creating ExitStatus from raw values
Sometimes a process may be waited on externally from the standard library, in
which case it can be useful to create a raw `ExitStatus` structure to return.
This commit extends the existing Unix `ExitStatusExt` extension trait and adds a
new Windows-specific `ExitStatusExt` extension trait to do this. The methods are
currently called `ExitStatus::from_raw`.
cc #32713
This commit is an implementation of [RFC 1513] which allows applications to
alter the behavior of panics at compile time. A new compiler flag, `-C panic`,
is added and accepts the values `unwind` or `panic`, with the default being
`unwind`. This model affects how code is generated for the local crate, skipping
generation of landing pads with `-C panic=abort`.
[RFC 1513]: https://github.com/rust-lang/rfcs/blob/master/text/1513-less-unwinding.md
Panic implementations are then provided by crates tagged with
`#![panic_runtime]` and lazily required by crates with
`#![needs_panic_runtime]`. The panic strategy (`-C panic` value) of the panic
runtime must match the final product, and if the panic strategy is not `abort`
then the entire DAG must have the same panic strategy.
With the `-C panic=abort` strategy, users can expect a stable method to disable
generation of landing pads, improving optimization in niche scenarios,
decreasing compile time, and decreasing output binary size. With the `-C
panic=unwind` strategy users can expect the existing ability to isolate failure
in Rust code from the outside world.
Organizationally, this commit dismantles the `sys_common::unwind` module in
favor of some bits moving part of it to `libpanic_unwind` and the rest into the
`panicking` module in libstd. The custom panic runtime support is pretty similar
to the custom allocator support with the only major difference being how the
panic runtime is injected (takes the `-C panic` flag into account).
rand: don't block before random pool is initialized
If we attempt a read with getrandom() on Linux the syscall can block
before the random pool is initialized unless the GRND_NONBLOCK flag is
passed. This flag causes getrandom() to instead return EAGAIN while the
pool is uninitialized. To avoid downstream users of crate or std
functionality that have no ability to avoid this blocking behavior this
change causes Rust to read bytes from /dev/urandom while getrandom()
would block and once getrandom() is available to use that. Fixes#32953.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
The Gecko folks currently use Android API level 9 for their builds, so they're
requesting that we move back our minimum supported API level from 18 to 9. Turns
out, ABI-wise at least, there's not that many changes we need to take care of.
The `ftruncate64` API appeared in android-12 and the `log2` and `log2f` APIs
appeared in android-18. We can have a simple shim for `ftruncate64` which falls
back on `ftruncate` and the `log2` function can be approximated with just
`ln(f) / ln(2)`.
This should at least get the standard library building on API level 9, although
the tests aren't quite happening there just yet. As we seem to be growing a
number of Android compatibility shims, they're now centralized in a common
`sys::android` module.
Sometimes a process may be waited on externally from the standard library, in
which case it can be useful to create a raw `ExitStatus` structure to return.
This commit extends the existing Unix `ExitStatusExt` extension trait and adds a
new Windows-specific `ExitStatusExt` extension trait to do this. The methods are
currently called `ExitStatus::from_raw`.
cc #32713
Add some comments so that people know why we are performing a fallback
from getrandom() and what that fallback aims to achieve.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
If we attempt a read with getrandom() on Linux the syscall can block
before the random pool is initialized unless the GRND_NONBLOCK flag is
passed. This flag causes getrandom() to instead return EAGAIN while the
pool is uninitialized. To avoid downstream users of crate or std
functionality that have no ability to avoid this blocking behavior this
change causes Rust to read bytes from /dev/urandom while getrandom()
would block and once getrandom() is available to use that. Fixes#32953.
Signed-off-by: Doug Goldstein <cardoe@cardoe.com>
This commit applies all stabilizations, renamings, and deprecations that the
library team has decided on for the upcoming 1.9 release. All tracking issues
have gone through a cycle-long "final comment period" and the specific APIs
stabilized/deprecated are:
Stable
* `std::panic`
* `std::panic::catch_unwind` (renamed from `recover`)
* `std::panic::resume_unwind` (renamed from `propagate`)
* `std::panic::AssertUnwindSafe` (renamed from `AssertRecoverSafe`)
* `std::panic::UnwindSafe` (renamed from `RecoverSafe`)
* `str::is_char_boundary`
* `<*const T>::as_ref`
* `<*mut T>::as_ref`
* `<*mut T>::as_mut`
* `AsciiExt::make_ascii_uppercase`
* `AsciiExt::make_ascii_lowercase`
* `char::decode_utf16`
* `char::DecodeUtf16`
* `char::DecodeUtf16Error`
* `char::DecodeUtf16Error::unpaired_surrogate`
* `BTreeSet::take`
* `BTreeSet::replace`
* `BTreeSet::get`
* `HashSet::take`
* `HashSet::replace`
* `HashSet::get`
* `OsString::with_capacity`
* `OsString::clear`
* `OsString::capacity`
* `OsString::reserve`
* `OsString::reserve_exact`
* `OsStr::is_empty`
* `OsStr::len`
* `std::os::unix::thread`
* `RawPthread`
* `JoinHandleExt`
* `JoinHandleExt::as_pthread_t`
* `JoinHandleExt::into_pthread_t`
* `HashSet::hasher`
* `HashMap::hasher`
* `CommandExt::exec`
* `File::try_clone`
* `SocketAddr::set_ip`
* `SocketAddr::set_port`
* `SocketAddrV4::set_ip`
* `SocketAddrV4::set_port`
* `SocketAddrV6::set_ip`
* `SocketAddrV6::set_port`
* `SocketAddrV6::set_flowinfo`
* `SocketAddrV6::set_scope_id`
* `<[T]>::copy_from_slice`
* `ptr::read_volatile`
* `ptr::write_volatile`
* The `#[deprecated]` attribute
* `OpenOptions::create_new`
Deprecated
* `std::raw::Slice` - use raw parts of `slice` module instead
* `std::raw::Repr` - use raw parts of `slice` module instead
* `str::char_range_at` - use slicing plus `chars()` plus `len_utf8`
* `str::char_range_at_reverse` - use slicing plus `chars().rev()` plus `len_utf8`
* `str::char_at` - use slicing plus `chars()`
* `str::char_at_reverse` - use slicing plus `chars().rev()`
* `str::slice_shift_char` - use `chars()` plus `Chars::as_str`
* `CommandExt::session_leader` - use `before_exec` instead.
Closes#27719
cc #27751 (deprecating the `Slice` bits)
Closes#27754Closes#27780Closes#27809Closes#27811Closes#27830Closes#28050Closes#29453Closes#29791Closes#29935Closes#30014Closes#30752Closes#31262
cc #31398 (still need to deal with `before_exec`)
Closes#31405Closes#31572Closes#31755Closes#31756
Following changes:
* birthtime does not exist on DragonFly
* errno: __dfly_error is no more. Use #[thread_local] static errno.
* clock_gettime expects a c_ulong (use a type alias)
These changes are required to build DragonFly snapshots again.
Previously, the thread name (&str) was converted to a CString in the
new thread, but outside unwind::try, causing a panic to continue into FFI.
This patch changes that behaviour, so that the panic instead happens
in the parent thread (where panic infrastructure is properly set up),
not the new thread.
This could potentially be a breaking change for architectures who don't
support thread names.
Signed-off-by: David Henningsson <diwic@ubuntu.com>
Automated conversion using the untry tool [1] and the following command:
```
$ find -name '*.rs' -type f | xargs untry
```
at the root of the Rust repo.
[1]: https://github.com/japaric/untry
This code was currently only exercised on OSX, but this applies the same method
of subtraction used on Linux which doesn't have the same overflow issues.
Note that this currently includes no tests, but that's because this is only
visible with debug assertions enabled. Soon, however, I'll enable debug
assertions on all auto builds on the bots so we should get testing for this.
Closes#32268
Removes all unstable and deprecated APIs prior to the 1.8 release. All APIs that
are deprecated in the 1.8 release are sticking around for the rest of this
cycle.
Some notable changes are:
* The `dynamic_lib` module was moved into `rustc_back` as the compiler still
relies on a few bits and pieces.
* The `DebugTuple` formatter now special-cases an empty struct name with only
one field to append a trailing comma.
Semantically there's actually no reason for us to spawn threads as part of the
call to `wait_with_output`, and that's generally an incredibly heavyweight
operation for just reading a few bytes (especially when stderr probably rarely
has bytes!). An equivalent operation in terms of what's implemented today would
be to just drain both pipes of all contents and then call `wait` on the child
process itself.
On Unix we can implement this through some convenient use of the `select`
function, whereas on Windows we can make use of overlapped I/O. Note that on
Windows this requires us to use named pipes instead of anonymous pipes, but
they're semantically the same under the hood.
For example if `Command::output` or `Command::status` is used then stdin is just
immediately closed. Add an option for this so as an optimization we can avoid
creating pipes entirely.
This should help reduce the number of active file descriptors when spawning
processes on Unix and the number of active handles on Windows.
This pushes the implementation detail of proxying `read_to_end` through to
`read_to_end_uninitialized` all the way down to the `FileDesc` and `Handle`
implementations on Unix/Windows. This way intermediate layers will also be able
to take advantage of this optimized implementation.
This commit also adds the optimized implementation for `ChildStdout` and
`ChildStderr`.
Squashed 10 commits:
1) The main cause of the problem is that libstd/os/mod.rs treats emscripten targets as an alias of linux targets, whereas liblibc treats emscripten targets as musl-compliant, so it gets a slightly different struct stat64 defined.
This commit adds conditional compilation checks to use the correct timestamp format on fs metadata functions in the case of compiling to emscripten targets.
2) Update previous commit to comply with rust formatting standards.
Removed tab characters, remove trailing whitespaces.
3) Move emscripten changes into their own file under libstd/os/emscripten
Put libstd/os/linux/fs back to the way it was.
4) Cannot use stat.st_ctim on emscripten to get created time.
5) Remove compile-time conditionals for target_env = musl, it looks like musl builds compile fine already.
6) Undone some formatting changes that are no longer needed,
Removed some more target_env="musl" compilation checks that I missed from my previous commit.
7) upgrade to liblibc e19309c, it fixes the differences in the musl stat and stat64 definitions.
8) Undo the compile-time checks to check for emscripten (or musl targets) in the FileAttr struct.
No longer needed after updating liblibc to e19309c.
9) Change the MetadataExt implementation of emscripten fs.rs module to match the changes in new liblibc.
10) remove a stray return statement, should have been removed in the previous commit.
Also back out keepalive support for TCP since the API is perhaps not
actually what we want. You can't read the interval on Windows, and
we should probably separate the functionality of turning keepalive on
and overriding the interval.
Android should use 64-bit LFS symbols for `lseek` and `ftruncate`, lest
those offset parameters suffer a lossy cast down to a 32-bit `off_t`.
Unlike GNU/Linux, Android's `stat`, `dirent`, and related functions are
always 64-bit LFS compatible, and `open` already implies `O_LARGEFILE`,
so all those don't need to follow Linux. It might be nice to unify them
anyway, but those other LFS symbols aren't present in API 18 bionic.
r? @alexcrichton
Some struct members have a slighty different name on NetBSD. This has been
fixed in the libc crate, but not in libstd.
This also removes `st_spare` from MetadataExt, since it is private field
reserved for future use.
Android should use 64-bit LFS symbols for `lseek` and `ftruncate`, lest
those offset parameters suffer a lossy cast down to a 32-bit `off_t`.
Unlike GNU/Linux, Android's `stat`, `dirent`, and related functions are
always 64-bit LFS compatible, and `open` already implies `O_LARGEFILE`,
so all those don't need to follow Linux. It might be nice to unify them
anyway, but those other LFS symbols aren't present in API 18 bionic.
r? @alexcrichton
Remove alternate stack with sigaltstack before unmaping it.
Also reuse existing signal stack if already set, this is especially
useful when working with sanitizers that configure alternate stack
themselves.
This change depends on SS_DISABLE recently introduced in libc crate and updates
this git submodule accordingly.
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md
Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.
To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.
The commit here incorporates two backwards-compatible components:
* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
accessors of all fields. The fields now returned widened types which are the
same across platforms (consistency across platforms is not required, however,
it's just convenient).
and two also backwards-incompatible components:
* Change the definition of all `std::os::*::raw` type aliases to
correspond to the newly widened types that are being returned on each
platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
definitions rather than the standard ones.
The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.
[audit]: https://github.com/rust-lang/rfcs/pull/1415#issuecomment-180645582
---
Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!
Closes#28978Closes#30050Closes#31549
This commit is an implementation of [RFC 1415][rfc] which deprecates all types
in the `std::os::*::raw` modules.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1415-trim-std-os.md
Many of the types in these modules don't actually have a canonical platform
representation, for example the definition of `stat` on 32-bit Linux will change
depending on whether C code is compiled with LFS support or not. Unfortunately
the current types in `std::os::*::raw` are billed as "compatible with C", which
in light of this means it isn't really possible.
To make matters worse, platforms like Android sometimes define these types as
*smaller* than the way they're actually represented in the `stat` structure
itself. This means that when methods like `DirEntry::ino` are called on Android
the result may be truncated as we're tied to returning a `ino_t` type, not the
underlying type.
The commit here incorporates two backwards-compatible components:
* Deprecate all `raw` types that aren't in `std::os::raw`
* Expand the `std::os::*::fs::MetadataExt` trait on all platforms for method
accessors of all fields. The fields now returned widened types which are the
same across platforms (consistency across platforms is not required, however,
it's just convenient).
and two also backwards-incompatible components:
* Change the definition of all `std::os::*::raw` type aliases to
correspond to the newly widened types that are being returned on each
platform.
* Change the definition of `std::os::*::raw::stat` on Linux to match the LFS
definitions rather than the standard ones.
The breaking changes here will specifically break code that assumes that `libc`
and `std` agree on the definition of `std::os::*::raw` types, or that the `std`
types are faithful representations of the types in C. An [audit] has been
performed of crates.io to determine the fallout which was determined two be
minimal, with the two found cases of breakage having been fixed now.
[audit]: https://github.com/rust-lang/rfcs/pull/1415#issuecomment-180645582
---
Ok, so after all that, we're finally able to support LFS on Linux! This commit
then simultaneously starts using `stat64` and friends on Linux to ensure that we
can open >4GB files on 32-bit Linux. Yay!
Closes#28978Closes#30050Closes#31549
This series of commits adds the initial implementation of a new build system for
the compiler and standard library based on Cargo. The high-level architecture
now looks like:
1. The `./configure` script is run with `--enable-rustbuild` and other standard
configuration options.
2. A `Makefile` is generate which proxies commands to the new build system.
3. The new build system has a Python script entry point which manages
downloading both a Rust and Cargo nightly. This initial script also manages
building the build system itself (which is written in Rust).
4. The build system, written in rust and called `bootstrap`, architects how to
call `cargo` and manages building all native libraries and such.
One might reasonably ask "why rewrite the build system?", which is a good
question! The Rust project has used Makefiles for as long as I can remember at
least, and while ugly and difficult to use are undeniably robust as they contain
years worth of tweaking and tuning for working on as many platforms in as many
situation as possible. The rationale behind this PR, however is:
* The makefiles are impenetrable to all but a few people on this
planet. This means that contributions to the build system are almost
nonexistent, and furthermore if a build system change is needed it's
incredibly difficult to figure out how to do so. This hindrance prevents us
from doing some "perhaps fancier" things we may wish to do in make.
* Our build system, while portable, is unfortunately not infinitely portable
everywhere. For example the recently-introduced MSVC target is quite unlikely
to have `make` installed by default (e.g. it requires building inside of an
MSYS2 shell currently). Conversely, the portability of make comes at a cost of
crazy and weird hacks to work around all sorts of versions of software
everywhere, especially when it comes to the configure script and makefiles.
By rewriting this logic in one of the most robust platforms there is, Rust,
we get to assuage all of these worries for free!
* There's a standard tool to build Rust crates, Cargo, but the standard library
and compiler don't use it. This means that they cannot benefit easily from the
crates.io ecosystem, nor can the ecosystem benefit from a standard way to
build this repository itself. Moving to Cargo should help assuage both of
these needs. This has the added benefit of making the compiler more
approachable for newbies as working on the compiler will just happen to be
working on a large Cargo project, all the same standard tools and tricks will
apply.
* There's a huge amount of portability information in the main distribution, for
example around cross compiling, compiling on new OSes, etc. Pushing this logic
into standard crates (like `gcc`) enables the community to immediately benefit
from new build logic.
Despite these benefits, it's going to be a long road to actually replace our
current build system. This PR is just the beginning and doesn't implement the
full suite of functionality as the current one, but there are many more to
follow! The current implementation strategy hopes to look like:
1. Land a second build system in-tree that can be itereated on an and
contributed to. This will not be used just yet in terms of gating new commits
to the repo.
2. Over time, bring the second build system to feature parity with the old build
system, start setting up CI for both build systems.
3. At some point in the future, switch the default to the new build system, but
keep the old one around.
4. At some further point in the future, delete the entire old build system.
---
Alright, so with all that out of the way, here's some more info on this PR
itself. The inital build system here is contained in the `src/bootstrap`
directory and just adds the necessary minimum bits to bootstrap the compiler
itself. There is currently no support for building documentation, running tests,
or installing, but the implemented support is:
* Compiling LLVM with `cmake` instead of `./configure` + `make`. The LLVM
project is removing their autotools build system, so we'd have to make this
transition eventually anyway.
* Compiling compiler-rt with `cmake` as well (for the same rationale as above).
* Adding `Cargo.toml` to map out the dependency graph to all crates, and also
adding `build.rs` files where appropriate. For example `alloc_jemalloc` has a
script to build jemalloc, `flate` has a script to build `miniz.c`, `std` will
build `libbacktrace`, etc.
* Orchestrating all the calls to `cargo` to build the standard distribution,
following the normal bootstrapping process. This also tracks dependencies
between steps to ensure cross-compilation targets happen as well.
* Configuration is intended to eventually be done through a `config.toml` file,
so support is implemented for this. The most likely vector of configuration
for now, however, is likely through `config.mk` (what `./configure` emits), so
the build system currently parses this information.
There's still quite a few steps left to do, and I'll open up some follow-up
issues (as well as a tracking issue) for this migration, but hopefully this is a
great start to get going! This PR is currently tested on all the
Windows/Linux/OSX triples for x86\_64 and x86, but more portability is always
welcome!
---
Future functionality left to implement
* [ ] Re-verify that multi-host builds work
* [ ] Verify android build works
* [ ] Verify iOS build work (mostly compiler-rt)
* [ ] Verify sha256 and ideally gpg of downloaded nightly compiler and nightly rustc
* [ ] Implement testing -- this is a huge bullet point with lots of sub-bullets
* [ ] Build and generate documentation (plus the various tools we have in-tree)
* [ ] Move various src/etc scripts into Rust -- not sure how this interacts with `make` build system
* [ ] Implement `make install` - like testing this is also quite massive
* [x] Deduplicate version information with makefiles
Have all Cargo-built crates pass `--cfg cargobuild` and then add appropriate
`#[cfg]` definitions to all crates to avoid linking anything if this is passed.
This should help allow libstd to compile with both the makefiles and with Cargo.
This commit implements the `exec` function proposed in [RFC 1359][rfc] which is
a function on the `CommandExt` trait to execute all parts of a `Command::spawn`
without the `fork` on Unix. More details on the function itself can be found in
the comments in the commit.
[rfc]: https://github.com/rust-lang/rfcs/pull/1359
cc #31398
Most of this is platform-specific anyway, and we generally have to jump through
fewer hoops to do the equivalent operation on Windows. One benefit for Windows
today is that this new structure avoids an extra `DuplicateHandle` when creating
pipes. For Unix, however, the behavior should be the same.
Note that this is just a pure refactoring, no functionality was added or
removed.
On Unix we have to be careful to not call `waitpid` twice, but we don't have to
be careful on Windows due to the way process handles work there. As a result the
cached `Option<ExitStatus>` is only necessary on Unix, and it's also just an
implementation detail of the Unix module.
At the same time. also update some code in `kill` on Unix to avoid a wonky
waitpid with WNOHANG. This was added in 0e190b9a to solve #13124, but the
`signal(0)` method is not supported any more so there's no need to for this
workaround. I believe that this is no longer necessary as it's not really doing
anything.
This is a Unix-specific function which adds the ability to register a closure to
run pre-exec to configure the child process as required (note that these
closures are run post-fork).
cc #31398
* Build up the argp/envp pointers while the `Command` is being constructed
rather than only when `spawn` is called. This will allow better sharing of
code between fork/exec paths.
* Rename `child_after_fork` to `exec` and have it only perform the exec half of
the spawning. This also means the return type has changed to `io::Error`
rather than `!` to represent errors that happen.
Here's another go at adding emscripten support. This needs to wait again on new [libc definitions](https://github.com/rust-lang-nursery/libc/pull/122) landing. To get the libc definitions right I had to add support for i686-unknown-linux-musl, which are very similar to emscripten's, which are derived from arm/musl.
This branch additionally removes the makefile dependency on the `EMSCRIPTEN` environment variable by not building the unused compiler-rt.
Again, this is not sufficient for actually compiling to asmjs since it needs additional LLVM patches.
r? @alexcrichton
Backtraces, and the compilation of libbacktrace for asmjs, are disabled.
This port doesn't use jemalloc so, like pnacl, it disables jemalloc *for all targets*
in the configure file.
It disables stack protection.
These commits finish up closing out https://github.com/rust-lang/rust/issues/24237 by filling out all locations we create new file descriptors with variants that atomically create the file descriptor and set CLOEXEC where possible. Previous support for doing this in `File::open` was added in #27971 and support for `try_clone` was added in #27980. This commit fills out:
* `Socket::new` now passes `SOCK_CLOEXEC`
* `Socket::accept` now uses `accept4`
* `pipe2` is used instead of `pipe`
Unfortunately most of this support is Linux-specific, and most of it is post-2.6.18 (our oldest supported version), so all of the detection here is done dynamically. It looks like OSX does not have equivalent variants for these functions, so there's nothing more we can do there. Support for BSDs can be added over time if they also have these functions.
Closes#24237
Abort on stack overflow instead of re-raising SIGSEGV
We use guard pages that cause the process to abort to protect against
undefined behavior in the event of stack overflow. We have a handler
that catches segfaults, prints out an error message if the segfault was
due to a stack overflow, then unregisters itself and returns to allow
the signal to be re-raised and kill the process.
This caused some confusion, as it was unexpected that safe code would be
able to cause a segfault, while it's easy to overflow the stack in safe
code. To avoid this confusion, when we detect a segfault in the guard
page, abort instead of the previous behavior of re-raising SIGSEGV.
To test this, we need to adapt the tests for segfault to actually check
the exit status. Doing so revealed that the existing test for segfault
behavior was actually invalid; LLVM optimizes the explicit null pointer
reference down to an illegal instruction, so the program aborts with
SIGILL instead of SIGSEGV and the test didn't actually trigger the
signal handler at all. Use a C helper function to get a null pointer
that LLVM can't optimize away, so we get our segfault instead.
This is a [breaking-change] if anyone is relying on the exact signal
raised to kill a process on stack overflow.
Closes#31273
We use guard pages that cause the process to abort to protect against
undefined behavior in the event of stack overflow. We have a handler
that catches segfaults, prints out an error message if the segfault was
due to a stack overflow, then unregisters itself and returns to allow
the signal to be re-raised and kill the process.
This caused some confusion, as it was unexpected that safe code would be
able to cause a segfault, while it's easy to overflow the stack in safe
code. To avoid this confusion, when we detect a segfault in the guard
page, abort instead of the previous behavior of re-raising the SIGSEGV.
To test this, we need to adapt the tests for segfault to actually check
the exit status. Doing so revealed that the existing test for segfault
behavior was actually invalid; LLVM optimizes the explicit null pointer
reference down to an illegal instruction, so the program aborts with
SIGILL instead of SIGSEGV and the test didn't actually trigger the
signal handler at all. Use a C helper function to get a null pointer
that LLVM can't optimize away, so we get our segfault instead.
This is a [breaking-change] if anyone is relying on the exact signal
raised to kill a process on stack overflow.
Closes#31273
This commit attempts to use the `pipe2` syscall on Linux to atomically set the
CLOEXEC flag for pipes created. Unfortunately this was added in 2.6.27 so we
have to dynamically determine whether we can use it or not.
This commit also updates the `fds-are-cloexec.rs` test to test stdio handles for
spawned processes as well.
This is necessary to atomically accept a socket and set the CLOEXEC flag at the
same time. Support only appeared in Linux 2.6.28 so we have to dynamically
determine which syscall we're supposed to call in this case.
Right now we only attempt to call one symbol which my not exist everywhere,
__pthread_get_minstack, but this pattern will come up more often as we start to
bind newer functionality of systems like Linux.
Take a similar strategy as the Windows implementation where we use `dlopen` to
lookup whether a symbol exists or not.
This commit adds support for creating sockets with the `SOCK_CLOEXEC` flag.
Support for this flag was added in Linux 2.6.27, however, and support does not
exist on platforms other than Linux. For this reason we still have the same
fallback as before but just special case Linux if we can.
Similar to the previous commit, if `F_DUPFD_CLOEXEC` succeeds then there's no
need for us to then call `set_cloexec` on platforms other than Linux. The bug
mentioned of kernels not actually setting the `CLOEXEC` flag has only been
repored on Linux, not elsewhere.
On Linux we have to do this for binary compatibility with 2.6.18, but for other
OSes (e.g. OSX/BSDs/etc) they all support this flag so we don't need to pass it.
These accessors are used to get at the last modification, last access, and
creation time of the underlying file. Currently not all platforms provide the
creation time, so that currently returns `Option`.
- use `symlink_file` and `symlink_dir` instead of the old `soft_link`
- create a junction instead of a directory symlink for testing recursive_rmdir (as it causes the
same troubles, but can be created by users without `SeCreateSymbolicLinkPrivilege`)
- `remove_dir_all` was unable to remove directory symlinks and junctions
- only run tests that create symlinks if we have the right permissions.
- rename `Path2` to `Path`
- remove the global `#[allow(deprecated)]` and outdated comments
- After factoring out `create_junction()` from the test `directory_junctions_are_directories` and
removing needlessly complex code, what I was left with was:
```
#[test]
#[cfg(windows)]
fn directory_junctions_are_directories() {
use sys::fs::create_junction;
let tmpdir = tmpdir();
let foo = tmpdir.join("foo");
let bar = tmpdir.join("bar");
fs::create_dir(&foo).unwrap();
check!(create_junction(&foo, &bar));
assert!(bar.metadata().unwrap().is_dir());
}
```
It test whether a junction is a directory instead of a reparse point. But it actually test the
target of the junction (which is a directory if it exists) instead of the junction itself, which
should always be a symlink. So this test is invalid, and I expect it only exists because the
author was suprised by it. So I removed it.
Some things that do not yet work right:
- relative symlinks do not accept forward slashes
- the conversion of paths for `create_junction` is hacky
- `remove_dir_all` now messes with the internal data of `FileAttr` to be able to remove symlinks.
We should add some method like `is_symlink_dir()` to it, so code outside the standard library
can see the difference between file and directory symlinks too.
I have it set as stable right now under the rationale that it's extending an existing, stable API to another type in the "obvious" way.
r? @alexcrichton
cc @reem
This pull request adds support for [Illumos](http://illumos.org/)-based operating systems: SmartOS, OpenIndiana, and others. For now it's x86-64 only, as I'm not sure if 32-bit installations are widespread. This PR is based on #28589 by @potatosalad, and also closes#21000, #25845, and #25846.
Required changes in libc are already merged: https://github.com/rust-lang-nursery/libc/pull/138
Here's a snapshot required to build a stage0 compiler:
https://s3-eu-west-1.amazonaws.com/nbaksalyar/rustc-sunos-snapshot.tar.gz
It passes all checks from `make check`.
There are some changes I'm not quite sure about, e.g. macro usage in `src/libstd/num/f64.rs` and `DirEntry` structure in `src/libstd/sys/unix/fs.rs`, so any comments on how to rewrite it better would be greatly appreciated.
Also, LLVM configure script might need to be patched to build it successfully, or a pre-built libLLVM should be used. Some details can be found here: https://llvm.org/bugs/show_bug.cgi?id=25409
Thanks!
r? @brson
Looks like the rumprun build has bitrotted over time, so this includes some libc
fixes and some various libstd fixes which gets it back to bootstrapping.
Tracking issue: #30014
This implements the RFC and makes a few other changes.
I have added a few extra tests, and made the Windows and
Unix code as similar as possible.
Part of the RFC mentions the unstable OpenOptionsExt trait
on Windows (see #27720). I have added a few extra methods
to future-proof it for CreateFile2.
This commit stabilizes and deprecates the FCP (final comment period) APIs for
the upcoming 1.7 beta release. The specific APIs which changed were:
Stabilized
* `Path::strip_prefix` (renamed from `relative_from`)
* `path::StripPrefixError` (new error type returned from `strip_prefix`)
* `Ipv4Addr::is_loopback`
* `Ipv4Addr::is_private`
* `Ipv4Addr::is_link_local`
* `Ipv4Addr::is_multicast`
* `Ipv4Addr::is_broadcast`
* `Ipv4Addr::is_documentation`
* `Ipv6Addr::is_unspecified`
* `Ipv6Addr::is_loopback`
* `Ipv6Addr::is_unique_local`
* `Ipv6Addr::is_multicast`
* `Vec::as_slice`
* `Vec::as_mut_slice`
* `String::as_str`
* `String::as_mut_str`
* `<[T]>::clone_from_slice` - the `usize` return value is removed
* `<[T]>::sort_by_key`
* `i32::checked_rem` (and other signed types)
* `i32::checked_neg` (and other signed types)
* `i32::checked_shl` (and other signed types)
* `i32::checked_shr` (and other signed types)
* `i32::saturating_mul` (and other signed types)
* `i32::overflowing_add` (and other signed types)
* `i32::overflowing_sub` (and other signed types)
* `i32::overflowing_mul` (and other signed types)
* `i32::overflowing_div` (and other signed types)
* `i32::overflowing_rem` (and other signed types)
* `i32::overflowing_neg` (and other signed types)
* `i32::overflowing_shl` (and other signed types)
* `i32::overflowing_shr` (and other signed types)
* `u32::checked_rem` (and other unsigned types)
* `u32::checked_neg` (and other unsigned types)
* `u32::checked_shl` (and other unsigned types)
* `u32::saturating_mul` (and other unsigned types)
* `u32::overflowing_add` (and other unsigned types)
* `u32::overflowing_sub` (and other unsigned types)
* `u32::overflowing_mul` (and other unsigned types)
* `u32::overflowing_div` (and other unsigned types)
* `u32::overflowing_rem` (and other unsigned types)
* `u32::overflowing_neg` (and other unsigned types)
* `u32::overflowing_shl` (and other unsigned types)
* `u32::overflowing_shr` (and other unsigned types)
* `ffi::IntoStringError`
* `CString::into_string`
* `CString::into_bytes`
* `CString::into_bytes_with_nul`
* `From<CString> for Vec<u8>`
* `From<CString> for Vec<u8>`
* `IntoStringError::into_cstring`
* `IntoStringError::utf8_error`
* `Error for IntoStringError`
Deprecated
* `Path::relative_from` - renamed to `strip_prefix`
* `Path::prefix` - use `components().next()` instead
* `os::unix::fs` constants - moved to the `libc` crate
* `fmt::{radix, Radix, RadixFmt}` - not used enough to stabilize
* `IntoCow` - conflicts with `Into` and may come back later
* `i32::{BITS, BYTES}` (and other integers) - not pulling their weight
* `DebugTuple::formatter` - will be removed
* `sync::Semaphore` - not used enough and confused with system semaphores
Closes#23284
cc #27709 (still lots more methods though)
Closes#27712Closes#27722Closes#27728Closes#27735Closes#27729Closes#27755Closes#27782Closes#27798
The following PR updates libc version to latest commits for correctly support openbsd.
It corrects several points in rustc to be compatible with libc changes.
r? @alexcrichton
This adds the ability to override the default OOM behavior by setting a handler function. This is used by libstd to print a message when running out of memory instead of crashing with an obscure "illegal hardware instruction" error (at least on Linux).
Fixes#14674
Tracking issue: #30014
This implements the RFC and makes a few other changes.
I have added a few extra tests, and made the Windows and
Unix code as similar as possible.
Part of the RFC mentions the unstable OpenOptionsExt trait
on Windows (see #27720). I have added a few extra methods
to future-proof it for CreateFile2.
`siginfo_si_addr()` function is used once, and the returned value is
casted to `usize`. So make the function returns a `usize`.
it simplifies OpenBSD case, where the return type wouldn't be a `*mut
libc::c_void` but a `*mut libc::c_char`.
* If the requested descriptors to inherit are stdio descriptors there
are situations where they will not be set correctly
* Example: parent's stdout --> child's stderr
parent's stderr --> child's stdout
* Solution: if the requested descriptors for the child are stdio
descriptors, `dup` them before overwriting the child's stdio
This commit is the standard API stabilization commit for the 1.6 release cycle.
The list of issues and APIs below have all been through their cycle-long FCP and
the libs team decisions are listed below
Stabilized APIs
* `Read::read_exact`
* `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`)
* libcore -- this was a bit of a nuanced stabilization, the crate itself is now
marked as `#[stable]` and the methods appearing via traits for primitives like
`char` and `str` are now also marked as stable. Note that the extension traits
themeselves are marked as unstable as they're imported via the prelude. The
`try!` macro was also moved from the standard library into libcore to have the
same interface. Otherwise the functions all have copied stability from the
standard library now.
* `fs::DirBuilder`
* `fs::DirBuilder::new`
* `fs::DirBuilder::recursive`
* `fs::DirBuilder::create`
* `os::unix::fs::DirBuilderExt`
* `os::unix::fs::DirBuilderExt::mode`
* `vec::Drain`
* `vec::Vec::drain`
* `string::Drain`
* `string::String::drain`
* `vec_deque::Drain`
* `vec_deque::VecDeque::drain`
* `collections::hash_map::Drain`
* `collections::hash_map::HashMap::drain`
* `collections::hash_set::Drain`
* `collections::hash_set::HashSet::drain`
* `collections::binary_heap::Drain`
* `collections::binary_heap::BinaryHeap::drain`
* `Vec::extend_from_slice` (renamed from `push_all`)
* `Mutex::get_mut`
* `Mutex::into_inner`
* `RwLock::get_mut`
* `RwLock::into_inner`
* `Iterator::min_by_key` (renamed from `min_by`)
* `Iterator::max_by_key` (renamed from `max_by`)
Deprecated APIs
* `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`)
* `OsString::from_bytes`
* `OsStr::to_cstring`
* `OsStr::to_bytes`
* `fs::walk_dir` and `fs::WalkDir`
* `path::Components::peek`
* `slice::bytes::MutableByteVector`
* `slice::bytes::copy_memory`
* `Vec::push_all` (renamed to `extend_from_slice`)
* `Duration::span`
* `IpAddr`
* `SocketAddr::ip`
* `Read::tee`
* `io::Tee`
* `Write::broadcast`
* `io::Broadcast`
* `Iterator::min_by` (renamed to `min_by_key`)
* `Iterator::max_by` (renamed to `max_by_key`)
* `net::lookup_addr`
New APIs (still unstable)
* `<[T]>::sort_by_key` (added to mirror `min_by_key`)
Closes#27585Closes#27704Closes#27707Closes#27710Closes#27711Closes#27727Closes#27740Closes#27744Closes#27799Closes#27801
cc #27801 (doesn't close as `Chars` is still unstable)
Closes#28968
This commit is the standard API stabilization commit for the 1.6 release cycle.
The list of issues and APIs below have all been through their cycle-long FCP and
the libs team decisions are listed below
Stabilized APIs
* `Read::read_exact`
* `ErrorKind::UnexpectedEof` (renamed from `UnexpectedEOF`)
* libcore -- this was a bit of a nuanced stabilization, the crate itself is now
marked as `#[stable]` and the methods appearing via traits for primitives like
`char` and `str` are now also marked as stable. Note that the extension traits
themeselves are marked as unstable as they're imported via the prelude. The
`try!` macro was also moved from the standard library into libcore to have the
same interface. Otherwise the functions all have copied stability from the
standard library now.
* The `#![no_std]` attribute
* `fs::DirBuilder`
* `fs::DirBuilder::new`
* `fs::DirBuilder::recursive`
* `fs::DirBuilder::create`
* `os::unix::fs::DirBuilderExt`
* `os::unix::fs::DirBuilderExt::mode`
* `vec::Drain`
* `vec::Vec::drain`
* `string::Drain`
* `string::String::drain`
* `vec_deque::Drain`
* `vec_deque::VecDeque::drain`
* `collections::hash_map::Drain`
* `collections::hash_map::HashMap::drain`
* `collections::hash_set::Drain`
* `collections::hash_set::HashSet::drain`
* `collections::binary_heap::Drain`
* `collections::binary_heap::BinaryHeap::drain`
* `Vec::extend_from_slice` (renamed from `push_all`)
* `Mutex::get_mut`
* `Mutex::into_inner`
* `RwLock::get_mut`
* `RwLock::into_inner`
* `Iterator::min_by_key` (renamed from `min_by`)
* `Iterator::max_by_key` (renamed from `max_by`)
Deprecated APIs
* `ErrorKind::UnexpectedEOF` (renamed to `UnexpectedEof`)
* `OsString::from_bytes`
* `OsStr::to_cstring`
* `OsStr::to_bytes`
* `fs::walk_dir` and `fs::WalkDir`
* `path::Components::peek`
* `slice::bytes::MutableByteVector`
* `slice::bytes::copy_memory`
* `Vec::push_all` (renamed to `extend_from_slice`)
* `Duration::span`
* `IpAddr`
* `SocketAddr::ip`
* `Read::tee`
* `io::Tee`
* `Write::broadcast`
* `io::Broadcast`
* `Iterator::min_by` (renamed to `min_by_key`)
* `Iterator::max_by` (renamed to `max_by_key`)
* `net::lookup_addr`
New APIs (still unstable)
* `<[T]>::sort_by_key` (added to mirror `min_by_key`)
Closes#27585Closes#27704Closes#27707Closes#27710Closes#27711Closes#27727Closes#27740Closes#27744Closes#27799Closes#27801
cc #27801 (doesn't close as `Chars` is still unstable)
Closes#28968
Allows a `HANDLE` to be extracted from a `JoinHandle` on Windows.
Allows a `pthread_t` to be extracted from a `JoinHandle` everywhere else.
Because https://github.com/rust-lang/rust/pull/29461 was closed.
r? @alexcrichton
This commit is an implementation of [RFC 1288][rfc] which adds two new unstable
types to the `std::time` module. The `Instant` type is used to represent
measurements of a monotonically increasing clock suitable for measuring time
withing a process for operations such as benchmarks or just the elapsed time to
do something. An `Instant` favors panicking when bugs are found as the bugs are
programmer errors rather than typical errors that can be encountered.
[rfc]: https://github.com/rust-lang/rfcs/pull/1288
The `SystemTime` type is used to represent a system timestamp and is not
monotonic. Very few guarantees are provided about this measurement of the system
clock, but a fixed point in time (`UNIX_EPOCH`) is provided to learn about the
relative distance from this point for any particular time stamp.
This PR takes the same implementation strategy as the `time` crate on crates.io,
namely:
| Platform | Instant | SystemTime |
|------------|--------------------------|--------------------------|
| Windows | QueryPerformanceCounter | GetSystemTimeAsFileTime |
| OSX | mach_absolute_time | gettimeofday |
| Unix | CLOCK_MONOTONIC | CLOCK_REALTIME |
These implementations can perhaps be refined over time, but they currently
satisfy the requirements of the `Instant` and `SystemTime` types while also
being portable across implementations and revisions of each platform.
Otherwise, the iterator and the functions for getting specific
environment variables might disagree, for environments like
FOOBAR
Variable names starting with equals sign are OK:
glibc only interprets equals signs not in the first position as
separators between variable name and variable value. Instead of skipping
them entirely, a leading equals sign is interpreted to be part of the
variable name.
* Delete `sys::unix::{c, sync}` as these are now all folded into libc itself
* Update all references to use `libc` as a result.
* Update all references to the new flat namespace.
* Moves all windows bindings into sys::c
* Store the native representation directly in the `ExitStatus` structure instead
of a "parsed version" (mostly for Unix).
* On Windows, be more robust against processes exiting with the status of 259.
Unfortunately this exit code corresponds to `STILL_ACTIVE`, causing libstd to
think the process was still alive, causing an infinite loop. Instead the loop
is removed altogether and `WaitForSingleObject` is used to wait for the
process to exit.
* Store the native representation directly in the `ExitStatus` structure instead
of a "parsed version" (mostly for Unix).
* On Windows, be more robust against processes exiting with the status of 259.
Unfortunately this exit code corresponds to `STILL_ACTIVE`, causing libstd to
think the process was still alive, causing an infinite loop. Instead the loop
is removed altogether and `WaitForSingleObject` is used to wait for the
process to exit.
As discovered in #29298, `env::set_var("", "")` will panic, but it turns out
that it *also* deadlocks on Unix systems. This happens because if a panic
happens while holding the environment lock, we then go try to read
RUST_BACKTRACE, grabbing the environment lock, causing a deadlock.
Specifically, the changes made here are:
* The environment lock is pushed into `std::sys` instead of `std::env`. This
also only puts it in the Unix implementation, not Windows where the functions
are already threadsafe.
* The `std::sys` implementation now returns `io::Result` so panics are
explicitly at the `std::env` level.
As discovered in #29298, `env::set_var("", "")` will panic, but it turns out
that it *also* deadlocks on Unix systems. This happens because if a panic
happens while holding the environment lock, we then go try to read
RUST_BACKTRACE, grabbing the environment lock, causing a deadlock.
Specifically, the changes made here are:
* The environment lock is pushed into `std::sys` instead of `std::env`. This
also only puts it in the Unix implementation, not Windows where the functions
are already threadsafe.
* The `std::sys` implementation now returns `io::Result` so panics are
explicitly at the `std::env` level. The panic messages have also been improved
in these situations.
This commit stabilizes and deprecates library APIs whose FCP has closed in the
last cycle, specifically:
Stabilized APIs:
* `fs::canonicalize`
* `Path::{metadata, symlink_metadata, canonicalize, read_link, read_dir, exists,
is_file, is_dir}` - all moved to inherent methods from the `PathExt` trait.
* `Formatter::fill`
* `Formatter::width`
* `Formatter::precision`
* `Formatter::sign_plus`
* `Formatter::sign_minus`
* `Formatter::alternate`
* `Formatter::sign_aware_zero_pad`
* `string::ParseError`
* `Utf8Error::valid_up_to`
* `Iterator::{cmp, partial_cmp, eq, ne, lt, le, gt, ge}`
* `<[T]>::split_{first,last}{,_mut}`
* `Condvar::wait_timeout` - note that `wait_timeout_ms` is not yet deprecated
but will be once 1.5 is released.
* `str::{R,}MatchIndices`
* `str::{r,}match_indices`
* `char::from_u32_unchecked`
* `VecDeque::insert`
* `VecDeque::shrink_to_fit`
* `VecDeque::as_slices`
* `VecDeque::as_mut_slices`
* `VecDeque::swap_remove_front` - (renamed from `swap_front_remove`)
* `VecDeque::swap_remove_back` - (renamed from `swap_back_remove`)
* `Vec::resize`
* `str::slice_mut_unchecked`
* `FileTypeExt`
* `FileTypeExt::{is_block_device, is_char_device, is_fifo, is_socket}`
* `BinaryHeap::from` - `from_vec` deprecated in favor of this
* `BinaryHeap::into_vec` - plus a `Into` impl
* `BinaryHeap::into_sorted_vec`
Deprecated APIs
* `slice::ref_slice`
* `slice::mut_ref_slice`
* `iter::{range_inclusive, RangeInclusive}`
* `std::dynamic_lib`
Closes#27706Closes#27725
cc #27726 (align not stabilized yet)
Closes#27734Closes#27737Closes#27742Closes#27743Closes#27772Closes#27774Closes#27777Closes#27781
cc #27788 (a few remaining methods though)
Closes#27790Closes#27793Closes#27796Closes#27810
cc #28147 (not all parts stabilized)
This commit adds `#[derive(Clone)]` to `std::fs::Metadata`, making that struct cloneable. Although the exact contents of that struct differ between OSes, they all have it contain only value types, meaning that the data can be re-used without repercussions.
It also adds `#[derive(Clone)]` to every type used by that struct across all OSes, including the various Unix `stat` structs and Windows's `WIN32_FILE_ATTRIBUTE_DATA`.
This stems from my comment here: https://github.com/rust-lang/rfcs/issues/939#issuecomment-140524439
This commit adds #[derive(Clone)] to std::fs::Metadata, making that struct
cloneable. Although the exact contents of that struct differ between OSes,
they all have it contain only value types, meaning that the data can be re-used without repercussions.
It also adds #[derive(Clone)] to every type used by that struct across all
OSes, including the various Unix `stat` structs and Windows's
`WIN32_FILE_ATTRIBUTE_DATA`.
Travis CI has new infrastructure using the Google Compute Engine which has both
faster CPUs and more memory, and we've been encouraged to switch as it should
help our build times! The only downside currently, however, is that IPv6 is
disabled, causing a number of standard library tests to fail.
Consequently this commit tweaks our travis config in a few ways:
* ccache is disabled as it's not working on GCE just yet
* Docker is used to run tests inside which reportedly will get IPv6 working
* A system LLVM installation is used instead of building LLVM itself. This is
primarily done to reduce build times, but we want automation for this sort of
behavior anyway and we can extend this in the future with building from source
as well if needed.
* gcc-specific logic is removed as the docker image for Ubuntu gives us a
recent-enough gcc by default.
For most parts, rumprun currently looks like NetBSD, as they share the same
libc and drivers. However, being a unikernel, rumprun does not support
process management, signals or virtual memory, so related functions
might fail at runtime. Stack guards are disabled exactly for this reason.
Code for rumprun is always cross-compiled, it uses always static
linking and needs a custom linker.
This commit does some refactoring to make almost all of the `std::rt` private.
Specifically, the following items are no longer part of its API:
* DEFAULT_ERROR_CODE
* backtrace
* unwind
* args
* at_exit
* cleanup
* heap (this is just alloc::heap)
* min_stack
* util
The module is now tagged as `#[doc(hidden)]` as the only purpose it's serve is
an entry point for the `panic!` macro via the `begin_unwind` and
`begin_unwind_fmt` reexports.
Fix formatting
Remove unused imports
Refactor
Fix msvc build
Fix line lengths
Formatting
Enable backtrace tests
Fix using directive on mac
pwd info
Work-around buildbot PWD bug, and fix libbacktrace configuration
Use alternative to `env -u` which is not supported on bitrig
Disable tests on 32-bit windows gnu
For Bitrig, NetBSD and OpenBSD the constant was incorrectly in posix01, when
it's actually posix08, so we move it. This is a [breaking-change], but we
already had one in #27930.
Fix NetBSD's F_DUPFD_CLOEXEC constant.
For a similar feature detection, see this musl thread:
http://comments.gmane.org/gmane.linux.lib.musl.general/2963
This assumes that an int literal has type `c_int` for varidic functions.
Make `std::sys::os::getcwd` call `Vec::reserve(1)` followed by
`Vec::set_len` to double the buffer. This is to align with other similar
functions, such as:
- `std::sys_common::io::read_to_end_uninitialized`
- `std::sys::fs::readlink`
Also, reduce the initial buffer size from 2048 to 512. The previous size was
introduced with 4bc26ce in 2013, but it seems a bit excessive. This is
probably because buffer doubling was not implemented back then.
- Rewrite `std::sys::fs::readlink` not to rely on `PATH_MAX`
It currently has the following problems:
1. It uses `_PC_NAME_MAX` to query the maximum length of a file path in
the underlying system. However, the meaning of the constant is the
maximum length of *a path component*, not a full path. The correct
constant should be `_PC_PATH_MAX`.
2. `pathconf` *may* fail if the referred file does not exist. This can
be problematic if the file which the symbolic link points to does not
exist, but the link itself does exist. In this case, the current
implementation resorts to the hard-coded value of `1024`, which is not
ideal.
3. There may exist a platform where there is no limit on file path
lengths in general. That's the reaon why GNU Hurd doesn't define
`PATH_MAX` at all, in addition to having `pathconf` always returning
`-1`. In these platforms, the content of the symbolic link can be
silently truncated if the length exceeds the hard-coded limit mentioned
above.
4. The value obtained by `pathconf` may be outdated at the point of
actually calling `readlink`. This is inherently racy.
This commit introduces a loop that gradually increases the length of the
buffer passed to `readlink`, eliminating the need of `pathconf`.
- Remove the arbitrary memory limit of `std::sys::fs::realpath`
As per POSIX 2013, `realpath` will return a malloc'ed buffer if the
second argument is a null pointer.[1]
[1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/realpath.html
- Comment on functions that are still using `PATH_MAX`
There are some functions that only work in terms of `PATH_MAX`, such as
`F_GETPATH` in OS X. Comments on them for posterity.
This commit removes all unstable and deprecated functions in the standard
library. A release was recently cut (1.3) which makes this a good time for some
spring cleaning of the deprecated functions.
This commit stabilizes the `std::time` module and the `Duration` type.
`Duration::span` remains unstable, and the `Display` implementation for
`Duration` has been removed as it is still being reworked and all trait
implementations for stable types are de facto stable.
This is a [breaking-change] to those using `Duration`'s `Display`
implementation.
I'm opening this PR as a platform for discussion - there may be some method renaming to do as part of the stabilization process.
This commit stabilizes the `std::time` module and the `Duration` type.
`Duration::span` remains unstable, and the `Display` implementation for
`Duration` has been removed as it is still being reworked and all trait
implementations for stable types are de facto stable.
This is a [breaking-change] to those using `Duration`'s `Display`
implementation.
This commit removes all morestack support from the compiler which entails:
* Segmented stacks are no longer emitted in codegen.
* We no longer build or distribute libmorestack.a
* The `stack_exhausted` lang item is no longer required
The only current use of the segmented stack support in LLVM is to detect stack
overflow. This is no longer really required, however, because we already have
guard pages for all threads and registered signal handlers watching for a
segfault on those pages (to print out a stack overflow message). Additionally,
major platforms (aka Windows) already don't use morestack.
This means that Rust is by default less likely to catch stack overflows because
if a function takes up more than one page of stack space it won't hit the guard
page. This is what the purpose of morestack was (to catch this case), but it's
better served with stack probes which have more cross platform support and no
runtime support necessary. Until LLVM supports this for all platform it looks
like morestack isn't really buying us much.
cc #16012 (still need stack probes)
Closes#26458 (a drive-by fix to help diagnostics on stack overflow)
r? @brson
This commit removes all morestack support from the compiler which entails:
* Segmented stacks are no longer emitted in codegen.
* We no longer build or distribute libmorestack.a
* The `stack_exhausted` lang item is no longer required
The only current use of the segmented stack support in LLVM is to detect stack
overflow. This is no longer really required, however, because we already have
guard pages for all threads and registered signal handlers watching for a
segfault on those pages (to print out a stack overflow message). Additionally,
major platforms (aka Windows) already don't use morestack.
This means that Rust is by default less likely to catch stack overflows because
if a function takes up more than one page of stack space it won't hit the guard
page. This is what the purpose of morestack was (to catch this case), but it's
better served with stack probes which have more cross platform support and no
runtime support necessary. Until LLVM supports this for all platform it looks
like morestack isn't really buying us much.
cc #16012 (still need stack probes)
Closes#26458 (a drive-by fix to help diagnostics on stack overflow)
The replacements are functions that usually use a single `mem::transmute` in their body and restrict input and output via more concrete types than `T` and `U`. Worth noting are the `transmute` functions for slices and the `from_utf8*` family for mutable slices. Additionally, `mem::transmute` was often used for casting raw pointers, when you can already cast raw pointers just fine with `as`.
This builds upon #27233.
The investigation into #14232 discovered that it's possible that signal delivery
to a newly spawned process is racy on OSX. This test has been failing spuriously
on the OSX bots for some time now, so ignore it as we don't currently know a
solution and it looks like it may be out of our control.
The replacements are functions that usually use a single `mem::transmute` in
their body and restrict input and output via more concrete types than `T` and
`U`. Worth noting are the `transmute` functions for slices and the `from_utf8*`
family for mutable slices. Additionally, `mem::transmute` was often used for
casting raw pointers, when you can already cast raw pointers just fine with
`as`.
This commit is an implementation of [RFC 1184][rfc] which tweaks the behavior of
the `#![no_std]` attribute and adds a new `#![no_core]` attribute. The
`#![no_std]` attribute now injects `extern crate core` at the top of the crate
as well as the libcore prelude into all modules (in the same manner as the
standard library's prelude). The `#![no_core]` attribute disables both std and
core injection.
[rfc]: https://github.com/rust-lang/rfcs/pull/1184Closes#27394
This commit removes the injection of `std::env::args()` from `--test` expanded
code, relying on the test runner itself to call this funciton. This is more
hygienic because we can't assume that `std` exists at the top layer all the
time, and it meaks the injected test module entirely self contained.
The investigation into #14232 discovered that it's possible that signal delivery
to a newly spawned process is racy on OSX. This test has been failing spuriously
on the OSX bots for some time now, so ignore it as we don't currently know a
solution and it looks like it may be out of our control.
This commit is an implementation of [RFC 1184][rfc] which tweaks the behavior of
the `#![no_std]` attribute and adds a new `#![no_core]` attribute. The
`#![no_std]` attribute now injects `extern crate core` at the top of the crate
as well as the libcore prelude into all modules (in the same manner as the
standard library's prelude). The `#![no_core]` attribute disables both std and
core injection.
[rfc]: https://github.com/rust-lang/rfcs/pull/1184
The API we're calling requires us to pass an absolute point in time as an
argument (`pthread_cond_timedwait`) so we call `gettimeofday` ahead of time to
then add the specified duration to. Unfortuantely the current "add the duration"
logic forgot to take into account the current time's sub-second precision (e.g.
the `tv_usec` field was ignored), causing sub-second duration waits to return
spuriously.
This can fail on linux for various reasons, such as the /proc filesystem not
being mounted. There are already many cases where we can't set up stack guards,
so just don't worry about this case and communicate that no guard was enabled.
I've confirmed that this allows the compiler to run in a chroot without /proc
mounted.
Closes#22642
This can fail on linux for various reasons, such as the /proc filesystem not
being mounted. There are already many cases where we can't set up stack guards,
so just don't worry about this case and communicate that no guard was enabled.
I've confirmed that this allows the compiler to run in a chroot without /proc
mounted.
Closes#22642
This commit is an implementation of [RFC 1174][rfc] which adds three new traits
to the standard library:
* `IntoRawFd` - implemented on Unix for all I/O types (files, sockets, etc)
* `IntoRawHandle` - implemented on Windows for files, processes, etc
* `IntoRawSocket` - implemented on Windows for networking types
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1174-into-raw-fd-socket-handle-traits.mdCloses#27062
This makes `Debug` for `File` show the file path and access mode of the file on OS X, just like on Linux.
I'd be happy about any feedback how to make this code better. In particular, I'm not sure how to handle the buffer passed to `fnctl`. This way works, but it feels a bit cumbersome. `fcntl` unfortunately doesn't return the length of the path.
signal(), sigemptyset(), and sigaddset() are only available as inline
functions until Android API 21. liblibc already handles signal()
appropriately, so drop it from c.rs; translate sigemptyset() and
sigaddset() (which is only used in a test) by hand from the C inlines.
We probably want to revert this commit when we bump Android API level.
Make sure that child processes don't get affected by libstd's desire to
ignore SIGPIPE, nor a third-party library's signal mask (which is needed
to use either a signal-handling thread correctly or to use signalfd /
kqueue correctly).
Both c.rs and stack_overflow.rs had bindings of libc's signal-handling
routines. It looks like the split dated from #16388, when (what is now)
c.rs was in libnative but not libgreen. Nobody is currently using the
c.rs bindings, but they're a bit more accurate in some places.
Move everything to c.rs (since I'll need signal handling in process.rs,
and we should avoid duplication), clean up the bindings, and manually
double-check everything against the relevant system headers (fixing a
few things in the process).
It looks like a lot of this dated to previous incarnations of the io
module, etc., and went unused in the reworking leading up to 1.0. Remove
everything we're not actively using (except for signal handling, which
will be reworked in the next commit).
The execv family of functions do not modify their arguments, so they do
not need mutable pointers. The C prototypes take a constant array of
mutable C-strings, but that's a legacy quirk from before C had const
(since C string literals have type `char *`). The Rust prototypes had
`*mut` in the wrong place, anyway: to match the C prototypes, it should
have been `*const *mut c_char`. But it is safe to pass constant strings
(like string literals) to these functions.
getopt is a special case, since GNU getopt modifies its arguments
despite the `const` claim in the prototype. It is apparently only
well-defined to call getopt on the actual argc and argv parameters
passed to main, anyway. Change it to take `*mut *mut c_char` for an
attempt at safety, but probably nobody should be using it from Rust,
since there's no great way to get at the parameters as passed to main.
Also fix the one caller of execvp in libstd, which now no longer needs
an unsafe cast.
Fixes#16290.
Closes#25977
The various `stdfoo_raw` methods in std::io now return `io::Result`s,
since they may not exist on Windows. They will always return `Ok` on
Unix-like platforms.
[breaking-change]
Closes#25977
The various `stdfoo_raw` methods in std::io now return `io::Result`s,
since they may not exist on Windows. They will always return `Ok` on
Unix-like platforms.
[breaking-change]
This commit stabilizes the following APIs, slating them all to be cherry-picked
into the 1.1 release.
* fs::FileType (and transitively the derived trait implementations)
* fs::Metadata::file_type
* fs::FileType::is_dir
* fs::FileType::is_file
* fs::FileType::is_symlink
* fs::DirEntry::metadata
* fs::DirEntry::file_type
* fs::DirEntry::file_name
* fs::set_permissions
* fs::symlink_metadata
* os::raw::{self, *}
* os::{android, bitrig, linux, ...}::raw::{self, *}
* os::{android, bitrig, linux, ...}::fs::MetadataExt
* os::{android, bitrig, linux, ...}::fs::MetadataExt::as_raw_stat
* os::unix::fs::PermissionsExt
* os::unix::fs::PermissionsExt::mode
* os::unix::fs::PermissionsExt::set_mode
* os::unix::fs::PermissionsExt::from_mode
* os::unix::fs::OpenOptionsExt
* os::unix::fs::OpenOptionsExt::mode
* os::unix::fs::DirEntryExt
* os::unix::fs::DirEntryExt::ino
* os::windows::fs::MetadataExt
* os::windows::fs::MetadataExt::file_attributes
* os::windows::fs::MetadataExt::creation_time
* os::windows::fs::MetadataExt::last_access_time
* os::windows::fs::MetadataExt::last_write_time
* os::windows::fs::MetadataExt::file_size
The `os::unix::fs::Metadata` structure was also removed entirely, moving all of
its associated methods into the `os::unix::fs::MetadataExt` trait instead. The
methods are all marked as `#[stable]` still.
As some minor cleanup, some deprecated and unstable fs apis were also removed:
* File::path
* Metadata::accessed
* Metadata::modified
Features that were explicitly left unstable include:
* fs::WalkDir - the semantics of this were not considered in the recent fs
expansion RFC.
* fs::DirBuilder - it's still not 100% clear if the naming is right here and if
the set of functionality exposed is appropriate.
* fs::canonicalize - the implementation on Windows here is specifically in
question as it always returns a verbatim path. Additionally the Unix
implementation is susceptible to buffer overflows on long paths unfortunately.
* fs::PathExt - as this is just a convenience trait, it is not stabilized at
this time.
* fs::set_file_times - this funciton is still waiting on a time abstraction.
* Slate these features to be stable in 1.2 instead of 1.1 (not being backported)
* Have the `FromRawFd` implementations follow the contract of the `FromRawFd`
trait by taking ownership of the primitive specified.
* Refactor the implementations slightly to remove the `unreachable!` blocks as
well as separating the stdio representation of `std::process` from
`std::sys::process`.
This commit stabilizes the following APIs, slating them all to be cherry-picked
into the 1.1 release.
* fs::FileType (and transitively the derived trait implementations)
* fs::Metadata::file_type
* fs::FileType::is_dir
* fs::FileType::is_file
* fs::FileType::is_symlink
* fs::DirEntry::metadata
* fs::DirEntry::file_type
* fs::DirEntry::file_name
* fs::set_permissions
* fs::symlink_metadata
* os::raw::{self, *}
* os::{android, bitrig, linux, ...}::raw::{self, *}
* os::{android, bitrig, linux, ...}::fs::MetadataExt
* os::{android, bitrig, linux, ...}::fs::MetadataExt::as_raw_stat
* os::unix::fs::PermissionsExt
* os::unix::fs::PermissionsExt::mode
* os::unix::fs::PermissionsExt::set_mode
* os::unix::fs::PermissionsExt::from_mode
* os::unix::fs::OpenOptionsExt
* os::unix::fs::OpenOptionsExt::mode
* os::unix::fs::DirEntryExt
* os::unix::fs::DirEntryExt::ino
* os::windows::fs::MetadataExt
* os::windows::fs::MetadataExt::file_attributes
* os::windows::fs::MetadataExt::creation_time
* os::windows::fs::MetadataExt::last_access_time
* os::windows::fs::MetadataExt::last_write_time
* os::windows::fs::MetadataExt::file_size
The `os::unix::fs::Metadata` structure was also removed entirely, moving all of
its associated methods into the `os::unix::fs::MetadataExt` trait instead. The
methods are all marked as `#[stable]` still.
As some minor cleanup, some deprecated and unstable fs apis were also removed:
* File::path
* Metadata::accessed
* Metadata::modified
Features that were explicitly left unstable include:
* fs::WalkDir - the semantics of this were not considered in the recent fs
expansion RFC.
* fs::DirBuilder - it's still not 100% clear if the naming is right here and if
the set of functionality exposed is appropriate.
* fs::canonicalize - the implementation on Windows here is specifically in
question as it always returns a verbatim path. Additionally the Unix
implementation is susceptible to buffer overflows on long paths unfortunately.
* fs::PathExt - as this is just a convenience trait, it is not stabilized at
this time.
* fs::set_file_times - this funciton is still waiting on a time abstraction.
This commit implements a number of standard traits for the standard library's
process I/O handles. The `FromRaw{Fd,Handle}` traits are now implemented for the
`Stdio` type and the `AsRaw{Fd,Handle}` traits are now implemented for the
`Child{Stdout,Stdin,Stderr}` types.
The stability markers for these implementations mention that they are stable for
1.1 as I will nominate this commit for cherry-picking to beta.
The change to split up soft_link to OS-specific symlink, symlink_file,
and symlink_dir didn't actually land in 1.0.0. Update the stability and
deprecation attributes to correctly indicate that these changes happend
in 1.1.0.
This commits adds a method to the `std::process` module to get the process
identifier of the child as a `u32`. On Windows the underlying identifier is
already a `u32`, and on Unix the type is typically defined as `c_int` (`i32` for
almost all our supported platforms), but the actually pid is normally a small
positive number.
Eventually we may add functions to load information about a process based on its
identifier or the ability to terminate a process based on its identifier, but
for now this function should enable this sort of functionality to exist outside
the standard library.
This commit implements a number of standard traits for the standard library's
process I/O handles. The `FromRaw{Fd,Handle}` traits are now implemented for the
`Stdio` type and the `AsRaw{Fd,Handle}` traits are now implemented for the
`Child{Stdout,Stdin,Stderr}` types. Additionally this implements the
`AsRawHandle` trait for `Child` on Windows.
The stability markers for these implementations mention that they are stable for
1.1 as I will nominate this commit for cherry-picking to beta.
This commits adds a method to the `std::process` module to get the process
identifier of the child as a `u32`. On Windows the underlying identifier is
already a `u32`, and on Unix the type is typically defined as `c_int` (`i32` for
almost all our supported platforms), but the actually pid is normally a small
positive number.
Eventually we may add functions to load information about a process based on its
identifier or the ability to terminate a process based on its identifier, but
for now this function should enable this sort of functionality to exist outside
the standard library.
This commit is an implementation of [RFC 1040][rfc] which is a redesign of the
currently-unstable `Duration` type. The API of the type has been scaled back to
be more conservative and it also no longer supports negative durations.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1040-duration-reform.md
The inner `duration` module of the `time` module has now been hidden (as
`Duration` is reexported) and the feature name for this type has changed from
`std_misc` to `duration`. All APIs accepting durations have also been audited to
take a more flavorful feature name instead of `std_misc`.
Closes#24874
This commit is an implementation of [RFC 1040][rfc] which is a redesign of the
currently-unstable `Duration` type. The API of the type has been scaled back to
be more conservative and it also no longer supports negative durations.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/1040-duration-reform.md
The inner `duration` module of the `time` module has now been hidden (as
`Duration` is reexported) and the feature name for this type has changed from
`std_misc` to `duration`. All APIs accepting durations have also been audited to
take a more flavorful feature name instead of `std_misc`.
Closes#24874
I've found that there are still huge amounts of occurrences of `task`s in the documentation. This PR tries to eliminate all of them in favor of `thread`.
An automated script was run against the `.rs` and `.md` files,
subsituting every occurrence of `task` with `thread`. In the `.rs`
files, only the texts in the comment blocks were affected.
Now that `std::old_io` has been removed for quite some time the naming real
estate here has opened up to allow these modules to move back to their proper
names.
Apparently implementations are allowed to return EDEADLK instead of blocking
forever, in which case this can lead to unsafety in the `RwLock` primitive
exposed by the standard library. A debug-build of the standard library would
have caught this error (due to the debug assert), but we don't ship debug
builds right now.
This commit adds explicit checks for the EDEADLK error code and triggers a panic
to ensure the call does not succeed.
Closes#25012
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.
[rfc]: https://github.com/rust-lang/rfcs/pull/1044
The new APIs added are:
* `fs::canonicalize` - bindings to `realpath` on unix and
`GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
function.
* `fs::PathExt::read_link` - convenience method for the top-level
function.
* `fs::PathExt::read_dir` - convenience method for the top-level
function.
* `std::os::raw` - type definitions for raw OS/C types available on all
platforms.
* `std::os::$platform` - new modules have been added for all currently supported
platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
are populated with the bare essentials necessary for lowing I/O types into
their raw representations, and currently largely consist of the `stat`
definition for unix platforms.
This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.
Closes https://github.com/rust-lang/rust/issues/24796
These commits build on [some great work on reddit](http://www.reddit.com/r/rust/comments/33boew/weekend_experiment_link_rust_programs_against/) for adding MUSL support to the compiler. This goal of this PR is to enable a `--target x86_64-unknown-linux-musl` argument to the compiler to work A-OK. The outcome here is that there are 0 compile-time dependencies for a MUSL-targeting build *except for a linker*. Currently this also assumes that MUSL is being used for statically linked binaries so there is no support for dynamically linked binaries with MUSL.
MUSL support largely just entailed munging around with the linker and where libs are located, and the major highlights are:
* The entirety of `libc.a` is included in `liblibc.rlib` (statically included as an archive).
* The entirety of `libunwind.a` is included in `libstd.rlib` (like with liblibc).
* The target specification for MUSL passes a number of ... flavorful options! Each option is documented in the relevant commit.
* The entire test suite currently passes with MUSL as a target, except for:
* Dynamic linking tests are all ignored as it's not supported with MUSL
* Stack overflow detection is not working MUSL yet (I'm not sure why)
* There is a language change included in this PR to add a `target_env` `#[cfg]` directive. This is used to conditionally build code for only MUSL (or for linux distros not MUSL). I highly suspect that this will also be used by Windows to target MSVC instead of a MinGW-based toolchain.
To build a compiler targeting MUSL you need to follow these steps:
1. Clone the current MUSL repo from `git://git.musl-libc.org/musl`. Build this as usual and install it.
2. Clone and build LLVM's [libcxxabi](http://libcxxabi.llvm.org/) library. Only the `libunwind.a` artifact is needed. I have tried using upstream libunwind's source repo but I have not gotten unwinding to work with it unfortunately. Move `libunwind.a` adjacent to MUSL's `libc.a`
3. Configure a Rust checkout with `--target=x86_64-unknown-linux-musl --musl-root=$MUSL_ROOT` where `MUSL_ROOT` is where you installed MUSL in step 1.
I hope to improve building a copy of libunwind as it's still a little sketchy and difficult to do today, but other than that everything should "just work"! This PR is not intended to include 100% comprehensive support for MUSL, as future modifications will probably be necessary.
This commit is an implementation of [RFC 1044][rfc] which adds additional
surface area to the `std::fs` module. All new APIs are `#[unstable]` behind
assorted feature names for each one.
[rfc]: https://github.com/rust-lang/rfcs/pull/1044
The new APIs added are:
* `fs::canonicalize` - bindings to `realpath` on unix and
`GetFinalPathNameByHandle` on windows.
* `fs::symlink_metadata` - similar to `lstat` on unix
* `fs::FileType` and accessor methods as `is_{file,dir,symlink}`
* `fs::Metadata::file_type` - accessor for the raw file type
* `fs::DirEntry::metadata` - acquisition of metadata which is free on Windows
but requires a syscall on unix.
* `fs::DirEntry::file_type` - access the file type which may not require a
syscall on most platforms.
* `fs::DirEntry::file_name` - access just the file name without leading
components.
* `fs::PathExt::symlink_metadata` - convenience method for the top-level
function.
* `fs::PathExt::canonicalize` - convenience method for the top-level
function.
* `fs::PathExt::read_link` - convenience method for the top-level
function.
* `fs::PathExt::read_dir` - convenience method for the top-level
function.
* `std::os::raw` - type definitions for raw OS/C types available on all
platforms.
* `std::os::$platform` - new modules have been added for all currently supported
platforms (e.g. those more specific than just `unix`).
* `std::os::$platform::raw` - platform-specific type definitions. These modules
are populated with the bare essentials necessary for lowing I/O types into
their raw representations, and currently largely consist of the `stat`
definition for unix platforms.
This commit also deprecates `Metadata::{modified, accessed}` in favor of
inspecting the raw representations via the lowering methods of `Metadata`.
Inspecting the current thread's info may not always work due to the TLS value
having been destroyed (or is actively being destroyed). The code for printing
a panic message assumed, however, that it could acquire the thread's name
through this method.
Instead this commit propagates the `Option` outwards to allow the
`std::panicking` module to handle the case where the current thread isn't
present.
While it solves the immediate issue of #24313, there is still another underlying
issue of panicking destructors in thread locals will abort the process.
Closes#24313
This commit modifies the standard library and its dependencies to link correctly
when built against MUSL. This primarily ensures that the right libraries are
linked against and when they're linked against they're linked against
statically.
These functions were intended to be introduced as `#[stable]` as a stable API
was deprecated in favor of them, but they just erroneously forgot the stability
attributes.
Much of this code hasn't been updated in quite some time and this commit does a
small audit of the functionality:
* Implementation functions now centralize all functionality on a locally defined
`Thread` type.
* The `detach` method has been removed in favor of a `Drop` implementation. This
notably fixes leaking thread handles on Windows.
* The `Thread` structure is now appropriately annotated with `Send` and `Sync`
automatically on Windows and in a custom fashion on Unix.
* The unsafety of creating a thread has been pushed out to the right boundaries
now.
Closes#24442
Implement [RFC #1048][rfc].
On Windows, when you create a symbolic link you must specify whether it
points to a directory or a file, even if it is created dangling, while
on Unix, the same symbolic link could point to a directory, a file, or
nothing at all. Furthermore, on Windows special privilege is necessary
to use a symbolic link, while on Unix, you can generally create a
symbolic link in any directory you have write privileges to.
This means that it is unlikely to be able to use symbolic links purely
portably; anyone who uses them will need to think about the cross
platform implications. This means that using platform-specific APIs
will make it easier to see where code will need to differ between the
platforms, rather than trying to provide some kind of compatibility
wrapper.
Furthermore, `soft_link` has no precedence in any other API, so to avoid
confusion, move back to the more standard `symlink` terminology.
Create a `std::os::unix::symlink` for the Unix version that is
destination type agnostic, as well as `std::os::windows::{symlink_file,
symlink_dir}` for Windows.
Because this is a stable API, leave a compatibility wrapper in
`std::fs::soft_link`, which calls `symlink` on Unix and `symlink_file`
on Windows, preserving the existing behavior of `soft_link`.
[rfc]: https://github.com/rust-lang/rfcs/pull/1048
This commit removes all the old casting/generic traits from `std::num` that are
no longer in use by the standard library. This additionally removes the old
`strconv` module which has not seen much use in quite a long time. All generic
functionality has been supplanted with traits in the `num` crate and the
`strconv` module is supplanted with the [rust-strconv crate][rust-strconv].
[rust-strconv]: https://github.com/lifthrasiir/rust-strconv
This is a breaking change due to the removal of these deprecated crates, and the
alternative crates are listed above.
[breaking-change]
On Windows, when you create a symbolic link you must specify whether it
points to a directory or a file, even if it is created dangling, while
on Unix, the same symbolic link could point to a directory, a file, or
nothing at all. Furthermore, on Windows special privilege is necessary
to use a symbolic link, while on Unix, you can generally create a
symbolic link in any directory you have write privileges to.
This means that it is unlikely to be able to use symbolic links purely
portably; anyone who uses them will need to think about the cross
platform implications. This means that using platform-specific APIs
will make it easier to see where code will need to differ between the
platforms, rather than trying to provide some kind of compatibility
wrapper.
Furthermore, `soft_link` has no precedence in any other API, so to avoid
confusion, move back to the more standard `symlink` terminology.
Create a `std::os::unix::symlink` for the Unix version that is
destination type agnostic, as well as `std::os::windows::{symlink_file,
symlink_dir}` for Windows.
Because this is a stable API, leave a compatibility wrapper in
`std::fs::soft_link`, which calls `symlink` on Unix and `symlink_file`
on Windows, preserving the existing behavior of `soft_link`.
This patch adds a `Debug` impl for `std::fs::File`.
On all platforms (Unix and Windows) it shows the file descriptor.
On Linux, it displays the path and access mode as well.
Ideally we should show the path/mode for all platforms, not just Linux,
but this will do for now.
cc #24570
As pointed out in [RFC issue 1043][rfc] it is quite useful to have the standard
I/O types to provide the contract that they are the sole owner of the underlying
object they represent. This guarantee enables writing safe interfaces like the
`MemoryMap` API sketched out in that issue.
[rfc]: https://github.com/rust-lang/rfcs/issues/1043
As constructing objects from these raw handles may end up violating these
ownership gurantees, the functions for construction are now marked unsafe.
[breaking-change]
Closesrust-lang/rfcs#1043
The logic for only closing file descriptors >= 3 was inherited from quite some
time ago and ends up meaning that some internal APIs are less consistent than
they should be. By unconditionally closing everything entering a `FileDesc` we
ensure that we're consistent in our behavior as well as robustly handling the
stdio case.
* De-indent quite a bit by removing usage of FnOnce closures
* Clearly separate code for the parent/child after the fork
* Use `fs2::{File, OpenOptions}` instead of calling `open` manually
* Use RAII to close I/O objects wherever possible
* Remove loop for closing all file descriptors, all our own ones are now
`CLOEXEC` by default so they cannot be inherited
This commit starts to set the CLOEXEC flag for all files and sockets opened by
the standard library by default on all unix platforms. There are a few points of
note in this commit:
* The implementation is not 100% satisfactory in the face of threads. File
descriptors only have the `F_CLOEXEC` flag set *after* they are opened,
allowing for a fork/exec to happen in the middle and leak the descriptor.
Some platforms do support atomically opening a descriptor while setting the
`CLOEXEC` flag, and it is left as a future extension to bind these apis as it
is unclear how to do so nicely at this time.
* The implementation does not offer a method of opting into the old behavior of
not setting `CLOEXEC`. This will possibly be added in the future through
extensions on `OpenOptions`, for example.
* This change does not yet audit any Windows APIs to see if the handles are
inherited by default by accident.
This is a breaking change for users who call `fork` or `exec` outside of the
standard library itself and expect file descriptors to be inherted. All file
descriptors created by the standard library will no longer be inherited.
[breaking-change]
As pointed out in [RFC issue 1043][rfc] it is quite useful to have the standard
I/O types to provide the contract that they are the sole owner of the underlying
object they represent. This guarantee enables writing safe interfaces like the
`MemoryMap` API sketched out in that issue.
[rfc]: https://github.com/rust-lang/rfcs/issues/1043
As constructing objects from these raw handles may end up violating these
ownership gurantees, the functions for construction are now marked unsafe.
[breaking-change]
Closesrust-lang/rfcs#1043
This commit stabilizes the old `io::Error::from_os_error` after being renamed to
use the `raw_os_error` terminology instead. This function is often useful when
writing bindings to OS functions but only actually converting to an I/O error at
a later point.
This commit stabilizes the old `io::Error::from_os_error` after being renamed to
use the `raw_os_error` terminology instead. This function is often useful when
writing bindings to OS functions but only actually converting to an I/O error at
a later point.
write_fmt calls write for each formatted field. The default implementation of write_fmt is used,
which will call write on not-yet-locked stdout (and write locking after), therefore making print!
in multithreaded environment still interleave contents of two separate prints.
I’m not sure whether we want to do this change, though, because it has the same deadlock hazard which we tried to avoid by not locking inside write_fmt itself (see [this comment](80def6c244/src/libstd/io/stdio.rs (L267))).
Spotted on [reddit].
cc @alexcrichton
[reddit]: http://www.reddit.com/r/rust/comments/31comh/println_with_multiple_threads/
write_fmt calls write for each formatted field. The default implementation of write_fmt is used,
which will call write on not-yet-locked stdout (and write locking after), therefore making print!
in multithreaded environment still interleave contents of two separate prints.
This patch implements reentrant mutexes, changes stdio handles to use these mutexes and overrides
write_fmt to lock the stdio handle for the whole duration of the call.
These constants are small and can fit even in `u8`, but semantically they have type `usize` because they denote sizes and are almost always used in `usize` context. The change of their type to `u32` during the integer audit led only to the large amount of `as usize` noise (see the second commit, which removes this noise).
This is a minor [breaking-change] to an unstable interface.
r? @aturon
This commit is an implementation of [RFC 979][rfc] which changes the meaning of
the count parameter to the `splitn` function on strings and slices. The
parameter now means the number of items that are returned from the iterator, not
the number of splits that are made.
[rfc]: https://github.com/rust-lang/rfcs/pull/979Closes#23911
[breaking-change]
This commit stabilizes a few remaining bits of the `io::Error` type:
* The `Error::new` method is now stable. The last `detail` parameter was removed
and the second `desc` parameter was generalized to `E: Into<Box<Error>>` to
allow creating an I/O error from any form of error. Currently there is no form
of downcasting, but this will be added in time.
* An implementation of `From<&str> for Box<Error>` was added to liballoc to
allow construction of errors from raw strings.
* The `Error::raw_os_error` method was stabilized as-is.
* Trait impls for `Clone`, `Eq`, and `PartialEq` were removed from `Error` as it
is not possible to use them with trait objects.
This is a breaking change due to the modification of the `new` method as well as
the removal of the trait implementations for the `Error` type.
[breaking-change]
This commit is an implementation of [RFC #1011][rfc] which adds an `exit`
function to the standard library for immediately terminating the current process
with a specified exit code.
[rfc]: https://github.com/rust-lang/rfcs/pull/1011Closes#23914
* Marks `#[stable]` the contents of the `std::convert` module.
* Added methods `PathBuf::as_path`, `OsString::as_os_str`,
`String::as_str`, `Vec::{as_slice, as_mut_slice}`.
* Deprecates `OsStr::from_str` in favor of a new, stable, and more
general `OsStr::new`.
* Adds unstable methods `OsString::from_bytes` and `OsStr::{to_bytes,
to_cstring}` for ergonomic FFI usage.
[breaking-change]
r? @alexcrichton
This commit cleans out a large amount of deprecated APIs from the standard
library and some of the facade crates as well, updating all users in the
compiler and in tests as it goes along.
This commit stabilizes the platform-specific `io` modules, specifically around
the traits having to do with the raw representation of each object on each
platform.
Specifically, the following material was stabilized:
* `AsRaw{Fd,Socket,Handle}`
* `RawFd` (renamed from `Fd`)
* `RawHandle` (renamed from `Handle`)
* `RawSocket` (renamed from `Socket`)
* `AsRaw{Fd,Socket,Handle}` implementations
* `std::os::{unix, windows}::io`
The following material was added as `#[unstable]`:
* `FromRaw{Fd,Socket,Handle}`
* Implementations for various primitives
There are a number of future improvements that are possible to make to this
module, but this should cover a good bit of functionality desired from these
modules for now. Some specific future additions may include:
* `IntoRawXXX` traits to consume the raw representation and cancel the
auto-destructor.
* `Fd`, `Socket`, and `Handle` abstractions that behave like Rust objects and
have nice methods for various syscalls.
At this time though, these are considered backwards-compatible extensions and
will not be stabilized at this time.
This commit is a breaking change due to the addition of `Raw` in from of the
type aliases in each of the platform-specific modules.
[breaking-change]
This commit is an implementation of [RFC #1011][rfc] which adds an `exit`
function to the standard library for immediately terminating the current process
with a specified exit code.
[rfc]: https://github.com/rust-lang/rfcs/pull/1011
* Marks `#[stable]` the contents of the `std::convert` module.
* Added methods `PathBuf::as_path`, `OsString::as_os_str`,
`String::as_str`, `Vec::{as_slice, as_mut_slice}`.
* Deprecates `OsStr::from_str` in favor of a new, stable, and more
general `OsStr::new`.
* Adds unstable methods `OsString::from_bytes` and `OsStr::{to_bytes,
to_cstring}` for ergonomic FFI usage.
[breaking-change]
This commit stabilizes the `std::num` module:
* The `Int` and `Float` traits are deprecated in favor of (1) the
newly-added inherent methods and (2) the generic traits available in
rust-lang/num.
* The `Zero` and `One` traits are reintroduced in `std::num`, which
together with various other traits allow you to recover the most
common forms of generic programming.
* The `FromStrRadix` trait, and associated free function, is deprecated
in favor of inherent implementations.
* A wide range of methods and constants for both integers and floating
point numbers are now `#[stable]`, having been adjusted for integer
guidelines.
* `is_positive` and `is_negative` are renamed to `is_sign_positive` and
`is_sign_negative`, in order to address #22985
* The `Wrapping` type is moved to `std::num` and stabilized;
`WrappingOps` is deprecated in favor of inherent methods on the
integer types, and direct implementation of operations on
`Wrapping<X>` for each concrete integer type `X`.
Closes#22985Closes#21069
[breaking-change]
This commit stabilizes the platform-specific `io` modules, specifically around
the traits having to do with the raw representation of each object on each
platform.
Specifically, the following material was stabilized:
* `AsRaw{Fd,Socket,Handle}`
* `RawFd` (renamed from `Fd`)
* `RawHandle` (renamed from `Handle`)
* `RawSocket` (renamed from `Socket`)
* `AsRaw{Fd,Socket,Handle}` implementations
* `std::os::{unix, windows}::io`
The following material was added as `#[unstable]`:
* `FromRaw{Fd,Socket,Handle}`
* Implementations for various primitives
There are a number of future improvements that are possible to make to this
module, but this should cover a good bit of functionality desired from these
modules for now. Some specific future additions may include:
* `IntoRawXXX` traits to consume the raw representation and cancel the
auto-destructor.
* `Fd`, `Socket`, and `Handle` abstractions that behave like Rust objects and
have nice methods for various syscalls.
At this time though, these are considered backwards-compatible extensions and
will not be stabilized at this time.
This commit is a breaking change due to the addition of `Raw` in from of the
type aliases in each of the platform-specific modules.
[breaking-change]
This commit revises `path` and `os_str` to use blanket impls for `From`
on reference types. This both cuts down on the number of required impls,
and means that you can pass through e.g. `T: AsRef<OsStr>` to
`PathBuf::from` without an intermediate call to `as_ref`.
It also makes a FIXME note for later generalizing the blanket impls for
`AsRef` and `AsMut` to use `Deref`/`DerefMut`, once it is possible to do
so.
Linking `__pthread_get_minstack`, even weakly, was causing Debian’s `dpkg-shlibdeps` to detect an unnecessarily strict versioned dependency on libc6.
Closes#23628.
This commit:
* Introduces `std::convert`, providing an implementation of
RFC 529.
* Deprecates the `AsPath`, `AsOsStr`, and `IntoBytes` traits, all
in favor of the corresponding generic conversion traits.
Consequently, various IO APIs now take `AsRef<Path>` rather than
`AsPath`, and so on. Since the types provided by `std` implement both
traits, this should cause relatively little breakage.
* Deprecates many `from_foo` constructors in favor of `from`.
* Changes `PathBuf::new` to take no argument (creating an empty buffer,
as per convention). The previous behavior is now available as
`PathBuf::from`.
* De-stabilizes `IntoCow`. It's not clear whether we need this separate trait.
Closes#22751Closes#14433
[breaking-change]
Linking __pthread_get_minstack, even weakly, was causing Debian’s
dpkg-shlibdeps to detect an unnecessarily strict versioned dependency
on libc6.
Closes#23628.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
This commit removes the reexports of `old_io` traits as well as `old_path` types
and traits from the prelude. This functionality is now all deprecated and needs
to be removed to make way for other functionality like `Seek` in the `std::io`
module (currently reexported as `NewSeek` in the io prelude).
Closes#23377Closes#23378
This commit removes the reexports of `old_io` traits as well as `old_path` types
and traits from the prelude. This functionality is now all deprecated and needs
to be removed to make way for other functionality like `Seek` in the `std::io`
module (currently reexported as `NewSeek` in the io prelude).
Closes#23377Closes#23378
This reverts commit aec67c2.
Closes#20012
This is temporarily rebased on #23245 as it would otherwise conflict, the last commit is the only one relevant to this PR though.
This commit stabilizes the `ErrorKind` enumeration which is consumed by and
generated by the `io::Error` type. The purpose of this type is to serve as a
cross-platform namespace to categorize errors into. Two specific issues are
addressed as part of this stablization:
* The naming of each variant was scrutinized and some were tweaked. An example
is how `FileNotFound` was renamed to simply `NotFound`. These names should not
show either a Unix or Windows bias and the set of names is intended to grow
over time. For now the names will likely largely consist of those errors
generated by the I/O APIs in the standard library.
* The mapping of OS error codes onto kinds has been altered. Coalescing no
longer occurs (multiple error codes become one kind). It is intended that each
OS error code, if bound, corresponds to only one `ErrorKind`. The current set
of error kinds was expanded slightly to include some networking errors.
This commit also adds a `raw_os_error` function which returns an `Option<i32>`
to extract the underlying raw error code from the `Error`.
Closes#16666
[breaking-change]
This commit stabilizes the `ErrorKind` enumeration which is consumed by and
generated by the `io::Error` type. The purpose of this type is to serve as a
cross-platform namespace to categorize errors into. Two specific issues are
addressed as part of this stablization:
* The naming of each variant was scrutinized and some were tweaked. An example
is how `FileNotFound` was renamed to simply `NotFound`. These names should not
show either a Unix or Windows bias and the set of names is intended to grow
over time. For now the names will likely largely consist of those errors
generated by the I/O APIs in the standard library.
* The mapping of OS error codes onto kinds has been altered. Coalescing no
longer occurs (multiple error codes become one kind). It is intended that each
OS error code, if bound, corresponds to only one `ErrorKind`. The current set
of error kinds was expanded slightly to include some networking errors.
This commit also adds a `raw_os_error` function which returns an `Option<i32>`
to extract the underlying raw error code from the `Error`.
unbreak openbsd/bitrig build
- remove `pub` from `struct` (error: visibility has no effect inside functions)
- move `pthread_main_np` into function
r? @alexcrichton
