This change is the final step in improving the semantics of
zx_cprng_draw. Now the syscall always generates the requested number of
bytes. If the syscall would have failed to generate the requested number
of bytes, the syscall either terminates the entire operating system or
terminates the calling process, depending on whether the error is a
result of the kernel misbehaving or the userspace program misbehaving.
Remove unnecessary stat64 pointer casts
In effect, these just casted `&mut stat64` to `*mut stat64`, twice.
That's harmless, but it masked a problem when this was copied to new
code calling `fstatat`, which takes a pointer to `struct stat`. That
will be fixed by #51785, but let's remove the unnecessary casts here
too.
Fix possibly endless loop in ReadDir iterator
Certain directories in `/proc` can cause the `ReadDir` iterator to loop indefinitely. We get an error code (22) when calling libc's `readdir_r` on these directories, but `entry_ptr` is `NULL` at the same time, signalling the end of the directory stream.
This change introduces an internal state to the iterator such that the `Some(Err(..))` value will only be returned once when calling `next`. Subsequent calls will return `None`.
fixes#50619
In effect, these just casted `&mut stat64` to `*mut stat64`, twice.
That's harmless, but it masked a problem when this was copied to new
code calling `fstatat`, which takes a pointer to `struct stat`. That
will be fixed by #51785, but let's remove the unnecessary casts here
too.
Certain directories in `/proc` can cause the `ReadDir`
iterator to loop indefinitely. We get an error code (22) when
calling libc's `readdir_r` on these directories, but `entry_ptr`
is `NULL` at the same time, signalling the end of the directory
stream.
This change introduces an internal state to the iterator such
that the `Some(Err(..))` value will only be returned once when
calling `next`. Subsequent calls will return `None`.
fixes#50619
Fix confusing error message for sub_instant
When subtracting an Instant from another, the function will panick when `RHS > self`, but the error message confusingly displays a different error:
```rust
let i = Instant::now();
let other = Instant::now();
if other > i {
println!("{:?}", i - other);
}
```
This results in a panic:
```
thread 'test_instant' panicked at 'other was less than the current instant', libstd/sys/unix/time.rs:292:17
```
But clearly, `other` was actually greater than the current instant.
fs: copy: Use File::set_permissions instead of fs::set_permissions
We already got the open file descriptor at this point.
Don't make the kernel resolve the path again.
When subtracting an Instant from another, the function will panick when `RHS > self`, but the error message confusingly displays a different error:
```rust
let i = Instant::now();
let other = Instant::now();
if other > i {
println!("{:?}", i - other);
}
```
This results in a panic:
```
thread 'test_instant' panicked at 'other was less than the current instant', libstd/sys/unix/time.rs:292:17
```
fs: copy: use copy_file_range on Linux
Linux 4.5 introduced a new system call [copy_file_range](http://man7.org/linux/man-pages/man2/copy_file_range.2.html) to copy data from one file to another.
This PR uses the new system call (if available). This has several advantages:
1. No need to constantly copy data from userspace to kernel space, if the buffer is small or the file is large
2. On some filesystems, like BTRFS, the kernel can leverage internal fs mechanisms for huge performance gains
3. Filesystems on the network dont need to copy data between the host and the client machine (they have to in the current read/write implementation)
I have created a small library that also implements the new system call for some huge performance gains here: https://github.com/nicokoch/fastcopy
Benchmark results are in the README
read2: Use inner function instead of closure
Very minor thing, but there doesn't appear to be a reason to use a closure here.
Generated code is identical in my tests, but I believe it's clearer that nothing from the environment is being used.
Don't unconditionally set CLOEXEC twice on every fd we open on Linux
Previously, every `open64` was accompanied by a `ioctl(…, FIOCLEX)`,
because some old Linux version would ignore the `O_CLOEXEC` flag we pass
to the `open64` function.
Now, we check whether the `CLOEXEC` flag is set on the first file we
open – if it is, we won't do extra syscalls for every opened file. If it
is not set, we fall back to the old behavior of unconditionally calling
`ioctl(…, FIOCLEX)` on newly opened files.
On old Linuxes, this amounts to one extra syscall per process, namely
the `fcntl(…, F_GETFD)` call to check the `CLOEXEC` flag.
On new Linuxes, this reduces the number of syscalls per opened file by
one, except for the first file, where it does the same number of
syscalls as before (`fcntl(…, F_GETFD)` to check the flag instead of
`ioctl(…, FIOCLEX)` to set it).
Previously, every `open64` was accompanied by a `ioctl(…, FIOCLEX)`,
because some old Linux version would ignore the `O_CLOEXEC` flag we pass
to the `open64` function.
Now, we check whether the `CLOEXEC` flag is set on the first file we
open – if it is, we won't do extra syscalls for every opened file. If it
is not set, we fall back to the old behavior of unconditionally calling
`ioctl(…, FIOCLEX)` on newly opened files.
On old Linuxes, this amounts to one extra syscall per process, namely
the `fcntl(…, F_GETFD)` call to check the `CLOEXEC` flag.
On new Linuxes, this reduces the number of syscalls per opened file by
one, except for the first file, where it does the same number of
syscalls as before (`fcntl(…, F_GETFD)` to check the flag instead of
`ioctl(…, FIOCLEX)` to set it).
While currently only NetBSD seems to be affected, all systems
implementing PAX MPROTECT in strict mode need this treatment,
and it does not hurt others.
On NetBSD the initial mmap() protection of a mapping can not be made
less restrictive with mprotect().
So when mapping a stack guard page, use the maximum protection
we ever want to use, then mprotect() it to the permission we
want it to have initially.
Android abstract unix domain sockets AddressKind correction
The prior check causes abstract unix domain sockets to return AddressKind::Unnamed instead of AddressKind::Abstract on Android.
Other than the immediately proceeding comment "macOS seems to return a len of 16 and a zeroed sun_path for unnamed addresses" the check as-implemented does not seem to have alternative explanation. I couldn't find an alternative explanation while stepping though git blame. I suspect the AddressKind::Unnamed nonzero check should instead be if macos, length 16, and zeroed array. @sfackler could you comment on this, the code as-is is the same from your initial addition of abstract uds support.
Add doc links to `std::os` extension traits
Addresses a small subset of #29367.
This adds documentation links to the original type for various OS-specific extension traits, and uses a common sentence for introducing such traits (which now consistently ends in a period).
std: Minimize size of panicking on wasm
This commit applies a few code size optimizations for the wasm target to
the standard library, namely around panics. We notably know that in most
configurations it's impossible for us to print anything in
wasm32-unknown-unknown so we can skip larger portions of panicking that
are otherwise simply informative. This allows us to get quite a nice
size reduction.
Finally we can also tweak where the allocation happens for the
`Box<Any>` that we panic with. By only allocating once unwinding starts
we can reduce the size of a panicking wasm module from 44k to 350 bytes.
Prevent broken pipes causing ICEs
As the private `std::io::print_to` panics if there is an I/O error, which is used by `println!`, the compiler would ICE if one attempted to use a broken pipe (e.g. `rustc --help | false`). This introduces a new (private) macro `try_println!` which allows us to avoid this.
As a side note, it seems this macro might be useful publicly (and actually there seems to be [a crate specifically for this purpose](https://crates.io/crates/try_print/)), though that can probably be left for a future discussion.
One slight alternative approach would be to simply early exit without an error (i.e. exit code `0`), which [this comment](https://github.com/rust-lang/rust/issues/34376#issuecomment-377822526) suggests is the usual approach. I've opted not to take that approach initially, because I think it's more helpful to know when there is a broken pipe.
Fixes#34376.
Add documentation links to the original type for various OS-specific
extension traits and normalize the language for introducing such traits.
Also, remove some outdated comments around the extension trait
definitions.
This commit applies a few code size optimizations for the wasm target to
the standard library, namely around panics. We notably know that in most
configurations it's impossible for us to print anything in
wasm32-unknown-unknown so we can skip larger portions of panicking that
are otherwise simply informative. This allows us to get quite a nice
size reduction.
Finally we can also tweak where the allocation happens for the
`Box<Any>` that we panic with. By only allocating once unwinding starts
we can reduce the size of a panicking wasm module from 44k to 350 bytes.
These were showing up in tests and in binaries but are trivially optimize-able
away, so add `#[inline]` attributes so LLVM has an opportunity to optimize them
out.
rustc_driver: get rid of the extra thread
**Do not rollup**
We can alter the stack size afterwards on Unix.
Having a separate thread causes poor debugging experience when interrupting with signals. I have to get the backtrace of the all thread, as the main thread is waiting to join doing nothing else. This patch allows me to just run `bt` to get the desired backtrace.
Reduce scope of unsafe block in sun_path_offset
I reduced the scope of the unsafe block to the `uninitialized` call which is the only actual unsafe bit.
The expected behavior is that the environment's PATH should be used
to find the process. posix_spawn() could be used if we iterated
PATH to search for the binary to execute. For now just skip
posix_spawn() if PATH is modified.
spawn() is expected to return an error if the specified file could not be
executed. FreeBSD's posix_spawn() supports returning ENOENT/ENOEXEC if
the exec() fails, which not all platforms support. This brings a very
significant performance improvement for FreeBSD, involving heavy use of
Command in threads, due to fork() invoking jemalloc fork handlers and
causing lock contention. FreeBSD's posix_spawn() avoids this problem
due to using vfork() internally.
Previously, the `guard::init()` and `guard::current()` functions were
returning a `usize` address representing the top of the stack guard,
respectively for the main thread and for spawned threads. The `SIGSEGV`
handler on `unix` targets checked if a fault was within one page below
that address, if so reporting it as a stack overflow.
Now `unix` targets report a `Range<usize>` representing the guard
memory, so it can cover arbitrary guard sizes. Non-`unix` targets which
always return `None` for guards now do so with `Option<!>`, so they
don't pay any overhead.
For `linux-gnu` in particular, the previous guard upper-bound was
`stackaddr + guardsize`, as the protected memory was *inside* the stack.
This was a glibc bug, and starting from 2.27 they are moving the guard
*past* the end of the stack. However, there's no simple way for us to
know where the guard page actually lies, so now we declare it as the
whole range of `stackaddr ± guardsize`, and any fault therein will be
called a stack overflow. This fixes#47863.
Only link res_init() on GNU/*nix
To workaround a bug in glibc <= 2.26 lookup_host() calls res_init() based on the glibc version detected at runtime. While this avoids calling res_init() on platforms where it's not required we will still end up linking against the symbol.
This causes an issue on macOS where res_init() is implemented in a separate library (libresolv.9.dylib) from the main libc. While this is harmless for standalone programs it becomes a problem if Rust code is statically linked against another program. If the linked program doesn't already specify -lresolv it will cause the link to fail. This is captured in issue #46797
Fix this by hooking in to the glibc workaround in `cvt_gai` and only activating it for the "gnu" environment on Unix This should include all glibc platforms while excluding musl, windows-gnu, macOS, FreeBSD, etc.
This has the side benefit of removing the #[cfg] in sys_common; only unix.rs has code related to the workaround now.
Before this commit:
```shell
> cat main.rs
use std::net::ToSocketAddrs;
#[no_mangle]
pub extern "C" fn resolve_test() -> () {
let addr_list = ("google.com.au", 0).to_socket_addrs().unwrap();
println!("{:?}", addr_list);
}
> rustc --crate-type=staticlib main.rs
> clang libmain.a test.c -o combined
Undefined symbols for architecture x86_64:
"_res_9_init", referenced from:
std::net::lookup_host::h93c17fe9ad38464a in libmain.a(std-826c8d3b356e180c.std0.rcgu.o)
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
```
Afterwards:
```shell
> rustc --crate-type=staticlib main.rs
> clang libmain.a test.c -o combined
> ./combined
IntoIter([V4(172.217.25.131:0)])
```
Fixes #46797
To workaround a bug in glibc <= 2.26 lookup_host() calls res_init()
based on the glibc version detected at runtime. While this avoids
calling res_init() on platforms where it's not required we will still
end up linking against the symbol.
This causes an issue on macOS where res_init() is implemented in a
separate library (libresolv.9.dylib) from the main libc. While this is
harmless for standalone programs it becomes a problem if Rust code is
statically linked against another program. If the linked program doesn't
already specify -lresolv it will cause the link to fail. This is
captured in issue #46797
Fix this by hooking in to the glibc workaround in `cvt_gai` and only
activating it for the "gnu" environment on Unix This should include all
glibc platforms while excluding musl, windows-gnu, macOS, FreeBSD, etc.
This has the side benefit of removing the #[cfg] in sys_common; only
unix.rs has code related to the workaround now.
Convert warning about `*const _` to a future-compat lint
#46664 was merged before I could convert the soft warning about method lookup on `*const _` into a future-compatibility lint. This PR makes that change.
fixes#46837
tracking issue for the future-compatibility lint: #46906
r? @arielb1
Use the CTL_KERN.KERN_PROC_ARGS.-1.KERN_PROC_PATHNAME sysctl in
preference over the /proc/curproc/exe symlink.
Additionally, perform more validation of aformentioned symlink.
Particularly on pre-8.x NetBSD this symlink will point to '/' when
accurate information is unavailable.
Provides the following conversion implementations:
* `From<`{`CString`,`&CStr`}`>` for {`Arc`,`Rc`}`<CStr>`
* `From<`{`OsString`,`&OsStr`}`>` for {`Arc`,`Rc`}`<OsStr>`
* `From<`{`PathBuf`,`&Path`}`>` for {`Arc`,`Rc`}`<Path>`
Use getrandom syscall for all Linux and Android targets.
I suppose we can use it in all Linux and Android targets. In function `is_getrandom_available` is checked if the syscall is available (getrandom syscall was add in version 3.17 of Linux kernel), if the syscall is not available `fill_bytes` fallback to reading from `/dev/urandom`.
Update libc to include getrandom related constants.
This commit removes usage of the `libc` crate in "portable" modules like
those at the top level and `sys_common`. Instead common types like `*mut
u8` or `u32` are used instead of `*mut c_void` or `c_int` as well as
switching to platform-specific functions like `sys::strlen` instead of
`libc::strlen`.
This commit moves the `f32::cmath` and `f64::cmath` modules into the
`sys` module. Note that these are not publicly exported modules, simply
implementation details. These modules are already platform-specific with
shims on MSVC and this is mostly just a reflection of that reality. This
should also help cut down on `#[cfg]` traffic if platforms are brought on
which don't directly support these functions.
This commit removes the reexport of `EBADF_ERR` as a constant from
libstd's portability facade, instead opting for a platform-specific
function that specifically queries an `io::Error`. Not all platforms may
have a constant for this, so it makes the intent a little more clear
that a code need not be supplied, just an answer to a query.
This commit removes the `rand` crate from the standard library facade as
well as the `__rand` module in the standard library. Neither of these
were used in any meaningful way in the standard library itself. The only
need for randomness in libstd is to initialize the thread-local keys of
a `HashMap`, and that unconditionally used `OsRng` defined in the
standard library anyway.
The cruft of the `rand` crate and the extra `rand` support in the
standard library makes libstd slightly more difficult to port to new
platforms, namely WebAssembly which doesn't have any randomness at all
(without interfacing with JS). The purpose of this commit is to clarify
and streamline randomness in libstd, focusing on how it's only required
in one location, hashmap seeds.
Note that the `rand` crate out of tree has almost always been a drop-in
replacement for the `rand` crate in-tree, so any usage (accidental or
purposeful) of the crate in-tree should switch to the `rand` crate on
crates.io. This then also has the further benefit of avoiding
duplication (mostly) between the two crates!
More fixes for x86_64-unknown-linux-gnux32
This update libc (all libc testing are passing) and fixes NR_GETRANDOM.
Fix all but one run-pass test (lto-unwind.rs, see https://github.com/rust-lang/rust/issues/45416)
Linux appears to set POLLOUT when a conection's refused, which is pretty
weird. Invert the check to look for an error explicitly. Also add an
explict test for this case.
Closes#45265.
Remove support for the PNaCl target (le32-unknown-nacl)
This removes support for the `le32-unknown-nacl` target which is currently supported by rustc on tier 3. Despite the "nacl" in the name, the target doesn't output native code (x86, ARM, MIPS), instead it outputs binaries in the PNaCl format.
There are two reasons for the removal:
* Google [has announced](https://blog.chromium.org/2017/05/goodbye-pnacl-hello-webassembly.html) deprecation of the PNaCl format. The suggestion is to migrate to wasm. Happens we already have a wasm backend!
* Our PNaCl LLVM backend is provided by the fastcomp patch set that the LLVM fork used by rustc contains in addition to vanilla LLVM (`src/llvm/lib/Target/JSBackend/NaCl`). Upstream LLVM doesn't have PNaCl support. Removing PNaCl support will enable us to move away from fastcomp (#44006) and have a lighter set of patches on top of upstream LLVM inside our LLVM fork. This will help distribution packagers of Rust.
Fixes#42420
zircon: the type of zx_handle_t is now unsigned
This is a kernel ABI change that landed today. I noticed some other ABI
issues and have left a note to cleanup once they are better defined.
The previous workaround for gibc's res_init bug is not thread-safe on
other implementations of libc, and it can cause crashes. Use a runtime
check to make sure we only call res_init when we need to, which is also
when it's safe. See https://github.com/rust-lang/rust/issues/43592.
* Use size_t where size_t is used, while it's not critical on our
specifically supported architectures, this is more accurate.
* Update HND_SPECIAL_COUNT to the correct value, and give it the size
that enum is likely to be.
Most users would expect set_permissions(Metadata.permissions()) to be
non-destructive. While we can't guarantee this, we can at least pass
the needed info to chmod.
Also update the PermissionsExt documentation to disambiguate what it
contains, and to refer to the underlying value as `st_mode` rather than
its type `mode_t`.
Closes#44147
Add the libstd-modifications needed for the L4Re target
This commit adds the needed modifications to compile the std crate for the L4 Runtime environment (L4Re).
A target for the L4Re was introduced in commit: c151220a84
In many aspects implementations for linux also apply for the L4Re microkernel.
Some uncommon characteristics had to be resolved:
* L4Re has no network funktionality
* L4Re has a maximum stacksize of 1Mb for threads
* L4Re has no uid or gid
Co-authored-by: Sebastian Humenda <sebastian.humenda@tu-dresden.de>
Properly detect overflow in Instance ± Duration.
Fix#44216.
Fix#42622
The computation `Instant::now() + Duration::from_secs(u64::max_value())` now panics. The call `receiver.recv_timeout(Duration::from_secs(u64::max_value()))`, which involves such time addition, will also panic.
The reason #44216 arises is because of an unchecked cast from `u64` to `i64`, making the duration equivalent to -1 second.
Note that the current implementation is over-conservative, since e.g. (-2⁶²) + (2⁶³) is perfectly fine for an `i64`, yet this is rejected because (2⁶³) overflows the `i64`.
As suggested in the discussion of PR #43972, std should provide a uniform API to
all platforms. Since there's no networking on L4Re, this now is a module in
`sys::net` providing types and functions/methods returning an error for each
action.
This commit adds the needed modifications to compile the std crate
for the L4 Runtime environment (L4Re).
A target for the L4Re was introduced in commit:
c151220a84
In many aspects implementations for linux also apply for the L4Re
microkernel.
Two uncommon characteristics had to be resolved:
* L4Re has no network funktionality
* L4Re has a maximum stacksize of 1Mb for threads
Co-authored-by: Sebastian Humenda <sebastian.humenda@tu-dresden.de>
* Match definition of c_char in os/raw.rs with the libc definition
Due to historic reasons, os/raw.rs redefines types for c_char from
libc, but these didn't match. Now they do :).
* Enable signal reset on exec for L4Re
L4Re has full signal emulation and hence it needs to reset the
signal set of the child with sigemptyset. However, gid and uid
should *not* be set.
Refactoring: move net specific file descriptor methods
Move the implementations of net specific file descriptor methods from
io to net. This makes it easier to exclude net at all if it is not needed
for a target.
Move the implementations of net specific file descriptior implementations
to net. This makes it easier to exclude net at all if not needed for a
target.
Expose all OS-specific modules in libstd doc.
1. Uses the special `--cfg dox` configuration passed by rustbuild when running `rustdoc`. Changes the `#[cfg(platform)]` into `#[cfg(any(dox, platform))]` so that platform-specific API are visible to rustdoc.
2. Since platform-specific implementations often won't compile correctly on other platforms, `rustdoc` is changed to apply `everybody_loops` to the functions during documentation and doc-test harness.
3. Since platform-specific code are documented on all platforms now, it could confuse users who found a useful API but is non-portable. Also, their examples will be doc-tested, so must be excluded when not testing on the native platform. An undocumented attribute `#[doc(cfg(...))]` is introduced to serve the above purposed.
Fixes#24658 (Does _not_ fully implement #1998).
Fixed mutable vars being marked used when they weren't
#### NB : bootstrapping is slow on my machine, even with `keep-stage` - fixes for occurances in the current codebase are <s>in the pipeline</s> done. This PR is being put up for review of the fix of the issue.
Fixes#43526, Fixes#30280, Fixes#25049
### Issue
Whenever the compiler detected a mutable deref being used mutably, it marked an associated value as being used mutably as well. In the case of derefencing local variables which were mutable references, this incorrectly marked the reference itself being used mutably, instead of its contents - with the consequence of making the following code emit no warnings
```
fn do_thing<T>(mut arg : &mut T) {
... // don't touch arg - just deref it to access the T
}
```
### Fix
Make dereferences not be counted as a mutable use, but only when they're on borrows on local variables.
#### Why not on things other than local variables?
* Whenever you capture a variable in a closure, it gets turned into a hidden reference - when you use it in the closure, it gets dereferenced. If the closure uses the variable mutably, that is actually a mutable use of the thing being dereffed to, so it has to be counted.
* If you deref a mutable `Box` to access the contents mutably, you are using the `Box` mutably - so it has to be counted.
This commit adds a disabled builder which will run all tests for the standard
library for aarch64 in a QEMU instance. Once we get enough capacity to run this
on Travis this can be used to boost our platform coverage of AArch64
Linux doesn't allocate the whole stack right away, and the kernel has
its own stack-guard mechanism to fault when growing too close to an
existing mapping. If we map our own guard, then the kernel starts
enforcing a rather large gap above that, rendering much of the possible
stack space useless.
Instead, we'll just note where we expect rlimit to start faulting, so
our handler can report "stack overflow", and trust that the kernel's own
stack guard will work.
Fixes#43052.
This PR is an implementation of [RFC 1974] which specifies a new method of
defining a global allocator for a program. This obsoletes the old
`#![allocator]` attribute and also removes support for it.
[RFC 1974]: https://github.com/rust-lang/rfcs/pull/197
The new `#[global_allocator]` attribute solves many issues encountered with the
`#![allocator]` attribute such as composition and restrictions on the crate
graph itself. The compiler now has much more control over the ABI of the
allocator and how it's implemented, allowing much more freedom in terms of how
this feature is implemented.
cc #27389
rustc: Enable #[thread_local] for Windows
I think LLVM has had support for quite some time now for this, we just never got
around to testing it out and binding it. We've had some trouble landing this in
the past I believe, but it's time to try again!
This commit flags the `#[thread_local]` attribute as being available for Windows
targets and adds an implementation of `register_dtor` in the `thread::local`
module to ensure we can destroy these keys. The same functionality is
implemented in clang via a function called `__tlregdtor` (presumably provided in
some Windows runtime somewhere), but this function unfortunately does not take a
data pointer (just a thunk) which means we can't easily call it. For now
destructors are just run in the same way the Linux fallback is implemented,
which is just keeping track via a single OS-based TLS key.
I think LLVM has had support for quite some time now for this, we just never got
around to testing it out and binding it. We've had some trouble landing this in
the past I believe, but it's time to try again!
This commit flags the `#[thread_local]` attribute as being available for Windows
targets and adds an implementation of `register_dtor` in the `thread::local`
module to ensure we can destroy these keys. The same functionality is
implemented in clang via a function called `__tlregdtor` (presumably provided in
some Windows runtime somewhere), but this function unfortunately does not take a
data pointer (just a thunk) which means we can't easily call it. For now
destructors are just run in the same way the Linux fallback is implemented,
which is just keeping track via a single OS-based TLS key.
Replaced by adding extra imports, adding hidden code (`# ...`), modifying
examples to be runnable (sorry Homura), specifying non-Rust code, and
converting to should_panic, no_run, or compile_fail.
Remaining "```ignore"s received an explanation why they are being ignored.
Fixes issue #37440: `pthread_cond_timedwait` on macOS Sierra seems
to overflow `ts_sec` parameter and returns immediately. To work
around this problem patch rounds timeout down to approximately 1000
years.
Patch also fixes overflow when converting `u64` to `time_t`.
`Stdio` now implements `From<ChildStdin>`, `From<ChildStdout>`,
`From<ChildStderr>`, and `From<File>`.
The `Command::stdin`/`stdout`/`stderr` methods now take any type that
implements `Into<Stdio>`.
This makes it much easier to write shell-like command chains, piping to
one another and redirecting to and from files. Otherwise one would need
to use the unsafe and OS-specific `from_raw_fd` or `from_raw_handle`.
The implementation of mx_job_default changed from a macro which
accessed the __magenta_job_default global variable to a proper
function call. This patch tracks that change.
Gecko recently had a bug reported [1] with a deadlock in the Rust TLS
implementation for Windows. TLS destructors are implemented in a sort of ad-hoc
fashion on Windows as it doesn't natively support destructors for TLS keys. To
work around this the runtime manages a list of TLS destructors and registers a
hook to get run whenever a thread exits. When a thread exits it takes a look at
the list and runs all destructors.
Unfortunately it turns out that there's a lock which is held when our "at thread
exit" callback is run. The callback then attempts to acquire a lock protecting
the list of TLS destructors. Elsewhere in the codebase while we hold a lock over
the TLS destructors we try to acquire the same lock held first before our
special callback is run. And as a result, deadlock!
This commit sidesteps the issue with a few small refactorings:
* Removed support for destroying a TLS key on Windows. We don't actually ever
exercise this as a public-facing API, and it's only used during `lazy_init`
during racy situations. To handle that we just synchronize `lazy_init`
globally on Windows so we never have to call `destroy`.
* With no need to support removal the global synchronized `Vec` was tranformed
to a lock-free linked list. With the removal of locks this means that
iteration no long requires a lock and as such we won't run into the deadlock
problem mentioned above.
Note that it's still a general problem that you have to be extra super careful
in TLS destructors. For example no code which runs a TLS destructor on Windows
can call back into the Windows API to do a dynamic library lookup. Unfortunately
I don't know of a great way around that, but this at least fixes the immediate
problem that Gecko was seeing which is that with "well behaved" destructors the
system would still deadlock!
[1]: https://bugzilla.mozilla.org/show_bug.cgi?id=1358151
std: Use `poll` instead of `select`
This gives us the benefit of supporting file descriptors over the limit that
select supports, which...
Closes#40894
This reverts commit 2cf686f2cd (#40842)
RawFd is a type alias for c_int, which is itself a type alias for i32.
As a result, adding AsRawFd and IntoRawFd impls for RawFd actually adds
them for i32.
As a result, the reverted commit makes this valid:
```
use std::os::unix::io::AsRawFd;
fn arf<T: AsRawFd>(_: T) {}
fn main() {
arf(32i32)
}
```
Implimenting AsRawFd and IntoRawFd for i32 breaks the promises of both
those traits that their methods return a valid RawFd.
r? @aturon
cc @Mic92 @kamalmarhubi
Implement AsRawFd/IntoRawFd for RawFd
This is useful to build os abstraction like the nix crate does.
It allows to define functions, which accepts generic arguments
of data structures convertible to RawFd, including RawFd itself.
For example:
```
fn write<FD: AsRawFd>(fd: FD, buf: &[u8]) -> Result<usize>
write(file, buf);
```
instead of:
```
fn write(fd: RawFd, buf: &[u8]) -> Result<usize>
write(file.as_raw_fd(), buf);
```
cc @kamalmarhubi
This is useful to build os abstraction like the nix crate does.
It allows to define functions, which accepts generic arguments
of data structures convertible to RawFd, including RawFd itself.
For example:
fn write<FD: AsRawFd>(fd: FD, buf: &[u8]) -> Result<usize>
instead of:
fn write(fd: RawFd, buf: &[u8]) -> Result<usize>
write(foo.as_raw_fd(), buf);
Leftovers from #39594; From<Box> impls
These are a few more impls that follow the same reasoning as those from #39594.
What's included:
* `From<Box<str>> for String`
* `From<Box<[T]>> for Vec<T>`
* `From<Box<CStr>> for CString`
* `From<Box<OsStr>> for OsString`
* `From<Box<Path>> for PathBuf`
* `Into<Box<str>> for String`
* `Into<Box<[T]>> for Vec<T>`
* `Into<Box<CStr>> for CString`
* `Into<Box<OsStr>> for OsString`
* `Into<Box<Path>> for PathBuf`
* `<Box<CStr>>::into_c_string`
* `<Box<OsStr>>::into_os_string`
* `<Box<Path>>::into_path_buf`
* Tracking issue for latter three methods + three from previous PR.
Currently, the opposite direction isn't doable with `From` (only `Into`) because of the separation between `liballoc` and `libcollections`. I'm holding off on those for a later PR.
std::process for fuchsia: updated to latest liblaunchpad
Our liblaunchpad changed a bit and so fuchsia's std::process impl needs to change a bit.
@raphlinus
Improve backtrace formating while panicking.
Fixes#37783.
Done:
- Fix alignment of file paths for better readability
- `RUST_BACKTRACE=full` prints all the informations (current behaviour)
- `RUST_BACKTRACE=(short|yes)` is the default and does:
- Skip irrelevant frames at the beginning and the end
- Remove function address
- Remove the current directory from the absolute paths
- Remove `::hfabe6541873` at the end of the symbols
- `RUST_BACKTRACE=(0|no)` disables the backtrace.
- `RUST_BACKTRACE=<everything else>` is equivalent to `short` for
backward compatibility.
- doc
- More uniform printing across platforms.
Removed, TODO in a new PR:
- Remove path prefix for libraries and libstd
Example of short backtrace:
```rust
fn fail() {
panic!();
}
fn main() {
let closure = || fail();
closure();
}
```
Short:
```
thread 'main' panicked at 'explicit panic', t.rs:2
Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
0: t::fail
at ./t.rs:2
1: t::main::{{closure}}
at ./t.rs:6
2: t::main
at ./t.rs:7
```
Full:
```
thread 'main' panicked at 'This function never returns!', t.rs:2
stack backtrace:
0: 0x558ddf666478 - std::sys:👿:backtrace::tracing:👿:unwind_backtrace::hec84c9dd8389cc5d
at /home/yamakaky/dev/rust/rust/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
1: 0x558ddf65d90e - std::sys_common::backtrace::_print::hfa25f8b31f4b4353
at /home/yamakaky/dev/rust/rust/src/libstd/sys_common/backtrace.rs:71
2: 0x558ddf65cb5e - std::sys_common::backtrace::print::h9b711e11ac3ba805
at /home/yamakaky/dev/rust/rust/src/libstd/sys_common/backtrace.rs:60
3: 0x558ddf66796e - std::panicking::default_hook::{{closure}}::h736d216e74748044
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:355
4: 0x558ddf66743c - std::panicking::default_hook::h16baff397e46ea10
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:371
5: 0x558ddf6682bc - std::panicking::rust_panic_with_hook::h6d5a9bb4eca42c80
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:559
6: 0x558ddf64ea93 - std::panicking::begin_panic::h17dc549df2f10b99
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:521
7: 0x558ddf64ec42 - t::diverges::he6bc43fc925905f5
at /tmp/p/t.rs:2
8: 0x558ddf64ec5a - t::main::h0ffc20356b8a69c0
at /tmp/p/t.rs:6
9: 0x558ddf6687f5 - core::ops::FnOnce::call_once::hce41f19c0db56f93
10: 0x558ddf667cde - std::panicking::try::do_call::hd4c8c97efb4291df
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:464
11: 0x558ddf698d77 - __rust_try
12: 0x558ddf698c57 - __rust_maybe_catch_panic
at /home/yamakaky/dev/rust/rust/src/libpanic_unwind/lib.rs:98
13: 0x558ddf667adb - std::panicking::try::h2c56ed2a59ec1d12
at /home/yamakaky/dev/rust/rust/src/libstd/panicking.rs:440
14: 0x558ddf66cc9a - std::panic::catch_unwind::h390834e0251cc9af
at /home/yamakaky/dev/rust/rust/src/libstd/panic.rs:361
15: 0x558ddf6809ee - std::rt::lang_start::hb73087428e233982
at /home/yamakaky/dev/rust/rust/src/libstd/rt.rs:57
16: 0x558ddf64ec92 - main
17: 0x7fecb869e290 - __libc_start_main
18: 0x558ddf64e8b9 - _start
19: 0x0 - <unknown>
```
Switch Fuchsia to readdir (instead of readdir_r)
The readdir_r function is deprecated on newer Posix systems because of
various problems, and not implemented at all for Fuchsia. There are
already implementations using both, and this patch switches Fuchsia
over to the readdir-based one.
Fixes#40021 for Fuchsia, but that issue also contains discussion of
what should happen for other Posix systems.
Follow rename of mx_handle_wait Magenta syscalls
The mx_handle_wait_* syscalls in Magenta were renamed to
mx_object_wait. The syscall is used in the Magenta/Fuchsia
implementation of std::process, to wait on child processes.
In addition, this patch enables the use of the system provided
libbacktrace library on Fuchsia targets. Symbolization is not yet
working, but at least it allows printing hex addresses in a backtrace
and makes building succeed when the backtrace feature is not disabled.
The readdir_r function is deprecated on newer Posix systems because of
various problems, and not implemented at all for Fuchsia. There are
already implementations using both, and this patch switches Fuchsia
over to the readdir-based one.
Fixes#40021 for Fuchsia, but that issue also contains discussion of
what should happen for other Posix systems.
The mx_handle_wait_* syscalls in Magenta were renamed to
mx_object_wait. The syscall is used in the Magenta/Fuchsia
implementation of std::process, to wait on child processes.
In addition, this patch enables the use of the system provided
libbacktrace library on Fuchsia targets. Symbolization is not yet
working, but at least it allows printing hex addresses in a backtrace
and makes building succeed when the backtrace feature is not disabled.
- `RUST_BACKTRACE=full` prints all the informations (old behaviour)
- `RUST_BACKTRACE=(0|no)` disables the backtrace.
- `RUST_BACKTRACE=<everything else>` (including `1`) shows a simplified
backtrace, without the function addresses and with cleaned filenames
and symbols. Also removes some unneded frames at the beginning and the
end.
Fixes#37783.
PR is #38165.
make Child::try_wait return io::Result<Option<ExitStatus>>
This is much nicer for callers who want to short-circuit real I/O errors
with `?`, because they can write this
if let Some(status) = foo.try_wait()? {
...
} else {
...
}
instead of this
match foo.try_wait() {
Ok(status) => {
...
}
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {
...
}
Err(err) => return Err(err),
}
The original design of `try_wait` was patterned after the `Read` and
`Write` traits, which support both blocking and non-blocking
implementations in a single API. But since `try_wait` is never blocking,
it makes sense to optimize for the non-blocking case.
Tracking issue: https://github.com/rust-lang/rust/issues/38903
Delete the makefile build system
This PR deletes the makefile build system in favor of the rustbuild build system. The beta has now been branched so 1.16 will continue to be buildable from the makefiles, but going forward 1.17 will only be buildable with rustbuild.
Rustbuild has been the default build system [since 1.15.0](https://github.com/rust-lang/rust/pull/37817) and the makefiles were [proposed for deletion](https://internals.rust-lang.org/t/proposal-for-promoting-rustbuild-to-official-status/4368) at this time back in November of last year.
And now with the deletion of these makefiles we can start getting those sweet sweet improvements of using crates.io crates in the compiler!
Add support for test suites emulated in QEMU
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
This is much nicer for callers who want to short-circuit real I/O errors
with `?`, because they can write this
if let Some(status) = foo.try_wait()? {
...
} else {
...
}
instead of this
match foo.try_wait() {
Ok(status) => {
...
}
Err(err) if err.kind() == io::ErrorKind::WouldBlock => {
...
}
Err(err) => return Err(err),
}
The original design of `try_wait` was patterned after the `Read` and
`Write` traits, which support both blocking and non-blocking
implementations in a single API. But since `try_wait` is never blocking,
it makes sense to optimize for the non-blocking case.
Tracking issue: https://github.com/rust-lang/rust/issues/38903
Use less syscalls in `FileDesc::set_{nonblocking,cloexec}`
Only set the flags if they differ from what the OS reported, use
`FIONBIO` to atomically set the non-blocking IO flag on Linux.
Add peek APIs to std::net
Adds "peek" APIs to `std::net` sockets, including:
- `UdpSocket.peek()`
- `UdpSocket.peek_from()`
- `TcpStream.peek()`
These methods enable socket reads without side-effects. That is, repeated calls to `peek()` return identical data. This is accomplished by providing the POSIX flag `MSG_PEEK` to the underlying socket read operations.
This also moves the current implementation of `recv_from` out of the platform-independent `sys_common` and into respective `sys/windows` and `sys/unix` implementations. This allows for more platform-dependent implementations where necessary.
Fixes#38980
Expand documentation of process::exit and exec
Show a conventional way to use process::exit when destructors are considered important and also
mention that the same caveats wrt destructors apply to exec as well.
These methods enable socket reads without side-effects. That is,
repeated calls to peek() return identical data. This is accomplished
by providing the POSIX flag MSG_PEEK to the underlying socket read
operations.
This also moves the current implementation of recv_from out of the
platform-independent sys_common and into respective sys/windows and
sys/unix implementations. This allows for more platform-dependent
implementations.
Use less syscalls in `anon_pipe()`
Save a `ENOSYS` failure from `pipe2` and don't try again.
Use `cvt` instead of `cvt_r` for `pipe2` - `EINTR` is not an error
`pipe2` can return.
This commit adds support to the build system to execute test suites that cannot
run natively but can instead run inside of a QEMU emulator. A proof-of-concept
builder was added for the `arm-unknown-linux-gnueabihf` target to show off how
this might work.
In general the architecture is to have a server running inside of the emulator
which a local client connects to. The protocol between the server/client
supports compiling tests on the host and running them on the target inside the
emulator.
Closes#33114
Make backtraces work on Windows GNU targets again.
This is done by adding a function that can return a filename
to pass to backtrace_create_state. The filename is obtained in
a safe way by first getting the filename, locking the file so it can't
be moved, and then getting the filename again and making sure it's the same.
See: https://github.com/rust-lang/rust/pull/37359#issuecomment-260123399
Issue: #33985
Note though that this isn't that pretty...
I had to implement a `WideCharToMultiByte` wrapper function to convert to the ANSI code page. This will work better than only allowing ASCII provided that the ANSI code page is set to the user's local language, which is often the case.
Also, please make sure that I didn't break the Unix build.
This is done by adding a function that can return a filename
to pass to backtrace_create_state. The filename is obtained in
a safe way by first getting the filename, locking the file so it can't
be moved, and then getting the filename again and making sure it's the same.
See: https://github.com/rust-lang/rust/pull/37359#issuecomment-260123399
Issue: #33985
Show a conventional way to use process::exit when destructors are considered important and also
mention that the same caveats wrt destructors apply to exec as well.
This commit adds a new method to the `Child` type in the `std::process` module
called `try_wait`. This method is the same as `wait` except that it will not
block the calling thread and instead only attempt to collect the exit status. On
Unix this means that we call `waitpid` with the `WNOHANG` flag and on Windows it
just means that we pass a 0 timeout to `WaitForSingleObject`.
Currently it's possible to build this method out of tree, but it's unfortunately
tricky to do so. Specifically on Unix you essentially lose ownership of the pid
for the process once a call to `waitpid` has succeeded. Although `Child` tracks
this state internally to be resilient to multiple calls to `wait` or a `kill`
after a successful wait, if the child is waited on externally then the state
inside of `Child` is not updated. This means that external implementations of
this method must be extra careful to essentially not use a `Child`'s methods
after a call to `waitpid` has succeeded (even in a nonblocking fashion).
By adding this functionality to the standard library it should help canonicalize
these external implementations and ensure they can continue to robustly reuse
the `Child` type from the standard library without worrying about pid ownership.
Turns out that even though all these functions take a `size_t` they don't
actually work that well with anything larger than the maximum value of
`ssize_t`, the return value. Furthermore it looks like OSX rejects any
read/write requests larger than `INT_MAX - 1`. Handle all these cases by just
clamping the maximum size of a read/write on Unix to a platform-specific value.
Closes#38590
Removes magenta build warning.
Small bug fix to remove an unused type in the magenta process code that causes build failures for magenta's rustc.
r? @alexcrichton
@tedsta @raphlinus
Fuchsia support for std::process via liblaunchpad.
Now we can launch processes on Fuchsia via the Rust standard library! ... Mostly.
Right now, ~5% of the time, reading the stdout/stderr off the pipes will fail. Some Magenta kernel people think it's probably a bug in Magenta's pipes. I wrote a unit test that demonstrates the issue in C, which I was told will expedite a fix. https://fuchsia-review.googlesource.com/#/c/15628/
Hopefully this can get merged once the issue is fixed :)
@raphlinus
On unix like systems, the underlying file corresponding to any given path may
change at any time. This function makes it possible to set the permissions of
the a file corresponding to a `File` object even if its path changes.
The mx_cprng_draw syscall has changed signature to separate the status
and size return values, rather than multiplexing them into a single
value with errors interpreted as a negative value. This patch tracks
that change.
The DirEntryExt::ino() implementation was omitted from the first
iteration of this patch, because a dependency needed to be
configured. The fix is straightforward enough.
It is good practice to implement Debug for public types, and
indicating what directory you're reading seems useful.
Signed-off-by: David Henningsson <diwic@ubuntu.com>
Implement `read_offset` and `write_offset`
These functions allow to read from and write to a file from multiple
threads without changing the per-file cursor, avoiding the race between
the seek and the read.
