WriteConsoleW can fail if called with a large buffer so we need to slice
any stdout/stderr output.
However the current slicing has a few problems:
1. It slices by byte but still expects valid UTF-8.
2. The slicing happens even when not outputting to a console.
3. panic! output is not sliced.
This fixes these issues by moving the slice to right before
WriteConsoleW and slicing on a char boundary.
This pushes the implementation detail of proxying `read_to_end` through to
`read_to_end_uninitialized` all the way down to the `FileDesc` and `Handle`
implementations on Unix/Windows. This way intermediate layers will also be able
to take advantage of this optimized implementation.
This commit also adds the optimized implementation for `ChildStdout` and
`ChildStderr`.
On all platforms, reading from stdin where the actual stdin isn't present should
return 0 bytes as having been read rather than the entire buffer.
On Windows, handle the case where we're inheriting stdio handles but one of them
isn't present. Currently the behavior is to fail returning an I/O error but
instead this commit corrects it to detecting this situation and propagating the
non-set handle.
Closes#31167
* Delete `sys::unix::{c, sync}` as these are now all folded into libc itself
* Update all references to use `libc` as a result.
* Update all references to the new flat namespace.
* Moves all windows bindings into sys::c
Currently if a print happens while a thread is being torn down it may cause a
panic if the LOCAL_STDOUT TLS slot has been destroyed by that point. This adds a
guard to check and prints to the process stdout if that's the case (as we do for
if the slot is already borrowed).
Closes#29488
Closes#25977
The various `stdfoo_raw` methods in std::io now return `io::Result`s,
since they may not exist on Windows. They will always return `Ok` on
Unix-like platforms.
[breaking-change]
An automated script was run against the `.rs` and `.md` files,
subsituting every occurrence of `task` with `thread`. In the `.rs`
files, only the texts in the comment blocks were affected.
Changes the style guidelines regarding unit tests to recommend using a
sub-module named "tests" instead of "test" for unit tests as "test"
might clash with imports of libtest.
write_fmt calls write for each formatted field. The default implementation of write_fmt is used,
which will call write on not-yet-locked stdout (and write locking after), therefore making print!
in multithreaded environment still interleave contents of two separate prints.
This patch implements reentrant mutexes, changes stdio handles to use these mutexes and overrides
write_fmt to lock the stdio handle for the whole duration of the call.
Previously a panic was generated for recursive prints due to a double-borrow of
a `RefCell`. This was solved by the second borrow's output being directed
towards the global stdout instead of the per-thread stdout (still experimental
functionality).
After this functionality was altered, however, recursive prints still deadlocked
due to the overridden `write_fmt` method which locked itself first and then
wrote all the data. This was fixed by removing the override of the `write_fmt`
method. This means that unlocked usage of `write!` on a `Stdout`/`Stderr` may be
slower due to acquiring more locks, but it's easy to make more performant with a
call to `.lock()`.
Closes#23781
r? @alexcrichton or @aturon
This still needs to somehow figure out how to avoid unstable warnings arising from the use of unstable functions. I tried to use `#[allow_internal_unstable]` but it still spits out warnings as far as I can see. @huonw (I think you implemented it) does `#[allow_internal_unstable]` not work for some reason or am I using it incorrectly?
std::io does not currently expose the stdin_raw, stdout_raw, or
stderr_raw functions. According to the current plans for stdio (see RFC
#517), raw access will likely be provided using the platform-specific
std::os::{unix,windows} modules. At the moment we don't expose any way
to do this. As such, delete all mention of the _raw functions from the
stdin/stdout/stderr function documentation.
While we're at it, remove a few `pub`s from items that aren't exposed.
This is done just to lessen the confusion experienced by anyone who
looks at the source in an attempt to find the _raw functions.
The new `std::io` module has had some time to bake now, and this commit
stabilizes its functionality. There are still portions of the module which
remain unstable, and below contains a summart of the actions taken.
This commit also deprecates the entire contents of the `old_io` module in a
blanket fashion. All APIs should now have a reasonable replacement in the
new I/O modules.
Stable APIs:
* `std::io` (the name)
* `std::io::prelude` (the name)
* `Read`
* `Read::read`
* `Read::{read_to_end, read_to_string}` after being modified to return a `usize`
for the number of bytes read.
* `Write`
* `Write::write`
* `Write::{write_all, write_fmt}`
* `BufRead`
* `BufRead::{fill_buf, consume}`
* `BufRead::{read_line, read_until}` after being modified to return a `usize`
for the number of bytes read.
* `BufReader`
* `BufReader::{new, with_capacity}`
* `BufReader::{get_ref, get_mut, into_inner}`
* `{Read,BufRead} for BufReader`
* `BufWriter`
* `BufWriter::{new, with_capacity}`
* `BufWriter::{get_ref, get_mut, into_inner}`
* `Write for BufWriter`
* `IntoInnerError`
* `IntoInnerError::{error, into_inner}`
* `{Error,Display} for IntoInnerError`
* `LineWriter`
* `LineWriter::{new, with_capacity}` - `with_capacity` was added
* `LineWriter::{get_ref, get_mut, into_inner}` - `get_mut` was added)
* `Write for LineWriter`
* `BufStream`
* `BufStream::{new, with_capacities}`
* `BufStream::{get_ref, get_mut, into_inner}`
* `{BufRead,Read,Write} for BufStream`
* `stdin`
* `Stdin`
* `Stdin::lock`
* `Stdin::read_line` - added method
* `StdinLock`
* `Read for Stdin`
* `{Read,BufRead} for StdinLock`
* `stdout`
* `Stdout`
* `Stdout::lock`
* `StdoutLock`
* `Write for Stdout`
* `Write for StdoutLock`
* `stderr`
* `Stderr`
* `Stderr::lock`
* `StderrLock`
* `Write for Stderr`
* `Write for StderrLock`
* `io::Result`
* `io::Error`
* `io::Error::last_os_error`
* `{Display, Error} for Error`
Unstable APIs:
(reasons can be found in the commit itself)
* `Write::flush`
* `Seek`
* `ErrorKind`
* `Error::new`
* `Error::from_os_error`
* `Error::kind`
Deprecated APIs
* `Error::description` - available via the `Error` trait
* `Error::detail` - available via the `Display` implementation
* `thread::Builder::{stdout, stderr}`
Changes in functionality:
* `old_io::stdio::set_stderr` is now a noop as the infrastructure for printing
backtraces has migrated to `std::io`.
* The `ReadExt`, `WriteExt`, and `BufReadExt` extension traits were all removed
by folding functionality into the corresponding trait.
[breaking-change]
Being a person who somehow has taken a liking to premature optimisation, my knee-jerk reaction to
locking in std handles was preamble resembling following snippet:
let stdout = stdout();
let lstdout = stdout.lock();
let stdin = stdin();
let lstdin = stdin.lock();
and then reading from the locked handle like this:
let mut letter = [0; 1];
lstdin.read(&mut letter).unwrap();
As it is now this code will deadlock because the `read` method attempts to lock stdout as well!
This is an implementation of RFC 899 and adds stdio functionality to the new
`std::io` module. Details of the API can be found on the RFC, but from a high
level:
* `io::{stdin, stdout, stderr}` constructors are now available. There are also
`*_raw` variants for unbuffered and unlocked access.
* All handles are globally shared (excluding raw variants).
* The stderr handle is no longer buffered.
* All handles can be explicitly locked (excluding the raw variants).
The `print!` and `println!` machinery has not yet been hooked up to these
streams just yet. The `std::fmt::output` module has also not yet been
implemented as part of this commit.
In preparation for the I/O rejuvination of the standard library, this commit
renames the current `io` module to `old_io` in order to make room for the new
I/O modules. It is expected that the I/O RFCs will land incrementally over time
instead of all at once, and this provides a fresh clean path for new modules to
enter into as well as guaranteeing that all old infrastructure will remain in
place for some time.
As each `old_io` module is replaced it will be deprecated in-place for new
structures in `std::{io, fs, net}` (as appropriate).
This commit does *not* leave a reexport of `old_io as io` as the deprecation
lint does not currently warn on this form of use. This is quite a large breaking
change for all imports in existing code, but all functionality is retained
precisely as-is and path statements simply need to be renamed from `io` to
`old_io`.
[breaking-change]
fmt::Show is for debugging, and can and should be implemented for
all public types. This trait is used with `{:?}` syntax. There still
exists #[derive(Show)].
fmt::String is for types that faithfully be represented as a String.
Because of this, there is no way to derive fmt::String, all
implementations must be purposeful. It is used by the default format
syntax, `{}`.
This will break most instances of `{}`, since that now requires the type
to impl fmt::String. In most cases, replacing `{}` with `{:?}` is the
correct fix. Types that were being printed specifically for users should
receive a fmt::String implementation to fix this.
Part of #20013
[breaking-change]
This commit is an implementation of [RFC 503][rfc] which is a stabilization
story for the prelude. Most of the RFC was directly applied, removing reexports.
Some reexports are kept around, however:
* `range` remains until range syntax has landed to reduce churn.
* `Path` and `GenericPath` remain until path reform lands. This is done to
prevent many imports of `GenericPath` which will soon be removed.
* All `io` traits remain until I/O reform lands so imports can be rewritten all
at once to `std::io::prelude::*`.
This is a breaking change because many prelude reexports have been removed, and
the RFC can be consulted for the exact list of removed reexports, as well as to
find the locations of where to import them.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/0503-prelude-stabilization.md
[breaking-change]
Closes#20068
This pass performs a second pass of stabilization through the `std::sync`
module, avoiding modules/types that are being handled in other PRs (e.g.
mutexes, rwlocks, condvars, and channels).
The following items are now stable
* `sync::atomic`
* `sync::atomic::ATOMIC_BOOL_INIT` (was `INIT_ATOMIC_BOOL`)
* `sync::atomic::ATOMIC_INT_INIT` (was `INIT_ATOMIC_INT`)
* `sync::atomic::ATOMIC_UINT_INIT` (was `INIT_ATOMIC_UINT`)
* `sync::Once`
* `sync::ONCE_INIT`
* `sync::Once::call_once` (was `doit`)
* C == `pthread_once(..)`
* Boost == `call_once(..)`
* Windows == `InitOnceExecuteOnce`
* `sync::Barrier`
* `sync::Barrier::new`
* `sync::Barrier::wait` (now returns a `bool`)
* `sync::Semaphore::new`
* `sync::Semaphore::acquire`
* `sync::Semaphore::release`
The following items remain unstable
* `sync::SemaphoreGuard`
* `sync::Semaphore::access` - it's unclear how this relates to the poisoning
story of mutexes.
* `sync::TaskPool` - the semantics of a failing task and whether a thread is
re-attached to a thread pool are somewhat unclear, and the
utility of this type in `sync` is question with respect to
the jobs of other primitives. This type will likely become
stable or move out of the standard library over time.
* `sync::Future` - futures as-is have yet to be deeply re-evaluated with the
recent core changes to Rust's synchronization story, and will
likely become stable in the future but are unstable until
that time comes.
[breaking-change]
The new semantics of this function are that the callbacks are run when the *main
thread* exits, not when all threads have exited. This implies that other threads
may still be running when the `at_exit` callbacks are invoked and users need to
be prepared for this situation.
Users in the standard library have been audited in accordance to these new rules
as well.
Closes#20012
This commit is a second pass stabilization for the `std::comm` module,
performing the following actions:
* The entire `std::comm` module was moved under `std::sync::mpsc`. This movement
reflects that channels are just yet another synchronization primitive, and
they don't necessarily deserve a special place outside of the other
concurrency primitives that the standard library offers.
* The `send` and `recv` methods have all been removed.
* The `send_opt` and `recv_opt` methods have been renamed to `send` and `recv`.
This means that all send/receive operations return a `Result` now indicating
whether the operation was successful or not.
* The error type of `send` is now a `SendError` to implement a custom error
message and allow for `unwrap()`. The error type contains an `into_inner`
method to extract the value.
* The error type of `recv` is now `RecvError` for the same reasons as `send`.
* The `TryRecvError` and `TrySendError` types have had public reexports removed
of their variants and the variant names have been tweaked with enum
namespacing rules.
* The `Messages` iterator is renamed to `Iter`
This functionality is now all `#[stable]`:
* `Sender`
* `SyncSender`
* `Receiver`
* `std::sync::mpsc`
* `channel`
* `sync_channel`
* `Iter`
* `Sender::send`
* `Sender::clone`
* `SyncSender::send`
* `SyncSender::try_send`
* `SyncSender::clone`
* `Receiver::recv`
* `Receiver::try_recv`
* `Receiver::iter`
* `SendError`
* `RecvError`
* `TrySendError::{mod, Full, Disconnected}`
* `TryRecvError::{mod, Empty, Disconnected}`
* `SendError::into_inner`
* `TrySendError::into_inner`
This is a breaking change due to the modification of where this module is
located, as well as the changing of the semantics of `send` and `recv`. Most
programs just need to rename imports of `std::comm` to `std::sync::mpsc` and
add calls to `unwrap` after a send or a receive operation.
[breaking-change]
All of the current std::sync primitives have poisoning enable which means that
when a task fails inside of a write-access lock then all future attempts to
acquire the lock will fail. This strategy ensures that stale data whose
invariants are possibly not upheld are never viewed by other tasks to help
propagate unexpected panics (bugs in a program) among tasks.
Currently there is no way to test whether a mutex or rwlock is poisoned. One
method would be to duplicate all the methods with a sister foo_catch function,
for example. This pattern is, however, against our [error guidelines][errors].
As a result, this commit exposes the fact that a task has failed internally
through the return value of a `Result`.
[errors]: https://github.com/rust-lang/rfcs/blob/master/text/0236-error-conventions.md#do-not-provide-both-result-and-fail-variants
All methods now return a `LockResult<T>` or a `TryLockResult<T>` which
communicates whether the lock was poisoned or not. In a `LockResult`, both the
`Ok` and `Err` variants contains the `MutexGuard<T>` that is being returned in
order to allow access to the data if poisoning is not desired. This also means
that the lock is *always* held upon returning from `.lock()`.
A new type, `PoisonError`, was added with one method `into_guard` which can
consume the assertion that a lock is poisoned to gain access to the underlying
data.
This is a breaking change because the signatures of these methods have changed,
often incompatible ways. One major difference is that the `wait` methods on a
condition variable now consume the guard and return it in as a `LockResult` to
indicate whether the lock was poisoned while waiting. Most code can be updated
by calling `.unwrap()` on the return value of `.lock()`.
[breaking-change]
This commit is an implementation of [RFC 503][rfc] which is a stabilization
story for the prelude. Most of the RFC was directly applied, removing reexports.
Some reexports are kept around, however:
* `range` remains until range syntax has landed to reduce churn.
* `Path` and `GenericPath` remain until path reform lands. This is done to
prevent many imports of `GenericPath` which will soon be removed.
* All `io` traits remain until I/O reform lands so imports can be rewritten all
at once to `std::io::prelude::*`.
This is a breaking change because many prelude reexports have been removed, and
the RFC can be consulted for the exact list of removed reexports, as well as to
find the locations of where to import them.
[rfc]: https://github.com/rust-lang/rfcs/blob/master/text/0503-prelude-stabilization.md
[breaking-change]
Closes#20068
This commit merges the `rustrt` crate into `std`, undoing part of the
facade. This merger continues the paring down of the runtime system.
Code relying on the public API of `rustrt` will break; some of this API
is now available through `std::rt`, but is likely to change and/or be
removed very soon.
[breaking-change]
followed by a semicolon.
This allows code like `vec![1i, 2, 3].len();` to work.
This breaks code that uses macros as statements without putting
semicolons after them, such as:
fn main() {
...
assert!(a == b)
assert!(c == d)
println(...);
}
It also breaks code that uses macros as items without semicolons:
local_data_key!(foo)
fn main() {
println("hello world")
}
Add semicolons to fix this code. Those two examples can be fixed as
follows:
fn main() {
...
assert!(a == b);
assert!(c == d);
println(...);
}
local_data_key!(foo);
fn main() {
println("hello world")
}
RFC #378.
Closes#18635.
[breaking-change]
This commit collapses the various prelude traits for slices into just one trait:
* SlicePrelude/SliceAllocPrelude => SliceExt
* CloneSlicePrelude/CloneSliceAllocPrelude => CloneSliceExt
* OrdSlicePrelude/OrdSliceAllocPrelude => OrdSliceExt
* PartialEqSlicePrelude => PartialEqSliceExt
This commit collapses the various prelude traits for slices into just one trait:
* SlicePrelude/SliceAllocPrelude => SliceExt
* CloneSlicePrelude/CloneSliceAllocPrelude => CloneSliceExt
* OrdSlicePrelude/OrdSliceAllocPrelude => OrdSliceExt
* PartialEqSlicePrelude => PartialEqSliceExt
Now that we have an overloaded comparison (`==`) operator, and that `Vec`/`String` deref to `[T]`/`str` on method calls, many `as_slice()`/`as_mut_slice()`/`to_string()` calls have become redundant. This patch removes them. These were the most common patterns:
- `assert_eq(test_output.as_slice(), "ground truth")` -> `assert_eq(test_output, "ground truth")`
- `assert_eq(test_output, "ground truth".to_string())` -> `assert_eq(test_output, "ground truth")`
- `vec.as_mut_slice().sort()` -> `vec.sort()`
- `vec.as_slice().slice(from, to)` -> `vec.slice(from_to)`
---
Note that e.g. `a_string.push_str(b_string.as_slice())` has been left untouched in this PR, since we first need to settle down whether we want to favor the `&*b_string` or the `b_string[]` notation.
This is rebased on top of #19167
cc @alexcrichton @aturon
In regards to:
https://github.com/rust-lang/rust/issues/19253#issuecomment-64836729
This commit:
* Changes the #deriving code so that it generates code that utilizes fewer
reexports (in particur Option::* and Result::*), which is necessary to
remove those reexports in the future
* Changes other areas of the codebase so that fewer reexports are utilized
io::stdin returns a new `BufferedReader` each time it's called, which
results in some very confusing behavior with disappearing output. It now
returns a `StdinReader`, which wraps a global singleton
`Arc<Mutex<BufferedReader<StdReader>>`. `Reader` is implemented directly
on `StdinReader`. However, `Buffer` is not, as the `fill_buf` method is
fundamentaly un-thread safe. A `lock` method is defined on `StdinReader`
which returns a smart pointer wrapping the underlying `BufferedReader`
while guaranteeing mutual exclusion.
Code that treats the return value of io::stdin as implementing `Buffer`
will break. Add a call to `lock`:
```rust
io::stdin().lines()
// =>
io::stdin().lock().lines()
```
Closes#14434
[breaking-change]
This commit removes the `std::local_data` module in favor of a new
`std::thread_local` module providing thread local storage. The module provides
two variants of TLS: one which owns its contents and one which is based on
scoped references. Each implementation has pros and cons listed in the
documentation.
Both flavors have accessors through a function called `with` which yield a
reference to a closure provided. Both flavors also panic if a reference cannot
be yielded and provide a function to test whether an access would panic or not.
This is an implementation of [RFC 461][rfc] and full details can be found in
that RFC.
This is a breaking change due to the removal of the `std::local_data` module.
All users can migrate to the new thread local system like so:
thread_local!(static FOO: Rc<RefCell<Option<T>>> = Rc::new(RefCell::new(None)))
The old `local_data` module inherently contained the `Rc<RefCell<Option<T>>>` as
an implementation detail which must now be explicitly stated by users.
[rfc]: https://github.com/rust-lang/rfcs/pull/461
[breaking-change]
Previously, the entire runtime API surface was publicly exposed, but
that is neither necessary nor desirable. This commit hides most of the
module, using librustrt directly as needed. The arrangement will need to
be revisited when rustrt is pulled into std.
[breaking-change]
This breaks code that referred to variant names in the same namespace as
their enum. Reexport the variants in the old location or alter code to
refer to the new locations:
```
pub enum Foo {
A,
B
}
fn main() {
let a = A;
}
```
=>
```
pub use self::Foo::{A, B};
pub enum Foo {
A,
B
}
fn main() {
let a = A;
}
```
or
```
pub enum Foo {
A,
B
}
fn main() {
let a = Foo::A;
}
```
[breaking-change]
This patch continues runtime removal by moving the tty implementations
into `sys`.
Because this eliminates APIs in `libnative` and `librustrt`, it is a:
[breaking-change]
This functionality is likely to be available publicly, in some form,
from `std` in the future.
This moves the filesystem implementation from libnative into the new
`sys` modules, refactoring along the way and hooking into `std::io::fs`.
Because this eliminates APIs in `libnative` and `librustrt`, it is a:
[breaking-change]
This functionality is likely to be available publicly, in some form,
from `std` in the future.
This commit renames a number of extension traits for slices and string
slices, now that they have been refactored for DST. In many cases,
multiple extension traits could now be consolidated. Further
consolidation will be possible with generalized where clauses.
The renamings are consistent with the [new `-Prelude`
suffix](https://github.com/rust-lang/rfcs/pull/344). There are probably
a few more candidates for being renamed this way, but that is left for
API stabilization of the relevant modules.
Because this renames traits, it is a:
[breaking-change]
However, I do not expect any code that currently uses the standard
library to actually break.
Closes#17917
https://github.com/rust-lang/rfcs/pull/221
The current terminology of "task failure" often causes problems when
writing or speaking about code. You often want to talk about the
possibility of an operation that returns a Result "failing", but cannot
because of the ambiguity with task failure. Instead, you have to speak
of "the failing case" or "when the operation does not succeed" or other
circumlocutions.
Likewise, we use a "Failure" header in rustdoc to describe when
operations may fail the task, but it would often be helpful to separate
out a section describing the "Err-producing" case.
We have been steadily moving away from task failure and toward Result as
an error-handling mechanism, so we should optimize our terminology
accordingly: Result-producing functions should be easy to describe.
To update your code, rename any call to `fail!` to `panic!` instead.
Assuming you have not created your own macro named `panic!`, this
will work on UNIX based systems:
grep -lZR 'fail!' . | xargs -0 -l sed -i -e 's/fail!/panic!/g'
You can of course also do this by hand.
[breaking-change]
compiletest: compact "linux" "macos" etc.as "unix".
liballoc: remove a superfluous "use".
libcollections: remove invocations of deprecated methods in favor of
their suggested replacements and use "_" for a loop counter.
libcoretest: remove invocations of deprecated methods; also add
"allow(deprecated)" for testing a deprecated method itself.
libglob: use "cfg_attr".
libgraphviz: add a test for one of data constructors.
libgreen: remove a superfluous "use".
libnum: "allow(type_overflow)" for type cast into u8 in a test code.
librustc: names of static variables should be in upper case.
libserialize: v[i] instead of get().
libstd/ascii: to_lowercase() instead of to_lower().
libstd/bitflags: modify AnotherSetOfFlags to use i8 as its backend.
It will serve better for testing various aspects of bitflags!.
libstd/collections: "allow(deprecated)" for testing a deprecated
method itself.
libstd/io: remove invocations of deprecated methods and superfluous "use".
Also add #[test] where it was missing.
libstd/num: introduce a helper function to effectively remove
invocations of a deprecated method.
libstd/path and rand: remove invocations of deprecated methods and
superfluous "use".
libstd/task and libsync/comm: "allow(deprecated)" for testing
a deprecated method itself.
libsync/deque: remove superfluous "unsafe".
libsync/mutex and once: names of static variables should be in upper case.
libterm: introduce a helper function to effectively remove
invocations of a deprecated method.
We still see a few warnings about using obsoleted native::task::spawn()
in the test modules for libsync. I'm not sure how I should replace them
with std::task::TaksBuilder and native::task::NativeTaskBuilder
(dependency to libstd?)
Signed-off-by: NODA, Kai <nodakai@gmail.com>
This commit removes the `iotest!` macro from `std::io`. The macro was
primarily used to ensure that all io-related tests were run on both
libnative and libgreen/librustuv. However, now that the librustuv stack
is being removed, the macro is no longer needed.
See the [runtime removal
RFC](https://github.com/rust-lang/rfcs/pull/230) for more context.
[breaking-change]
I've found that 64k is still too much and continue to see the errors as reported
in #14940. I've locally found that 32k fails, and 24k succeeds, so I've trimmed
the size down to 10000 which the included links in the added comment end up
recommending.
It sounds like the limit can still be hit with many threads in play, but I have
yet to reproduce this, so I figure we can wait until that's hit (if it's
possible) and then take action.
