bors cc9bb1522e Auto merge of #83342 - Count-Count:win-console-incomplete-utf8, r=m-ou-se ... Allow writing of incomplete UTF-8 sequences to the Windows console via stdout/stderr # Problem Writes of just an incomplete UTF-8 byte sequence (e.g. `b"\xC3"` or `b"\xF0\x9F"`) to stdout/stderr with a Windows console attached error with `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` even though further writes could complete the codepoint. This is currently a rare occurence since the [linewritershim](2c56ea38b0/library/std/src/io/buffered/linewritershim.rs) implementation flushes complete lines immediately and buffers up to 1024 bytes for incomplete lines. It can still happen as described in #83258. The problem will become more pronounced once the developer can switch stdout/stderr from line-buffered to block-buffered or immediate when the changes in the "Switchable buffering for Stdout" pull request (#78515) get merged. # Patch description If there is at least one valid UTF-8 codepoint all valid UTF-8 is passed through to the extracted `write_valid_utf8_to_console()` fn. The new code only comes into play if `write()` is being passed a short byte slice comprising an incomplete UTF-8 codepoint. In this case up to three bytes are buffered in the `IncompleteUtf8` struct associated with `Stdout` / `Stderr`. The bytes are accepted one at a time. As soon as an error can be detected `io::ErrorKind::InvalidData, "Windows stdio in console mode does not support writing non-UTF-8 byte sequences"` is returned. Once a complete UTF-8 codepoint is received it is passed to the `write_valid_utf8_to_console()` and the buffer length is set to zero. Calling `flush()` will neither error nor write anything if an incomplete codepoint is present in the buffer. # Tests Currently there are no Windows-specific tests for console writing code at all. Writing (regression) tests for this problem is a bit challenging since unit tests and UI tests don't run in a console and suddenly popping up another console window might be surprising to developers running the testsuite and it might not work at all in CI builds. To just test the new functionality in unit tests the code would need to be refactored. Some guidance on how to proceed would be appreciated. # Public API changes * `std::str::verifications::utf8_char_width()` would be exposed as `std::str::utf8_char_width()` behind the "str_internals" feature gate. # Related issues * Fixes #83258. * PR #78515 will exacerbate the problem. # Open questions * Add tests? * Squash into one commit with better commit message?		2021-09-02 03:31:17 +00:00
..
alloc	Rollup merge of #88040 - nbdd0121:btreemap, r=m-ou-se	2021-09-01 09:23:23 +02:00
backtrace@4f925f8d81	Update the backtrace crate in libstd	2021-08-19 07:31:49 -07:00
core	Auto merge of #83342 - Count-Count:win-console-incomplete-utf8, r=m-ou-se	2021-09-02 03:31:17 +00:00
panic_abort	Use C-unwind ABI for __rust_start_panic in panic_abort	2021-08-05 18:01:17 +08:00
panic_unwind	STD support for the ESP-IDF framework	2021-08-10 12:09:00 +03:00
proc_macro	Fix typos “a”→“an”	2021-08-22 15:35:11 +02:00
profiler_builtins	rfc3052: Remove authors field from Cargo manifests	2021-07-29 14:56:05 -07:00
rtstartup	Bump bootstrap compiler to 1.50 beta	2020-12-30 09:27:19 -05:00
rustc-std-workspace-alloc	rfc3052: Remove authors field from Cargo manifests	2021-07-29 14:56:05 -07:00
rustc-std-workspace-core	rfc3052: Remove authors field from Cargo manifests	2021-07-29 14:56:05 -07:00
rustc-std-workspace-std	rfc3052: Remove authors field from Cargo manifests	2021-07-29 14:56:05 -07:00
std	Auto merge of #83342 - Count-Count:win-console-incomplete-utf8, r=m-ou-se	2021-09-02 03:31:17 +00:00
stdarch@89b0e355bc	Update the stdarch submodule	2021-08-24 09:02:44 -07:00
test	rfc3052: Remove authors field from Cargo manifests	2021-07-29 14:56:05 -07:00
unwind	build llvm libunwind.a in rustbuild	2021-08-28 14:14:22 +08:00