rust/library/std
bors d25de31a0e Auto merge of #89165 - jkugelman:read-to-end-overallocation, r=joshtriplett
Fix read_to_end to not grow an exact size buffer

If you know how much data to expect and use `Vec::with_capacity` to pre-allocate a buffer of that capacity, `Read::read_to_end` will still double its capacity. It needs some space to perform a read, even though that read ends up returning `0`.

It's a bummer to carefully pre-allocate 1GB to read a 1GB file into memory and end up using 2GB.

This fixes that behavior by special casing a full buffer and reading into a small "probe" buffer instead. If that read returns `0` then it's confirmed that the buffer was the perfect size. If it doesn't, the probe buffer is appended to the normal buffer and the read loop continues.

Fixing this allows several workarounds in the standard library to be removed:

- `Take` no longer needs to override `Read::read_to_end`.
- The `reservation_size` callback that allowed `Take` to inhibit the previous over-allocation behavior isn't needed.
- `fs::read` doesn't need to reserve an extra byte in `initial_buffer_size`.

Curiously, there was a unit test that specifically checked that `Read::read_to_end` *does* over-allocate. I removed that test, too.
2021-10-04 04:44:56 +00:00
..
benches mv std libs to library/ 2020-07-27 19:51:13 -05:00
primitive_docs Add primitive documentation to libcore 2021-09-12 02:23:08 +00:00
src Auto merge of #89165 - jkugelman:read-to-end-overallocation, r=joshtriplett 2021-10-04 04:44:56 +00:00
tests Update aarch64 runtime feature detection tests 2021-08-03 12:07:56 +00:00
build.rs Add SOLID targets 2021-09-28 11:31:47 +09:00
Cargo.toml Expose the std_detect env_override feature 2021-09-25 20:30:25 +02:00