Include executed tests in the build metrics (and use a custom test display impl)
The main goal of this PR is to include all tests executed in CI inside the build metrics JSON files. I need this for Ferrocene, and `@Mark-Simulacrum` expressed desire to have this as well to ensure all tests are executed at least once somewhere in CI.
Unfortunately implementing this required rewriting inside of bootstrap all of the code to render the test output to console. libtest supports outputting JSON instead of raw text, which we can indeed use to populate the build metrics. Doing that suppresses the console output though, and compared to rustc and Cargo the console output is not included as a JSON field.
Because of that, this PR had to reimplement both the "pretty" format (one test per line, with `rust.verbose-tests = true`), and the "terse" format (the wall of dots, with `rust.verbose-tests = false`). The current implementation should have the exact same output as libtest, except for the benchmark output. libtest's benchmark output is broken in the "terse" format, so since that's our default I slightly improved how it's rendered.
Also, to bring parity with libtest I had to introduce support for coloring output from bootstrap, using the same dependencies `annotate-snippets` uses. It's now possible to use `builder.color_for_stdout(Color::Red, "text")` and `builder.color_for_stderr(Color::Green, "text")` across all of bootstrap, automatically respecting the `--color` flag and whether the stream is a terminal or not.
I recommend reviewing the PR commit-by-commit.
r? `@Mark-Simulacrum`
Rollup of 10 pull requests
Successful merges:
- #106434 (Document `Iterator::sum/product` for Option/Result)
- #108326 (Implement read_buf for a few more types)
- #108842 (Enforce non-lifetime-binders in supertrait preds are not object safe)
- #108896 (new solver: make all goal evaluation able to be automatically rerun )
- #109124 (Add `dist.compression-profile` option to control compression speed)
- #109240 (Walk un-shifted nested `impl Trait` in trait when setting up default trait method assumptions)
- #109385 (fix typo)
- #109386 (add myself to mailmap)
- #109390 (Custom MIR: Support aggregate expressions)
- #109408 (not *all* retags might be explicit in Runtime MIR)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
not *all* retags might be explicit in Runtime MIR
In https://github.com/rust-lang/rust/pull/105317 I made Miri treat `Rvalue::Ref/AddrOf` as implicit retagging sites. This updates the MIR docs accordingly.
For `Rvalue::Ref` I think this makes a lot more sense: creating a new reference is their entire point, so we can avoid bloating the MIR with retags. Also this seems to be the best way to handle cases like `*ptr = &[mut] ...`, where doing a retag is somewhat questionable since maybe `*ptr` points to another place now?
For `Rvalue::AddrOf`, Stacked Borrows needs this because even raw ptrs need some retagging, but Tree Borrows doesn't do ant retagging here and I hope we'll end up with a model where raw pointers don't get retagged.
Custom MIR: Support aggregate expressions
Add support for tuple, array and ADT expressions in custom mir
r? `````@oli-obk````` or `````@tmiasko````` or `````@JakobDegen`````
Walk un-shifted nested `impl Trait` in trait when setting up default trait method assumptions
Fixes a double subtraction in some binder math in return-position `impl Trait` in trait handling code.
Fixes#109239
Add `dist.compression-profile` option to control compression speed
PR #108534 reduced the size of compressed archives, but (as expected) it also resulted in way longer compression times and memory usage during compression.
It's desirable to keep status quo (smaller archives but more CI usage), but it should also be configurable so that downstream users don't have to waste that much time on CI. As a data point, this resulted in doubling the time of Ferrocene's dist jobs, and required us to increase the RAM allocation for one of such jobs.
This PR adds a new `config.toml` setting, `dist.compression-profile`. The values can be:
* `fast`: equivalent to the gzip and xz preset of "1"
* `balanced`: equivalent to the gzip and xz preset of "6" (the CLI defaults as far as I'm aware)
* `best`: equivalent to the gzip present of "9", and our custom xz profile
The default has also been moved back to `balanced`, to try and avoid the compression time regression for downstream users. I don't feel too strongly on the default, and I'm open to changing it.
Also, for the `best` profile the XZ settings do not match the "9" preset used by the CLI, and it might be confusing. Should we create a `custom-rustc-ci`/`ultra` profile for that?
r? ``@Mark-Simulacrum``
new solver: make all goal evaluation able to be automatically rerun
It is generally wrong to call `evaluate_goal` multiple times or `evaluate_goal` and `evaluate_all` for the same `QueryResult` without correctly handling rerunning the goals when inference makes progress. Not doing so will result in the assertion in `evaluate_goal` firing because rerunning the goal will lead to a more accurate `QueryResult`.
Currently there are lots of places that get this wrong and generally it is complex and error prone to handle correctly everywhere. This PR introduces a way to add goals to the `EvalCtxt` and then run all the added goals in a loop so that `evaluate_goal`/`evaluate_all` is not necessary to call manually.
There are a few complications for making everything work "right":
1. the `normalizes-to` hack that replaces the rhs with an unconstrained infer var requires special casing in the new `try_evaluate_added_goals` function similar to how `evaluate_goal`'s assertion special cases that hack.
2. `assemble_candidates_after_normalizing_self_ty`'s normalization step needs to be reran for each candidate otherwise the found candidates will potentially get a more accurate `QueryResult` when rerunning the projection/trait goal which can effect the `QueryResult` of the projection/trait goal.
This is implemented via `EvalCtxt::probe`'s closure's `EvalCtxt` inheriting the added goals of the `EvalCtxt` that `probe` is called on, allowing us to add goals in a probe, and then enter a nested probe for each candidate and evaluate added goals which include the normalization step's goals.
I made `make_canonical_response` evaluate added goals so that it will be hard to mess up the impl of the solver by forgetting to evaluate added goals. Right now the only way to mess this up would be to call `response_no_constraints` (which from the name is obviously weird).
The visibility of `evaluate_goal` means that it can be called from various `compute_x_goal` or candidate assembly functions, this is generally wrong and we should never call `evaluate_goal` manually, instead we should be calling `add_goal`/`add_goals`. This is solved by moving `evaluate_goal` `evaluate_canonical_goal` and `compute_goal` into `eval_ctxt`'s module and making them private so they cannot be called from elsewhere, forcing people to call `add_goal/s` and `evaluate_added_goals_and_make_canonical_resposne`/`try_evaluate_added_goals`
---
Other changes:
- removed the `&& false` that was introduced to the assertion in `evaluate_goal` in #108839
- remove a `!self.did_overflow()` requirement in `search_graph.is_empty()` which causes goals that overflow to ICE
- made `EvalCtxt::eq` take `&mut self` and add all the nested goals via `add_goals` instead of returning them as 99% of call sites just immediately called `EvalCtxt::add_goals` manually.
r? `````@lcnr`````
Implement read_buf for a few more types
Implement read_buf for TcpStream, Stdin, StdinLock, ChildStdout,
ChildStderr (and internally for AnonPipe, Handle, Socket), so
that it skips buffer initialization.
The other provided methods like read_to_string and read_to_end are
implemented in terms of read_buf and so benefit from the optimization
as well.
This commit also implements read_vectored and is_read_vectored where
applicable.
Document `Iterator::sum/product` for Option/Result
Closes#105266
We already document the similar behavior for `collect()` so I believe it makes sense to add this too. The Option/Result implementations *are* documented on their respective pages and the page for `Sum`, but buried amongst many other trait impls which doesn't make it very discoverable.
`````@rustbot````` label +A-docs
Remove the assume(!is_null) from Vec::as_ptr
At a guess, this code is leftover from LLVM was worse at keeping track of the niche information here. In any case, we don't need this anymore: Removing this `assume` doesn't get rid of the `nonnull` attribute on the return type.
Add inlining annotations in `dec2flt`.
Currently, the combination of `dec2flt` being generic and the `FromStr` implementaions
containing inline anttributes causes massive amounts of assembly to be generated whenever
these implementation are used. In addition, the assembly has calls to function which ought to
be inlined, but they are not (even when using lto).
This Pr fixes this.
Distribute libntdll.a with windows-gnu toolchains
This allows the OS loader to load essential functions (e.g. read/write file) at load time instead of lazily doing so at runtime.
r? libs
Split `execute_job` into `execute_job_incr` and `execute_job_non_incr`
`execute_job` was a bit large, so this splits it in 2. Performance was neutral locally, but this may affect bootstrap times.
migrate compiler, bootstrap and compiletest to windows-rs
This PR migrates the compiler, bootstrap, and compiletest to use [windows-rs](https://github.com/microsoft/windows-rs) instead of winapi-rs. windows-rs is the bindings crate provided by Microsoft, and is actively maintained compared to winapi-rs. Not all ecosystem crates have migrated over yet, so there will be a period of time where both crates are used.
windows-rs also provides some nice ergonomics over winapi-rs to convert return values to `Result`s (which found a case where we forgot to check the return value of `CreateFileW`).
Optimize dep node backtrace and ignore fatal errors
This attempts to optimize https://github.com/rust-lang/rust/pull/91742 while also passing through fatal errors.
r? `@cjgillot`
Only expect a GAT const param for `type_of` of GAT const arg
IDK why we were account for both `is_ty_or_const` instead of just for a const param, since we're computing the `type_of` a const param specifically.
Fixes#109300
Fix generics_of for impl's RPITIT synthesized associated type
The only useful commit is the last one.
This makes `generics_of` for the impl side RPITIT copy from the trait's associated type and avoid the fn on the impl side which was previously wrongly used.
This solution is better but we still need to fix resolution of the generated generics.
r? ``@compiler-errors``
Add tests for configure.py
I highly recommend reviewing this with whitespace disabled.
Notably, verifying that we generate valid toml relies on python 3.11 so
we can use `tomllib`, so this also switches`x86_64-gnu-llvm-14` (one of the PR builders) to use 3.11.
While fixing that, I noticed that we stopped testing python2.7 support on PR CI in https://github.com/rust-lang/rust/pull/106085. `@fee1-dead` `@pietroalbini` please be more careful in the future, there is no CI for CI itself that verifies we are testing everything we should be.
- Separate out functions so that each unit test doesn't create a file on disk
- Add a few unit tests
Set `CMAKE_SYSTEM_NAME` for Linux targets
When bootstrap compiles native dependencies like LLVM, it should set `CMAKE_SYSTEM_NAME` for the target system; otherwise cmake may not identify that it is cross-compiling.
In particular, when building a Linux rustc on a macOS host, cmake was including `-isysroot /path/to/macOS.sdk` options that caused things to break. By setting `CMAKE_SYSTEM_NAME=Linux` when building for Linux targets, the macOS SDK is no longer passed as sysroot to the compiler.
r? bootstrap
fix ClashingExternDeclarations lint ICE
Fixes#109334
First "real" contribution, please let me know if I did something wrong.
As I understand it, it's OK if a `#[repr(transparent)]` type has no non-zero sized types (aka is a ZST itself) and the function should just return the type normally instead of panicking
r? `@Nilstrieb`
rustdoc: Remove footnote references from doc summary
Since it's one line, we don't have the footnote definition so it doesn't make sense to have the reference.
Part of https://github.com/rust-lang/rust/issues/109024.
r? `@notriddle`
Improve `Iterator::collect_into` documentation
This improves the examples in the documentation of `Iterator::collect_into`, replacing the usages of `println!` with `assert_eq!` as suggested on [IRLO](https://internals.rust-lang.org/t/18534/9).
rustdoc: implement bag semantics for function parameter search
This tweak to the function signature search engine makes things so that, if a type is repeated in the search query, it'll only match if the function actually includes it that many times.
