Import rust-installer & adjust compression settings
This brings in rust-lang/rust-installer#123, which enables a larger compression window (8 -> 64MB) amongst other changes to the xz compression settings. The net effect should be smaller compressed tarballs which will decrease bandwidth usage for
static.rust-lang.org, download times, and decompression time.
This comes at the cost of higher baseline requirements for running rustup to use these files, which we believe should be largely acceptable (running rustc is likely to use at least this much memory) but if we get specific reports we may explore options to decrease impact (e.g., using the gzip tarballs automatically in rustup).
To simplify iteration on compression settings this also imports the rust-lang/rust-installer submodule, it is now hosted fully inside rust-lang/rust. Once we land this I'll file a followup to add a note to that repo and we can subsequently archive it.
--
CI times for dist-x86_64-linux builds:
* threads=6, master - 2h 50m
* threads=1, new - 3h 40m
* threads=6, new - 2h 50m
Do not implement HashStable for HashSet (MCP 533)
This PR removes all occurrences of `HashSet` in query results, replacing it either with `FxIndexSet` or with `UnordSet`, and then removes the `HashStable` implementation of `HashSet`. This is part of implementing [MCP 533](https://github.com/rust-lang/compiler-team/issues/533), that is, removing the `HashStable` implementations of all collection types with unstable iteration order.
The changes are mostly mechanical. The only place where additional sorting is happening is in Miri's override implementation of the `exported_symbols` query.
Add a new config flag, dist.include-mingw-linker.
The flag controls whether to copy the linker, DLLs, and various libraries from MinGW into the rustc toolchain.
It applies only when the host or target is pc-windows-gnu.
The flag is true by default to preserve existing behavior.
Update askama to 0.12 and improve whitespace control
`askama` 0.12 was just released, and it adds the possibility to have "whitespace suppression" enabled by default in the configuration so I switched to it.
r? ``@notriddle``
Remove the option to disable `llvm-version-check`
We don't support old versions of LLVM; there's no reason to have an easy way to force bootstrap to use them anyway. If someone really needs to use an unsupported version, they can modify bootstrap to change the version range.
r? ``@cuviper`` on whether we want to do this or not, since you maintain rust on Fedora and touched this config last.
This moves the rust-installer code to be directly hosted in
rust-lang/rust, since it's not used elsewhere and this makes it easier
to make and review changes without needing a separate upstream commit.
This brings in rust-lang/rust-installer#123, which enables a larger
compression window (8 -> 64MB) amongst other changes to the xz
compression settings. The net effect should be smaller compressed
tarballs which will decrease bandwidth usage for
static.rust-lang.org, download times, and decompression time.
This comes at the cost of higher baseline requirements for running
rustup to use these files, which we believe should be largely acceptable
(running rustc is likely to use at least this much memory) but if we get
specific reports we may explore options to decrease impact (e.g., using
the gzip tarballs automatically in rustup).
x fmt: Only check modified files locally
Previously, `x fmt` would only format modified files, while `x fmt .` and `x fmt --check` would still look at all files. After this change, `x fmt --check` only looks at modified files locally.
I feel pretty confident in this change - other than https://github.com/rust-lang/rust/issues/106261, no one has reported bugs in `get_modified_rs_files` since it was added in https://github.com/rust-lang/rust/pull/105702.
Combined with the changes in https://github.com/rust-lang/rust/pull/108772, this brings the time for me to run `x t tidy` with a hot FS cache down from 5 to 2 seconds (and moves the majority of the time spent back to `tidy check`, which means it can be sped up more in the future).
We don't support old versions of LLVM; there's no reason to have an easy
way to force bootstrap to use them anyway. If someone really needs to
use an unsupported version, they can modify bootstrap to change the
version range.
Previously, `x fmt` would only format modified files, while `x fmt .`
and `x fmt --check` would still look at all files. After this change, `x
fmt --check` only looks at modified files locally.
I feel pretty confident in this change - other than
https://github.com/rust-lang/rust/issues/106261, no one has reported
bugs in `get_modified_rs_files` since it was added in
https://github.com/rust-lang/rust/pull/105702.
rustdoc-json: switch from HashMap to FxHashMap to fix non-determinism
Using `HashMap` in `rustdoc_json_types::Crate` were causing creating randomly ordered objects in the json doc files. Which might cause problems to people who are doing comparison on those files specially in CI pipelines. See https://github.com/rust-lang/rust/issues/103785#issuecomment-1307425590
This PR fixes that issue and extends the coverage of `tests/run-make/rustdoc-verify-output-files` testing ability.
Remove `llvm.skip-rebuild` option
This was added to in 2019 to speed up rebuild times when LLVM was modified. Now that download-ci-llvm exists, I don't think it makes sense to support an unsound option like this that can lead to miscompiles; and the code cleanup is nice too.
r? `@Mark-Simulacrum` cc `@varkor` #65612
Ignore files in .gitignore in tidy
- Switch from `walkdir` to `ignore`. This required various changes to make `skip` thread-safe.
- Ignore `build` anywhere in the source tree, not just at the top-level. We support this in bootstrap, we should support it in tidy too.
As a nice side benefit, this also makes tidy a bit faster.
Before:
```
; hyperfine -i '"/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"'
Benchmark 1: "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"
Time (mean ± σ): 1.080 s ± 0.008 s [User: 2.616 s, System: 3.243 s]
Range (min … max): 1.069 s … 1.099 s 10 runs
```
After:
```
; hyperfine '"/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"'
Benchmark 1: "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"
Time (mean ± σ): 705.0 ms ± 1.4 ms [User: 3179.1 ms, System: 1517.5 ms]
Range (min … max): 702.3 ms … 706.9 ms 10 runs
```
r? `@the8472`
`WalkBuilder` handles top-level paths differently than `fn walk` used
to: it doesn't run the `skip` function to determine if it should be
skipped, instead assuming the top-level function is always included.
This is a reasonable assumption; adapt our code so it doesn't make
pointless calls to `walk`.
- Switch from `walkdir` to `ignore`. This required various changes to
make `skip` thread-safe.
- Ignore `build` anywhere in the source tree, not just at the top-level.
We support this in bootstrap, we should support it in tidy too.
As a nice side benefit, this also makes tidy a bit faster.
Before:
```
; hyperfine -i '"/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"'
Benchmark 1: "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"
Time (mean ± σ): 1.080 s ± 0.008 s [User: 2.616 s, System: 3.243 s]
Range (min … max): 1.069 s … 1.099 s 10 runs
```
After:
```
; hyperfine '"/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"'
Benchmark 1: "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0-tools-bin/rust-tidy" "/home/gh-jyn514/rust2" "/home/gh-jyn514/rust2/build/aarch64-unknown-linux-gnu/stage0/bin/cargo" "/home/gh-jyn514/rust2/build" "32"
Time (mean ± σ): 705.0 ms ± 1.4 ms [User: 3179.1 ms, System: 1517.5 ms]
Range (min … max): 702.3 ms … 706.9 ms 10 runs
```
rustc_middle: Remove trait `DefIdTree`
This trait was a way to generalize over both `TyCtxt` and `Resolver`, but now `Resolver` has access to `TyCtxt`, so this trait is no longer necessary.
Apply BOLT optimizations without rebuilding LLVM
This PR adds an explicit BOLT bootstrap step which applies BOLT on the fly when LLVM artifacts are copied to a sysroot (it only does this once per bootstrap invocation, the result is cached). This avoids one LLVM rebuild in the Linux CI dist build.
r? `@jyn514`
Rollup of 7 pull requests
Successful merges:
- #108627 (Properly colorize multi-part suggestions in the same line)
- #108632 (Omit unchanged options from config.toml in `configure.py`)
- #108715 (Remove unclosed_delims from parser)
- #108723 (rustdoc: function signature search with traits in `where` clause)
- #108724 (field is not used outside the crate)
- #108734 (rustdoc: Note in a type's layout/size if it is uninhabited)
- #108736 (Remove `allow(potential_query_instability)` from `ast_passes`)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Omit unchanged options from config.toml in `configure.py`
Leaves section tags, but removes options that are unchanged.
Change in `config.toml.example` is to prevent comments from sneaking in by being directly after a section tag
closes#108612
