enhancements on build_helper utilization and rustdoc-gui-test
This change provides codebase improvements, resolves `FIXME` in `rustdoc-gui-test` and makes `rustdoc-gui` test able to find local `node_modules` directory outside of the source root.
Roll compiler_builtins to 0.1.92
This pulls in the weak-intrinsics feature (which currently defaults off), and a minor version update to libm for the compiler_builtins crate to 0.2.7.
Include better context for "already exists" error in compiletest
I encountered the following error from `x.py test tests/ui` today.
```console
---- [ui] tests/ui/impl-trait/multiple-lifetimes/multiple-lifetimes.rs stdout ----
thread '[ui] tests/ui/impl-trait/multiple-lifetimes/multiple-lifetimes.rs' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 17, kind: AlreadyExists, message: "File exists" }', src/tools/compiletest/src/runtest.rs:134:43
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
I found it impossible to unblock myself without knowing which directory it was stuck on.
Error message after this PR:
```console
---- [ui] tests/ui/impl-trait/multiple-lifetimes/multiple-lifetimes.rs stdout ----
thread '[ui] tests/ui/impl-trait/multiple-lifetimes/multiple-lifetimes.rs' panicked at 'called `Result::unwrap()` on an `Err` value: failed to create output base directory /git/rust/build/x86_64-unknown-linux-gnu/test/ui/impl-trait/multiple-lifetimes/multiple-lifetimes
Caused by:
File exists (os error 17)', src/tools/compiletest/src/runtest.rs:139:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Now I was able to run `rm build/x86_64-unknown-linux-gnu/test/ui/impl-trait/multiple-lifetimes/multiple-lifetimes` and unblock myself.
Seems to be related to #109509 moving *tests/ui/impl-trait/multiple-lifetimes.rs* to *tests/ui/impl-trait/multiple-lifetimes/multiple-lifetimes.rs*.
Update serde in workspace and non-synced dependencies
The main workspace, bootstrap, cargo, miri, and rust-analyzer all lock serde to different versions. It's preferable to share the same version where possible, so update it.
Rustfmt is synced from another repository and has its own Cargo.lock, but since it's added to the overall workspace it should respect the version here.
Cargo is already at the latest version. Miri and rust-analyzer would require upstream updates.
Use dynamic dispatch for queries
This replaces most concrete query values `V` with `MaybeUninit<[u8; { size_of::<V>() }]>` reducing the code instantiated by queries. The compile time of `rustc_query_impl` is reduced by 27%. It is an alternative to https://github.com/rust-lang/rust/pull/107937 which uses unstable const generics while this uses a `EraseType` trait which maps query values to their erased variant.
This is achieved by introducing an `Erased` type which does sanity check with `cfg(debug_assertions)`. The query caches gets instantiated with these erased types leaving the code in `rustc_query_system` unaware of them. `rustc_query_system` is changed to use instances of `QueryConfig` so that `rustc_query_impl` can pass in `DynamicConfig` which holds a pointer to a virtual table.
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">1.7055s</td><td align="right">1.6949s</td><td align="right"> -0.62%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.2547s</td><td align="right">0.2528s</td><td align="right"> -0.73%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.9590s</td><td align="right">0.9553s</td><td align="right"> -0.39%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">1.5457s</td><td align="right">1.5440s</td><td align="right"> -0.11%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check</td><td align="right">5.9092s</td><td align="right">5.9009s</td><td align="right"> -0.14%</td></tr><tr><td>Total</td><td align="right">10.3741s</td><td align="right">10.3479s</td><td align="right"> -0.25%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9960s</td><td align="right"> -0.40%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:initial</td><td align="right">2.0605s</td><td align="right">2.0575s</td><td align="right"> -0.15%</td></tr><tr><td>🟣 <b>hyper</b>:check:initial</td><td align="right">0.3218s</td><td align="right">0.3216s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>regex</b>:check:initial</td><td align="right">1.1848s</td><td align="right">1.1839s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>syn</b>:check:initial</td><td align="right">1.9409s</td><td align="right">1.9376s</td><td align="right"> -0.17%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:initial</td><td align="right">7.3105s</td><td align="right">7.2928s</td><td align="right"> -0.24%</td></tr><tr><td>Total</td><td align="right">12.8185s</td><td align="right">12.7935s</td><td align="right"> -0.20%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9986s</td><td align="right"> -0.14%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4606s</td><td align="right">0.4617s</td><td align="right"> 0.24%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1335s</td><td align="right">0.1336s</td><td align="right"> 0.08%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3324s</td><td align="right">0.3346s</td><td align="right"> 0.65%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.6268s</td><td align="right">0.6307s</td><td align="right"> 0.64%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.8248s</td><td align="right">1.8508s</td><td align="right">💔 1.43%</td></tr><tr><td>Total</td><td align="right">3.3779s</td><td align="right">3.4113s</td><td align="right"> 0.99%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">1.0061s</td><td align="right"> 0.61%</td></tr></table>
It's based on https://github.com/rust-lang/rust/pull/108167.
r? `@cjgillot`
Introduce `DynSend` and `DynSync` auto trait for parallel compiler
part of parallel-rustc #101566
This PR introduces `DynSend / DynSync` trait and `FromDyn / IntoDyn` structure in rustc_data_structure::marker. `FromDyn` can dynamically check data structures for thread safety when switching to parallel environments (such as calling `par_for_each_in`). This happens only when `-Z threads > 1` so it doesn't affect single-threaded mode's compile efficiency.
r? `@cjgillot`
bump windows crate 0.46 -> 0.48
This drops duped version of crate(0.46), reduces `rustc_driver.dll` ~800kb and reduces exported functions number from 26k to 22k.
Also while here, added `tidy-alphabetical` sorting to lists in tidy allowed lists.
Start using `windows sys` for Windows FFI bindings in std
Switch to using windows-sys for FFI. In order to avoid some currently contentious issues, this uses windows-bindgen to generate a smaller set of bindings instead of using the full crate.
Unlike the windows-sys crate, the generated bindings uses `*mut c_void` for handle types instead of `isize`. This to sidestep opsem concerns about mixing pointer types and integers between languages. Note that `SOCKET` remains defined as an integer but instead of being a usize, it's changed to fit the [standard library definition](a41fc00eaf/library/std/src/os/windows/raw.rs (L12-L16)):
```rust
#[cfg(target_pointer_width = "32")]
pub type SOCKET = u32;
#[cfg(target_pointer_width = "64")]
pub type SOCKET = u64;
```
The generated bindings also customizes the `#[link]` imports. I hope to switch to using raw-dylib but I don't want to tie that too closely with the switch to windows-sys.
---
Changes outside of the bindings are, for the most part, fairly minimal (e.g. some differences in `*mut` vs. `*const` or a few types differ). One issue is that our own bindings sometimes mix in higher level types, like `BorrowedHandle`. This is pretty adhoc though.
STD support for PSVita
This PR adds std support for `armv7-sony-vita-newlibeabihf` target.
The work here is fairly similar to #95897, just for a different target platform.
This depends on the following pull requests:
rust-lang/backtrace-rs#523rust-lang/libc#3209
The main workspace, bootstrap, cargo, miri, and rust-analyzer all lock
serde to different versions. It's preferable to share the same version
where possible, so update it.
Rustfmt is synced from another repository and has its own Cargo.lock,
but since it's added to the overall workspace it should respect the
version here.
Cargo is already at the latest version. Miri and rust-analyzer would
require upstream updates.
Add a `sysroot` crate to represent the standard library crates
This adds a dummy crate named `sysroot` to represent the standard library target instead of using the `test` crate. This allows the removal of `proc_macro` as a dependency of `test` allowing these 2 crates to build in parallel saving around 9 seconds locally.
Support loading version information from xz tarballs
This is intended to allow us to move recompression from xz (produced in CI) to gz after an initial manifest run, which produces a list of actually required artifacts. The rest are then deleted, which means that we can avoid recompressing them, saving a bunch of time.
This is essentially untested and more might be needed, will run a patched promote-release against try artifacts from this PR. If we do go ahead with this we'll either need to backport this patch to beta/stable, wait for it to propagate, or temporarily recompress to gzip but not xz tarballs (or similar).
r? `@pietroalbini`
It's only used in tests. Which is bad, because it means that
`FileEncoder` is used in the compiler but isn't used in tests!
`tests/opaque.rs` now tests encoding/decoding round-trips via file.
Because this is slower than memory, this commit also adjusts the
`u16`/`i16` tests so they are more like the `u32`/`i32` tests, i.e. they
don't test every possible value.
Rollup of 7 pull requests
Successful merges:
- #110586 (Fix Unreadable non-UTF-8 output on localized MSVC)
- #110652 (Add test for warning-free builds of `core` under `no_global_oom_handling`)
- #110973 (improve error notes for packed struct reference diagnostic)
- #110981 (Move most rustdoc-ui tests into subdirectories)
- #110983 (rustdoc: Get `repr` information through `AdtDef` for foreign items)
- #110984 (Do not resolve anonymous lifetimes in consts to be static.)
- #110997 (Improve internal field comments on `slice::Iter(Mut)`)
Failed merges:
r? `@ghost`
`@rustbot` modify labels: rollup
Fix Unreadable non-UTF-8 output on localized MSVC
Fixes#35785 by converting non UTF-8 linker output to Unicode using the OEM code page.
Before:
```text
= note: Non-UTF-8 output: LINK : fatal error LNK1181: cannot open input file \'m\x84rchenhaft.obj\'\r\n
```
After:
```text
= note: LINK : fatal error LNK1181: cannot open input file 'märchenhaft.obj'
```
The difference is more dramatic if using a non-ascii language pack for Windows.
Remove `QueryEngine` trait
This removes the `QueryEngine` trait and `Queries` from `rustc_query_impl` and replaced them with function pointers and fields in `QuerySystem`. As a side effect `OnDiskCache` is moved back into `rustc_middle` and the `OnDiskCache` trait is also removed.
This has a couple of benefits.
- `TyCtxt` is used in the query system instead of the removed `QueryCtxt` which is larger.
- Function pointers are more flexible to work with. A variant of https://github.com/rust-lang/rust/pull/107802 is included which avoids the double indirection. For https://github.com/rust-lang/rust/pull/108938 we can name entry point `__rust_end_short_backtrace` to avoid some overhead. For https://github.com/rust-lang/rust/pull/108062 it avoids the duplicate `QueryEngine` structs.
- `QueryContext` now implements `DepContext` which avoids many `dep_context()` calls in `rustc_query_system`.
- The `rustc_driver` size is reduced by 0.33%, hopefully that means some bootstrap improvements.
- This avoids the unsafe code around the `QueryEngine` trait.
r? `@cjgillot`
Add support for LibreSSL 3.7.x
This updates the `openssl-sys` crate to 0.9.87 to support building the toolchain against the system libraries provided by LibreSSL version 3.7.x.
LibreSSL 3.7.x has been supported since `openssl-sys` version 0.9.85.
This updates the `openssl-sys` crate to 0.9.87 to support building the
toolchain against the system libraries provided by LibreSSL version 3.7.x.
LibreSSL 3.7.x has been supported since `openssl-sys` version 0.9.85.
Fixes#35785 by converting non UTF-8 linker output to Unicode using the OEM code page.
Before:
```text
= note: Non-UTF-8 output: LINK : fatal error LNK1181: cannot open input file \'m\x84rchenhaft.obj\'\r\n
```
After:
```text
= note: LINK : fatal error LNK1181: cannot open input file 'märchenhaft.obj'
```
The difference is more dramatic if using a non-ascii language pack for Visual Studio.
Support AIX-style archive type
Reading facility of AIX big archive has been supported by `object` since 0.30.0.
Writing facility of AIX big archive has already been supported by `ar_archive_writer`, but we need to bump the version to support the new archive type enum.
Fluent, with all the icu4x it brings in, takes quite some time to
compile. `fluent_messages!` is only needed in further downstream rustc
crates, but is blocking more upstream crates like `rustc_index`. By
splitting it out, we allow `rustc_macros` to be compiled earlier, which
speeds up `x check compiler` by about 5 seconds (and even more after the
needless dependency on `serde_json` is removed from
`rustc_data_structures`).
Don't use `serde_json` to serialize a simple JSON object
This avoids `rustc_data_structures` depending on `serde_json` which allows it to be compiled much earlier, unlocking most of rustc.
This used to not matter, but after #110407 we're not blocked on fluent anymore, which means that it's now a blocking edge.
This saves a few more seconds.
cc ````@Zoxc```` who added it recently
This also
* bumps cargo to the latest in rust-lang/cargo.
* adds 0BSD to allowed list of licenses
Co-authored-by: Scott Schafer <schaferjscott@gmail.com>
Co-authored-by: Eric Huss <eric@huss.org>
This allows us to get rid of the `rustc_const_eval->rustc_borrowck`
dependency edge which was delaying the compilation of borrowck.
The added utils in `rustc_middle` are small and should not affect
compile times there.
Create "suggested tests" tool in `rustbuild`
Not the claimed person in #97339 but:
I've done a very rough implementation of this feature in-tree. I'm very new to `rustc` development (outside of docs) so some help would be greatly appreciated. The UI of this new subcommand obviously will change and I need some mentoring with the `--run` flag.
r? ```@jyn514```
