Test wasm32-wasip1 in CI, not wasm32-unknown-unknown
This commit changes CI to no longer test the `wasm32-unknown-unknown` target and instead test the `wasm32-wasip1` target. There was some discussion of this in a [Zulip thread], and the motivations for this PR are:
* Runtime failures on `wasm32-unknown-unknown` print nothing, meaning all you get is "something failed". In contrast `wasm32-wasip1` can print to stdout/stderr.
* The unknown-unknown target is missing lots of pieces of libstd, and while `wasm32-wasip1` is also missing some pieces (e.g. threads) it's missing fewer pieces. This means that many more tests can be run.
Overall my hope is to improve the debuggability of wasm failures on CI and ideally be a bit less of a maintenance burden.
This commit specifically removes the testing of `wasm32-unknown-unknown` and replaces it with testing of `wasm32-wasip1`. Along the way there were a number of other archiectural changes made as well, including:
* A new `target.*.runtool` option can now be specified in `config.toml` which is passed as `--runtool` to `compiletest`. This is used to reimplement execution of WebAssembly in a less-wasm-specific fashion.
* The default value for `runtool` is an ambiently located WebAssembly runtime found on the system, if any. I've implemented logic for Wasmtime.
* Existing testing support for `wasm32-unknown-unknown` and Emscripten has been removed. I'm not aware of Emscripten testing being run any time recently and otherwise `wasm32-wasip1` is in theory the focus now.
* I've added a new `//@ needs-threads` directive for `compiletest` and classified a bunch of wasm-ignored tests as needing threads. In theory these tests can run on `wasm32-wasi-preview1-threads`, for example.
* I've tried to audit all existing tests that are either `ignore-emscripten` or `ignore-wasm*`. Many now run on `wasm32-wasip1` due to being able to emit error messages, for example. Many are updated with comments as to why they can't run as well.
* The `compiletest` output matching for `wasm32-wasip1` automatically uses "match a subset" mode implemented in `compiletest`. This is because WebAssembly runtimes often add extra information on failure, such as the `unreachable` instruction in `panic!`, which isn't able to be matched against the golden output from native platforms.
* I've ported most existing `run-make` tests that use custom Node.js wrapper scripts to the new run-make-based-in-Rust infrastructure. To do this I added `wasmparser` as a dependency of `run-make-support` for the various wasm tests to use that parse wasm files. The one test that executed WebAssembly now uses `wasmtime`-the-CLI to execute the test instead. I have not ported over an exception-handling test as Wasmtime doesn't implement this yet.
* I've updated the `test` crate to print out timing information for WASI targets as it can do that (gets a previously ignored test now passing).
* The `test-various` image now builds a WASI sysroot for the WASI target and additionally downloads a fixed release of Wasmtime, currently the latest one at 18.0.2, and uses that for testing.
[Zulip thread]: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Have.20wasm.20tests.20ever.20caused.20problems.20on.20CI.3F/near/424317944
Make `DefiningAnchor::Bind` only store the opaque types that may be constrained, instead of the current infcx root item.
This makes `Bind` almost always be empty, so we can start forwarding it to queries, allowing us to remove `Bubble` entirely (not done in this PR)
The only behaviour change is in diagnostics.
r? `@lcnr` `@compiler-errors`
Rollup of 15 pull requests
Successful merges:
- #116791 (Allow codegen backends to opt-out of parallel codegen)
- #116793 (Allow targets to override default codegen backend)
- #117458 (LLVM Bitcode Linker: A self contained linker for nvptx and other targets)
- #119385 (Fix type resolution of associated const equality bounds (take 2))
- #121438 (std support for wasm32 panic=unwind)
- #121893 (Add tests (and a bit of cleanup) for interior mut handling in promotion and const-checking)
- #122080 (Clarity improvements to `DropTree`)
- #122152 (Improve diagnostics for parenthesized type arguments)
- #122166 (Remove the unused `field_remapping` field from `TypeLowering`)
- #122249 (interpret: do not call machine read hooks during validation)
- #122299 (Store backtrace for `must_produce_diag`)
- #122318 (Revision-related tweaks for next-solver tests)
- #122320 (Use ptradd for vtable indexing)
- #122328 (unix_sigpipe: Replace `inherit` with `sig_dfl` in syntax tests)
- #122330 (bootstrap readme: fix, improve, update)
r? `@ghost`
`@rustbot` modify labels: rollup
* The WASI targets deal with the `main` symbol a bit differently than
native so some `codegen` and `assembly` tests have been ignored.
* All `ignore-emscripten` directives have been updated to
`ignore-wasm32` to be more clear that all wasm targets are ignored and
it's not just Emscripten.
* Most `ignore-wasm32-bare` directives are now gone.
* Some ignore directives for wasm were switched to `needs-unwind`
instead.
* Many `ignore-wasm32*` directives are removed as the tests work with
WASI as opposed to `wasm32-unknown-unknown`.
This commit updates compiletest to automatically compare test output
with subsets if a `--runner` argument is configured. Runners might
inject extra information on failures, for example a WebAssembly runtime
printing a wasm stack trace, which won't be in the output of a native
runtime. The output with a `--runner` argument, however, should still
have all the native output present.
This commit updates the libtest conditionals to use `std::time::Instant`
on WASI targets where it's implemented. Previously all wasm targets
wouldn't use this type.
This commit rewrites a number of `run-make` tests centered around wasm
to instead use `rmake.rs` and additionally use the `wasm32-wasip1`
target instead of `wasm32-unknown-unknown`. Testing no longer requires
Node.js and additionally uses the `wasmparser` crate from crates.io to
parse outputs and power assertions.
unix_sigpipe: Replace `inherit` with `sig_dfl` in syntax tests
The `sig_dfl` variant of the attribute is the most likely variant to be stabilized first, and thus to become the "most allowed" variant of the attribute. Because of this, it is the most appropriate variant to use in syntax tests, because even if the most allowed variant is used, the compiler shall still emit errors if it e.g. is used in the wrong places.
r? ``@davidtwco`` who already [approved ](https://github.com/rust-lang/rust/pull/120832#pullrequestreview-1875075341) this commit in https://github.com/rust-lang/rust/pull/120832.
It would be nice to land the last preparatory commit of that PR before we begin to [rename ](https://github.com/rust-lang/rust/pull/120832#issuecomment-1987023484) things which will of course create a lot of code conflicts.
Use ptradd for vtable indexing
Extension of #121665.
After this, the only remaining usages of GEP are [this](cd81f5b27e/compiler/rustc_codegen_llvm/src/intrinsic.rs (L909-L920)) kinda janky Emscription EH code, which I'll change in a future PR, and array indexing / pointer offsets, where there isn't yet a canonical `ptradd` form. (Out of curiosity I tried converting the latter to `ptradd(ptr, mul(size, index))`, but that causes codegen regressions right now.)
r? `@nikic`
Revision-related tweaks for next-solver tests
1. Add `ignore-compare-mode-next-solver` to any test that already has explicit `current next` revisions, since the test failures when testing with `--compare-mode=next-solver` will be false positives.
2. Explicitly add revisions to a handful of tests where we expect behavior to diverge.
r? lcnr
Store backtrace for `must_produce_diag`
This makes it significantly easier to debug a `must_produce_diag` ICE, since we have no other way to know where the heck the bug originates from.
Backtrace rendering kinda sucks right now since we're just printing it in the panic message; happy to apply some suggestions to make it prettier or reuse other bug printing machinery, but also don't want to iterate too much on the rendering since this really is just for debug purposes.
r? nnethercote
Remove the unused `field_remapping` field from `TypeLowering`
The `field_remapping` field of `TypeLowering` has been unused since #121665. This PR removes it, then replaces the `TypeLowering` struct with its only remaining member `&'ll Type`.
Clarity improvements to `DropTree`
These changes are based on some points of confusion I had when initially trying to understand this code.
The only “functional” change is an additional assertion in `<ExitScopes as DropTreeBuilder>::link_entry_point`, checking that the dummy terminator is `TerminatorKind::UnwindResume` as expected.
Add tests (and a bit of cleanup) for interior mut handling in promotion and const-checking
Basically these are the parts of https://github.com/rust-lang/rust/pull/121786 that can be salvaged.
r? ``@oli-obk``
std support for wasm32 panic=unwind
Tracking issue: #118168
This adds std support for `-Cpanic=unwind` on wasm, and with it slightly more fleshed out rustc support. Now, the stable default is still panic=abort without exception-handling, but if you `-Zbuild-std` with `RUSTFLAGS=-Cpanic=unwind`, you get wasm exception-handling try/catch blocks in the binary:
```rust
#[no_mangle]
pub fn foo_bar(x: bool) -> *mut u8 {
let s = Box::<str>::from("hello");
maybe_panic(x);
Box::into_raw(s).cast()
}
#[inline(never)]
#[no_mangle]
fn maybe_panic(x: bool) {
if x {
panic!("AAAAA");
}
}
```
```wat
;; snip...
(try $label$5
(do
(call $maybe_panic
(local.get $0)
)
(br $label$1)
)
(catch_all
(global.set $__stack_pointer
(local.get $1)
)
(call $__rust_dealloc
(local.get $2)
(i32.const 5)
(i32.const 1)
)
(rethrow $label$5)
)
)
;; snip...
```
Fix type resolution of associated const equality bounds (take 2)
Instead of trying to re-resolve the type of assoc const bindings inside the `type_of` query impl in an incomplete manner, transfer the already (correctly) resolved type from `add_predicates_for_ast_type_binding` to `type_of`/`anon_type_of` through query feeding.
---
Together with #118668 (merged) and #121258, this supersedes #118360.
Fixes#118040.
r? ``@ghost``
LLVM Bitcode Linker: A self contained linker for nvptx and other targets
This PR introduces a new linker named `llvm-bitcode-linker`. It is a `self-contained` linker that can be used to link programs in `llbc` before optimizing and compiling to native code. It will first be used internally in the Rust compiler to enable tests for the `nvptx64-nvidia-cuda` target as the original `rust-ptx-linker` is deprecated. It will then be provided to users of the `nvptx64-nvidia-cuda` target with the purpose of linking ptx. More targets than nvptx will also be supported eventually.
The PR introduces a new unstable `LinkerFlavor` for the compiler. The compiler will also not be shipped with rustc but most likely instead be shipped in it's own unstable component (a follow up PR will be opened for this). This means that merging this PR should not add any stability guarantees.
When more details of `self-contained` is implemented it will only be possible to use the linker when `-Clink-self-contained=+linker` is passed.
<details>
<summary>Original Description</summary>
**When this PR was created it was focused a bit differently. The original text is preserved here in case there's some interests in it**
I have experimenting with approaches to replace the ptx-linker and enable the nvptx target tests again. I think it's time to get some feedback on the approach.
### The problem
The only useful linker for the nvptx target is [this crate](https://github.com/denzp/rust-ptx-linker). Since this linker performs linking on llvm bitcode it needs to track the llvm version of rustc and use the same format. It has not been maintained for 3+ years and must be considered abandoned. Over the years rust have upgraded LLVM while the linker has been left to bitrot. It is no longer in a usable state.
Due to the difficulty of keeping the ptx-linker up to date outside of tree the nvptx tests was [disabled a long time ago](f8f9a2869c). It was [previously discussed](https://github.com/rust-lang/rust/pull/96842#issuecomment-1146470177) if adding the ptx-linker to the rust repo would be a possibility. My efforts in doing this stopped at getting an answered if the license would prohibit it from inclusion in the [Rust repo](https://github.com/rust-lang/rust/pull/96842#issuecomment-1148397554). I therefore concluded that a re-write would be necessary.
### The possible solution presented here
The llvm tools know perfectly well how to link and optimize llvm bitcode. Each of them only perform a single task, and are therefore a bit cumbersome to call with the current linker approach rustc takes.
This PR adds a simple tool (current name `embedded-linker`) which can link self contained (often embedded) programs in llvm bitcode before compiling to the target format. Optimization will also be performed if lto is enabled. The rust compiler will make a single invocation to this tool, while the tool will orchestrate the many calls to the llvm tools.
### The questions
- Is having control over the nvptx linking and therefore also tests worth it to add such tool? or should the tool live outside the rust repo?
- Is the approach of calling llvm tools acceptable? Or would we want to keep the ptx-linker approach of using the llvm library? The tools seems to provide more simplicity and stability, but more intermediate files are being written. Perhaps there also are some performance penalty for the calling tools approach.
- What is the process for adding such tool? MCP?
- Does adding `llvm-link` to the llvm-tool component require any process?
- Does it require some sort of FCP to remove ptx-linker as the default linker for ptx? Or is it sufficient that using the upstream ptx-linker is broken in its current state. it is possible to use a somewhat patched version of ptx-linker.
</details>
Run a single huge par_body_owners instead of many small ones after each other.
This improves parallel rustc parallelism by avoiding the bottleneck after each individual `par_body_owners` (because it needs to wait for queries to finish, so if there is one long running one, a lot of cores will be idle while waiting for the single query).
Add metadata to targets
follow up to #121905 and #122157
This adds four pieces of metadata to every target:
- description
- tier
- host tools
- std
This information is currently scattered across target docs and both
- not machine readable, making validation harder
- sometimes subtly encoding by the table it's in, causing mistakes and making it harder to review changes to the properties
By putting it in the compiler, we improve this. Later, we will use this canonical information to generate target documentation from it.
I used find-replace for all the `description: None`.
One thing I'm not sure about is the behavior for the JSON. It doesn't really make sense that custom targets supply this information, especially the tier. But for the roundtrip tests, we do need to print and parse it. Maybe emit a warning when a custom target provides the metadata key? Either way, I don't think that's important right now, this PR should get merged ASAP or it will conflict all over the place.
r? davidtwco
This improves parallel rustc parallelism by avoiding the bottleneck after each individual `par_body_owners` (because it needs to wait for queries to finish, so if there is one long running one, a lot of cores will be idle while waiting for the single query).
