Both test-panic-abort-nocapture.rs and test-panic-abort.rs assert the
stderr output of the test. On Fuchsia, if a test fails an assertion,
this output will contain a line noting the process returned the code
-1028 (ZX_TASK_RETCODE_EXCEPTION_KILL). But the asserted stderr output
lacks this note. Presumably this is because other platforms implement
-Cpanic=abort by killing the process instead of returned a status
code.
* The WASI targets deal with the `main` symbol a bit differently than
native so some `codegen` and `assembly` tests have been ignored.
* All `ignore-emscripten` directives have been updated to
`ignore-wasm32` to be more clear that all wasm targets are ignored and
it's not just Emscripten.
* Most `ignore-wasm32-bare` directives are now gone.
* Some ignore directives for wasm were switched to `needs-unwind`
instead.
* Many `ignore-wasm32*` directives are removed as the tests work with
WASI as opposed to `wasm32-unknown-unknown`.
compiletest: Add a `//@ needs-threads` directive
This commit is extracted from #122036 and adds a new directive to the `compiletest` test runner, `//@ needs-threads`. This is intended to capture the need that a target must implement threading to execute a specific test, typically one that uses `std::thread`. This is primarily done for WebAssembly targets which currently do not have threads by default. This enables transitioning a lot of `//@ ignore-wasm*`-style ignores into a more self-documenting `//@ needs-threads` directive. Additionally the `wasm32-wasi-preview1-threads` target, for example, does actually have threads, but isn't tested in CI at this time. This change enables running these tests for that target, but not other wasm targets.
This commit is extracted from #122036 and adds a new directive to the
`compiletest` test runner, `//@ needs-threads`. This is intended to
capture the need that a target must implement threading to execute a
specific test, typically one that uses `std::thread`. This is primarily
done for WebAssembly targets which currently do not have threads by
default. This enables transitioning a lot of `//@ ignore-wasm*`-style
ignores into a more self-documenting `//@ needs-threads` directive.
Additionally the `wasm32-wasi-preview1-threads` target, for example,
does actually have threads, but isn't tested in CI at this time. This
change enables running these tests for that target, but not other wasm
targets.
Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.
This also adds a test for the terse output; previously it was untested.
Modify panic message for `assert_eq!`, `assert_ne!`, the currently unstable `assert_matches!`, as well as the corresponding `debug_assert_*` macros.
```rust
assert_eq!(1 + 1, 3);
assert_eq!(1 + 1, 3, "my custom message value={}!", 42);
```
```plain
thread 'main' panicked at $DIR/main.rs:6:5:
assertion failed: `(left == right)`
left: `2`,
right: `3`
```
```plain
thread 'main' panicked at $DIR/main.rs:6:5:
assertion failed: `(left == right)`
left: `2`,
right: `3`: my custom message value=42!
```
```plain
thread 'main' panicked at $DIR/main.rs:6:5:
assertion `left == right` failed
left: 2
right: 3
```
```plain
thread 'main' panicked at $DIR/main.rs:6:5:
assertion `left == right` failed: my custom message value=42!
left: 2
right: 3
```
This PR is a simpler subset of the #111030, but it does NOT stringify the original left and right source code assert expressions, thus should be faster to compile.
Test benchmarks with `-Z panic-abort-tests`
During test execution, when a `#[bench]` benchmark is encountered it's executed once to check whether it works. Unfortunately that was not compatible with `-Z panic-abort-tests`: the feature works by spawning a subprocess for each test, which prevents the use of dynamic tests as we cannot pass closures to child processes, and before this PR the conversion from benchmark to test was done by turning benchmarks into dynamic tests whose closures execute the benchmark once.
The approach this PR took was to add two new kinds of `TestFn`s: `StaticBenchAsTestFn` and `DynBenchAsTestFn` (⚠️ **this is a breaking change** ⚠️). With that change, a `StaticBenchFn` can be converted into a `StaticBenchAsTestFn` without creating dynamic tests, and making it possible to test `#[bench]` functions with `-Z panic-abort-tests`. The subprocess test runner also had to be updated to perform the conversion from benchmark to test when appropriate.
Along with the bug fix, in the first commit I refactored how tests are executed: rather than executing the test function in multiple places across `libtest`, there is now a private `TestFn::into_runnable()` method, which returns either a `RunnableTest` or `RunnableBench`, on which you can call the `run()` method. This simplified the rest of the changes in the PR.
This PR is best reviewed commit-by-commit.
Fixes https://github.com/rust-lang/rust/issues/73509
fix: fix ICE in `custom-test-frameworks` feature
Fixes#107454
Simple fix to emit error instead of ICEing. At some point, all the code in `tests.rs` should be refactored, there is a bit of duplication (this PR's code is repeated five times over lol).
r? `@Nilstrieb` (active on the linked issue?)
