Show test type during prints
Test output can sometimes be confusing. For example doctest with the no_run argument are displayed the same way than test that are run.
During #83857 I got the feedback that test output can be confusing.
For the moment test output is
```
test $DIR/test-type.rs - f (line 12) ... ignored
test $DIR/test-type.rs - f (line 15) ... ok
test $DIR/test-type.rs - f (line 21) ... ok
test $DIR/test-type.rs - f (line 6) ... ok
```
I propose to change output by indicating the test type as
```
test $DIR/test-type.rs - f (line 12) ... ignored
test $DIR/test-type.rs - f (line 15) - compile ... ok
test $DIR/test-type.rs - f (line 21) - compile fail ... ok
test $DIR/test-type.rs - f (line 6) ... ok
```
by indicating the test type after the test name (and in the case of doctest after the function name and line) and before the "...".
------------
Note: this is a proof of concept, the implementation is probably not optimal as the properties added in `TestDesc` are only use in the display and does not represent actual change of behavior, maybe `TestType::DocTest` could have fields
## User-facing changes
- Intra-doc links to primitives that currently go to rust-lang.org/nightly/std/primitive.x.html will start going to channel that rustdoc was built with. Nightly will continue going to /nightly; Beta will link to /beta; stable compilers will link to /1.52.1 (or whatever version they were built as).
- Cross-crate links from std to core currently go to /nightly unconditionally. They will start going to /1.52.0 on stable channels (but remain the same on nightly channels).
- Intra-crate links from std to std (or core to core) currently go to the same URL they are hosted at; they will continue to do so. Notably, this is different from everything else because it can preserve the distinction between /stable and /1.52.0 by using relative links.
Note that "links" includes both intra-doc links and rustdoc's own
automatically generated hyperlinks.
## Implementation changes
- Update the testsuite to allow linking to /beta and /1.52.1 in docs
- Use an html_root_url for the standard library that's dependent on the channel
This avoids linking to nightly docs on stable.
- Update rustdoc to use channel-dependent links for primitives from an
unknown crate
- Set DOC_RUST_LANG_ORG_CHANNEL from bootstrap to ensure it's in sync
- Include doc.rust-lang.org in the channel
Add a chapter on the test harness.
There isn't really any online documentation on the test harness, so this adds a chapter to the rustc book which provides information on how the harness works and details on the command-line options.
test: Print test name only once on timeout
Pretty formatter when using multiple test threads displays test name twice on
timeout event. This implicitly suggest that those are two different events,
while in fact they are always printed together.
Print test name only once.
Before:
```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) ... test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```
After:
```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```
Pretty formatter when using multiple test threads displays test name twice on
timeout event. This implicitly suggest that those are two different events,
while in fact they are always printed together.
Print test name only once.
Before:
```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) ... test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```
After:
```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```
libtest: Fix unwrap panic on duplicate TestDesc
It is possible for different tests to collide to the same `TestDesc` when macros are involved. That is a bug, but it didn’t cause a panic until #81367. For now, change the code to ignore this problem.
Fixes#81852.
This will need to be applied to `beta` too.
It is possible for different tests to collide to the same TestDesc
when macros are involved. That is a bug, but it didn’t cause a panic
until #81367. For now, change the code to ignore this problem.
Fixes#81852.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
[libtest] Run the test synchronously when hitting thread limit
libtest currently panics if it hits the thread limit. This often results in spurious test failures (<code>thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }'</code> ... `error: test failed, to rerun pass '--lib'`). This PR makes it continue to run the test synchronously if it runs out of threads.
Closes#78165.
``@rustbot`` label: A-libtest T-libs
libtest: allow multiple filters
Libtest ignores any filters after the first. This changes it so that if multiple filters are passed, it will test against all of them.
This also affects compiletest to do the same.
Closes#30422
Upgrade wasm32 image to Ubuntu 20.04
This switches the wasm32 image, which is used to test
wasm32-unknown-emscripten, to Ubuntu 20.04. While at it, enable
most of the excluded tests, as they seem to work fine with some
minor fixes.
This switches the wasm32 image, which is used to test
wasm32-unknown-emscripten to Ubuntu 20.04. While at it, enable
most of the excluded tests, as they seem to work fine with some
minor fixes.
stabilise `cargo test -- --include-ignored`
stabilise `cargo test -- --include-ignored`
On stable there's no way to run ignored tests as well as the normal tests.
An example use case where stabilising this would help:
Exercism has some initial tests and then some additional ignored tests that people run currently with --ignore but currently they can't run all the tests in one go without being on nightly. It would be a little more ergonomic if this flag was stablilised.
( Fixes #65770 )
I built with ./x.py build -i library/test - but as libtest is a dylib is there an easy way to invoke it manually to check it's working as expected? (I've updated the automated tests.)
Print failure message on all tests that should panic, but don't
Fixes#80861. Tests with the `#[should_panic]` attribute should always print a failure message if no panic occurs, regardless of whether or not an `expected` panic message is specified.
This reduces the total complexity of checking timeouts from quadratic
to linear, and should also fix an unwrap of None on completion of an
already timed-out test.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Otherwise we can miss bugs where a test reports that it succeeded but
then panics within a TLS destructor.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
This already happens with should_panic tests without an expected
message. This commit fixes should_panic tests with an expected message
to have the same behavior.
