Commit Graph

100 Commits

Author SHA1 Message Date
bjorn3
e015b9ee80 Remove unused allow deprecated 2021-12-29 18:06:01 +01:00
bjorn3
f710bc5fc0 Remove unused feature gates from libtest 2021-12-29 16:36:22 +01:00
bjorn3
1ea4810c84 Remove #![crate_name] attribute from libtest
The crate name is already set in Cargo.toml. The comment says there is
some logic in the compiler that reads #![crate_name] and not
--crate-name, but I can't find it. Removing it seems to work fine.
2021-12-29 16:36:08 +01:00
bjorn3
75d8339cdd Replace TDynBenchFn with Fn(&mut Bencher) 2021-12-29 16:17:50 +01:00
Alex Crichton
d2a3c24a95 Update more rustc/libtest things for wasm64
* Add wasm64 variants for inline assembly along the same lines as wasm32
* Update a few directives in libtest to check for `target_family`
  instead of `target_arch`
* Update some rustc codegen and typechecks specialized for wasm32 to
  also work for wasm64.
2021-11-10 08:35:42 -08:00
Matthias Krüger
28ef4169cc clippy::perf fixes 2021-11-04 21:07:56 +01:00
Jubilee
37f17bca7c
Rollup merge of #89082 - smoelius:master, r=kennytm
Implement #85440 (Random test ordering)

This PR adds `--shuffle` and `--shuffle-seed` options to `libtest`. The options are similar to the [`-shuffle` option](c894b442d1/src/testing/testing.go (L1482-L1499)) that was recently added to Go.

Here are the relevant parts of the help message:
```
        --shuffle       Run tests in random order
        --shuffle-seed SEED
                        Run tests in random order; seed the random number
                        generator with SEED
...
By default, the tests are run in alphabetical order. Use --shuffle or set
RUST_TEST_SHUFFLE to run the tests in random order. Pass the generated
"shuffle seed" to --shuffle-seed (or set RUST_TEST_SHUFFLE_SEED) to run the
tests in the same order again. Note that --shuffle and --shuffle-seed do not
affect whether the tests are run in parallel.
```
Is an RFC needed for this?
2021-10-07 20:26:12 -07:00
Manish Goregaokar
b4615b5bf9
Rollup merge of #89324 - yoshuawuyts:hardware-parallelism, r=m-ou-se
Rename `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`

_Tracking issue: https://github.com/rust-lang/rust/issues/74479_

This PR renames  `std:🧵:available_conccurrency` to `std:🧵:available_parallelism`.

## Rationale

The API was initially named `std:🧵:hardware_concurrency`, mirroring the [C++ API of the same name](https://en.cppreference.com/w/cpp/thread/thread/hardware_concurrency). We eventually decided to omit any reference to the word "hardware" after [this comment](https://github.com/rust-lang/rust/pull/74480#issuecomment-662045841). And so we ended up with `available_concurrency` instead.

---

For a talk I was preparing this week I was reading through ["Understanding and expressing scalable concurrency" (A. Turon, 2013)](http://aturon.github.io/academic/turon-thesis.pdf), and the following passage stood out to me (emphasis mine):

> __Concurrency is a system-structuring mechanism.__ An interactive system that deals with disparate asynchronous events is naturally structured by division into concurrent threads with disparate responsibilities. Doing so creates a better fit between problem and solution, and can also decrease the average latency of the system by preventing long-running computations from obstructing quicker ones.

> __Parallelism is a resource.__ A given machine provides a certain capacity for parallelism, i.e., a bound on the number of computations it can perform simultaneously. The goal is to maximize throughput by intelligently using this resource. For interactive systems, parallelism can decrease latency as well.

_Chapter 2.1: Concurrency is not Parallelism. Page 30._

---

_"Concurrency is a system-structuring mechanism. Parallelism is a resource."_ — It feels like this accurately captures the way we should be thinking about these APIs. What this API returns is not "the amount of concurrency available to the program" which is a property of the program, and thus even with just a single thread is effectively unbounded. But instead it returns "the amount of _parallelism_ available to the program", which is a resource hard-constrained by the machine's capacity (and can be further restricted by e.g. operating systems).

That's why I'd like to propose we rename this API from `available_concurrency` to `available_parallelism`. This still meets the criteria we previously established of not attempting to define what exactly we mean by "hardware", "threads", and other such words. Instead we only talk about "concurrency" as an abstract resource available to our program.

r? `@joshtriplett`
2021-10-06 12:33:17 -07:00
Samuel E. Moelius III
32b6ac5b44 Check allow_unstable before checking environment variables 2021-09-30 12:57:34 -04:00
Samuel E. Moelius III
a6738c7231 Add tests 2021-09-29 21:51:59 -04:00
Samuel E. Moelius III
fa23d4fe93 Implement #85440 2021-09-29 21:51:46 -04:00
Guillaume Gomez
e601554dc0
Rollup merge of #89235 - yaahc:junit-formatting, r=kennytm
make junit output more consistent with default format

The default format of libtest includes new-lines between each section to ensure the label output from cargo is on it's own line

<pre><font color="#A1B56C"><b>❯</b></font> <font color="#A1B56C">cargo</font><font color="#D8D8D8"> </font><font color="#A1B56C">test</font>
<font color="#A1B56C"><b>   Compiling</b></font> test-test v0.1.0 (/home/jlusby/tmp/test-test)
<font color="#A1B56C"><b>    Finished</b></font> test [unoptimized + debuginfo] target(s) in 0.59s
<font color="#A1B56C"><b>     Running</b></font> unittests (target/debug/deps/test_test-639f369234319c09)

running 1 test
test tests::it_works ... <font color="#A1B56C">ok</font>

test result: <font color="#A1B56C">ok</font>. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

<font color="#A1B56C"><b>   Doc-tests</b></font> test-test

running 0 tests

test result: <font color="#A1B56C">ok</font>. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

</pre>

But when the junit outputter was added to libtest these newlines were omitted, resulting in some "fun" output when run via cargo.

Note the `Doc-tests` text at the end of the first line of xml.

<pre><font color="#A1B56C"><b>❯</b></font> <font color="#A1B56C">cargo</font><font color="#D8D8D8"> </font><font color="#A1B56C">test</font><font color="#D8D8D8"> </font><font color="#A1B56C">--</font><font color="#D8D8D8"> </font><font color="#A1B56C">-Zunstable-options</font><font color="#D8D8D8"> </font><font color="#A1B56C">--format</font><font color="#D8D8D8"> </font><font color="#A1B56C">junit</font>
<font color="#A1B56C"><b>    Finished</b></font> test [unoptimized + debuginfo] target(s) in 0.00s
<font color="#A1B56C"><b>     Running</b></font> unittests (target/debug/deps/test_test-639f369234319c09)
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;1&quot; skipped=&quot;0&quot; &gt;&lt;testcase classname=&quot;tests&quot; name=&quot;it_works&quot; time=&quot;0&quot;/&gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;<font color="#A1B56C"><b>   Doc-tests</b></font> test-test
&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;0&quot; skipped=&quot;0&quot; &gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;

</pre>

After this PR the junit output includes the same style of newlines as the pretty format

<pre><font color="#A1B56C"><b>❯</b></font> <font color="#A1B56C">cargo</font><font color="#D8D8D8"> </font><font color="#A1B56C">test</font><font color="#D8D8D8"> </font><font color="#A1B56C">--</font><font color="#D8D8D8"> </font><font color="#A1B56C">-Zunstable-options</font><font color="#D8D8D8"> </font><font color="#A1B56C">--format</font><font color="#D8D8D8"> </font><font color="#A1B56C">junit</font>
<font color="#A1B56C"><b>   Compiling</b></font> test-test v0.1.0 (/home/jlusby/tmp/test-test)
<font color="#A1B56C"><b>    Finished</b></font> test [unoptimized + debuginfo] target(s) in 0.39s
<font color="#A1B56C"><b>     Running</b></font> unittests (target/debug/deps/test_test-42c2320bb9450c69)

&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;1&quot; skipped=&quot;0&quot; &gt;&lt;testcase classname=&quot;tests&quot; name=&quot;it_works&quot; time=&quot;0&quot;/&gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;

<font color="#A1B56C"><b>   Doc-tests</b></font> test-test

&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt;&lt;testsuites&gt;&lt;testsuite name=&quot;test&quot; package=&quot;test&quot; id=&quot;0&quot; errors=&quot;0&quot; failures=&quot;0&quot; tests=&quot;0&quot; skipped=&quot;0&quot; &gt;&lt;system-out/&gt;&lt;system-err/&gt;&lt;/testsuite&gt;&lt;/testsuites&gt;

</pre>
2021-09-28 20:00:15 +02:00
Yoshua Wuyts
6cc91cb3d8 Rename std:🧵:available_onccurrency to std:🧵:available_parallelism 2021-09-28 14:59:33 +02:00
Jane Lusby
0911069feb
Apply suggestions from code review
Co-authored-by: kennytm <kennytm@gmail.com>
2021-09-27 14:50:35 -07:00
Luca Barbato
160b93903c Expose the std_detect env_override feature 2021-09-25 20:30:25 +02:00
Jane Lusby
7779eb74c8 make junit output more consistent with default format 2021-09-24 14:45:09 -07:00
Matthias Krüger
c1e96085d3 don't clone types that are Copy (clippy::clone_on_copy) 2021-09-11 10:18:56 +02:00
Fabian Wolff
79adda930f Ignore automatically derived impls of Clone and Debug in dead code analysis 2021-09-09 19:49:07 +02:00
Jade
3cf820e17d rfc3052: Remove authors field from Cargo manifests
Since RFC 3052 soft deprecated the authors field anyway, hiding it from
crates.io, docs.rs, and making Cargo not add it by default, and it is
not generally up to date/useful information, we should remove it from
crates in this repo.
2021-07-29 14:56:05 -07:00
Charles Lew
4486795d02 Remove unused stuff and switch to pub(crate) whenever possible. 2021-07-18 22:00:41 +08:00
Charles Lew
950f569c91 Fix compilation errors. 2021-07-18 20:51:47 +08:00
Charles Lew
9c11113b4f Move library/term/src to library/test/src/term/. 2021-07-18 20:49:18 +08:00
Mark Rousskov
06661ba759 Update to new bootstrap compiler 2021-06-28 11:30:49 -04:00
Smitty
bdfcb88e8b Use HTTPS links where possible 2021-06-23 16:26:46 -04:00
Deadbeef
e4b3131584
Use as_secs_f64 in JunitFormatter 2021-06-17 03:23:17 +08:00
bors
3740ba2a7d Auto merge of #84863 - ABouttefeux:libtest, r=m-ou-se
Show test type during prints

Test output can sometimes be confusing. For example doctest with the no_run argument are displayed the same way than test that are run.

During #83857 I got the feedback that test output can be confusing.

For the moment test output is
```
test $DIR/test-type.rs - f (line 12) ... ignored
test $DIR/test-type.rs - f (line 15) ... ok
test $DIR/test-type.rs - f (line 21) ... ok
test $DIR/test-type.rs - f (line 6) ... ok
```

I propose to change output by indicating the test type as
```
test $DIR/test-type.rs - f (line 12) ... ignored
test $DIR/test-type.rs - f (line 15) - compile ... ok
test $DIR/test-type.rs - f (line 21) - compile fail ... ok
test $DIR/test-type.rs - f (line 6) ... ok
```
by indicating the test type after the test name (and in the case of doctest after the function name and line) and before the "...".

------------

Note: this is a proof of concept, the implementation is probably not optimal as the properties added in `TestDesc` are only use in the display and does not represent actual change of behavior, maybe `TestType::DocTest` could have fields
2021-06-06 09:13:59 +00:00
Joshua Nelson
7411a9e7cc rustdoc: link to stable/beta docs consistently in documentation
## User-facing changes

- Intra-doc links to primitives that currently go to rust-lang.org/nightly/std/primitive.x.html will start going to channel that rustdoc was built with. Nightly will continue going to /nightly; Beta will link to /beta; stable compilers will link to /1.52.1 (or whatever version they were built as).
- Cross-crate links from std to core currently go to /nightly unconditionally. They will start going to /1.52.0 on stable channels (but remain the same on nightly channels).
- Intra-crate links from std to std (or core to core) currently go to the same URL they are hosted at; they will continue to do so. Notably, this is different from everything else because it can preserve the distinction between /stable and /1.52.0 by using relative links.

Note that "links" includes both intra-doc links and rustdoc's own
automatically generated hyperlinks.

 ## Implementation changes

- Update the testsuite to allow linking to /beta and /1.52.1 in docs
- Use an html_root_url for the standard library that's dependent on the channel

  This avoids linking to nightly docs on stable.

- Update rustdoc to use channel-dependent links for primitives from an
  unknown crate

- Set DOC_RUST_LANG_ORG_CHANNEL from bootstrap to ensure it's in sync
- Include doc.rust-lang.org in the channel
2021-06-04 14:18:21 -04:00
bors
1c6868aa21 Auto merge of #84568 - andoriyu:libtest/junit_formatter, r=yaahc
feat(libtest): Add JUnit formatter

tracking issue: https://github.com/rust-lang/rust/issues/85563

Add an alternative formatter to `libtest`. Formatter produces valid xml that later can be interpreted as JUnit report.

Caveats:

- `timestamp` is required by schema, but every viewer/parser ignores it. Attribute is not set to avoid depending on chrono;
- Running all "suits" (unit tests, doc-tests and integration tests) will produce a mess;
- I couldn't find a way to get integration test binary name, so it's just goes by "integration";

Sample output for unit tests (pretty printed by 3rd party tool):
```
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" package="test" id="0" errors="0" failures="0" tests="13" skipped="1">
    <testcase classname="results::tests" name="test_completed_bad" time="0"/>
    <testcase classname="results::tests" name="suite_started" time="0"/>
    <testcase classname="results::tests" name="suite_ended_ok" time="0"/>
    <testcase classname="results::tests" name="suite_ended_bad" time="0"/>
    <testcase classname="junit::tests" name="test_failed_output" time="0"/>
    <testcase classname="junit::tests" name="test_simple_output" time="0"/>
    <testcase classname="junit::tests" name="test_multiple_outputs" time="0"/>
    <testcase classname="results::tests" name="test_completed_ok" time="0"/>
    <testcase classname="results::tests" name="test_stared" time="0"/>
    <testcase classname="junit::tests" name="test_generate_xml_no_error_single_testsuite" time="0"/>
    <testcase classname="results::tests" name="test_simple_output" time="0"/>
    <testcase classname="test" name="should_panic" time="0"/>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>
```

Sample output for integration tests (pretty printed by 3rd party tool):

```
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" package="test" id="0" errors="0" failures="0" tests="1" skipped="0">
    <testcase classname="integration" name="test_add" time="0"/>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>
```

Sample output for Doc-tests (pretty printed by 3rd party tool):

```
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="test" package="test" id="0" errors="0" failures="0" tests="1" skipped="0">
    <testcase classname="src/lib.rs" name="(line 2)" time="0"/>
    <system-out/>
    <system-err/>
  </testsuite>
</testsuites>
```
2021-05-27 21:14:55 +00:00
Aliénore Bouttefeux
6de13c3ffc change based on review 2021-05-18 18:18:28 +02:00
Aliénore Bouttefeux
3d81806352 remove mode for run and ignore tests 2021-05-16 22:33:41 +02:00
Alan Egerton
67e8f12307
Expose Concurrent (private type in public i'face) 2021-05-14 13:28:56 +01:00
Aliénore Bouttefeux
f6b8b78063 add bootstrap cfg 2021-05-09 13:37:09 +02:00
Aliénore Bouttefeux
6e99cb3989 change based on review 2021-05-03 20:17:15 +02:00
Aliénore Bouttefeux
347ed001e8 proof of concept add test type on prints 2021-05-03 15:22:19 +02:00
Andrey Cherkashin
9f83e2290a Better output for junit formatter 2021-04-30 17:16:09 -07:00
Andrey Cherkashin
4b4d06ae82
Update junit.rs 2021-04-26 11:42:13 -07:00
Andrey Cherkashin
38485a9e34 feat(libtest): Add JUnit formatter 2021-04-25 15:51:50 -07:00
Ralf Jung
23d54ad96f move core::hint::black_box under its own feature gate 2021-04-25 11:08:12 +02:00
bors
5e73bd1040 Auto merge of #82300 - andersk:libtest-id, r=Amanieu
libtest: Index tests by a unique TestId

This more robustly avoids problems with duplicate `TestDesc`. See #81852 and #82274.

Cc `@Mark-Simulacrum.`
2021-04-12 13:30:30 +00:00
klensy
229d199994 lazily calls some fns 2021-03-27 10:20:32 +03:00
Anders Kaseorg
3c42d9fe20 libtest: Index tests by a unique TestId
This more robustly avoids problems with duplicate TestDesc.  See #81852
and #82274.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-03-24 23:08:13 -07:00
Mara Bos
8dc0ae24bc Remove Option::{unwrap_none, expect_none}. 2021-03-14 12:54:34 +01:00
bors
247337f409 Auto merge of #82350 - ehuss:test-chapter, r=jyn514
Add a chapter on the test harness.

There isn't really any online documentation on the test harness, so this adds a chapter to the rustc book which provides information on how the harness works and details on the command-line options.
2021-02-28 09:18:27 +00:00
Guillaume Gomez
0db8349fff
Rollup merge of #81940 - jhpratt:stabilize-str_split_once, r=m-ou-se
Stabilize str_split_once

Closes #74773
2021-02-26 15:52:29 +01:00
Yuki Okushi
d7fb4de791
Rollup merge of #82349 - tmiasko:pretty-test-timeout, r=Mark-Simulacrum
test: Print test name only once on timeout

Pretty formatter when using multiple test threads displays test name twice on
timeout event. This implicitly suggest that those are two different events,
while in fact they are always printed together.

Print test name only once.

Before:

```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) ... test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```

After:

```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```
2021-02-21 15:26:50 +09:00
Eric Huss
87ac39c800 Add a chapter on the test harness. 2021-02-20 16:12:11 -08:00
Tomasz Miąsko
88753cead8 test: Print test name only once on timeout
Pretty formatter when using multiple test threads displays test name twice on
timeout event. This implicitly suggest that those are two different events,
while in fact they are always printed together.

Print test name only once.

Before:

```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) ... test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```

After:

```
running 3 tests
test src/lib.rs - c (line 16) ... ok
test src/lib.rs - a (line 3) ... ok
test src/lib.rs - b (line 9) has been running for over 60 seconds
test src/lib.rs - b (line 9) ... ok
```
2021-02-21 00:00:00 +00:00
Dylan DPC
36a348bdc0
Rollup merge of #82274 - andersk:test-unwrap, r=Mark-Simulacrum
libtest: Fix unwrap panic on duplicate TestDesc

It is possible for different tests to collide to the same `TestDesc` when macros are involved. That is a bug, but it didn’t cause a panic until #81367. For now, change the code to ignore this problem.

Fixes #81852.

This will need to be applied to `beta` too.
2021-02-19 02:49:13 +01:00
Tomasz Miąsko
2380090f49 Remove unsafe impl Send for CompletedTest & TestResult 2021-02-19 00:00:00 +00:00
Anders Kaseorg
1605af015c libtest: Fix unwrap panic on duplicate TestDesc.
It is possible for different tests to collide to the same TestDesc
when macros are involved.  That is a bug, but it didn’t cause a panic
until #81367.  For now, change the code to ignore this problem.

Fixes #81852.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2021-02-18 12:42:45 -08:00