Display walltime benchmarks with subnanosecond precision
With modern CPUs running at more than one cycle per nanosecond the current precision is insufficient to resolve differences worth several cycles per iteration.
Granted, walltime benchmarks often are noisy but occasionally, especially when no allocations are involved, the difference really is just a few cycles.
example results when benchmarking 1-4 serialized ADD instructions and an empty bench body
```
running 4 tests
test add ... bench: 0.24 ns/iter (+/- 0.00)
test add2 ... bench: 0.48 ns/iter (+/- 0.01)
test add3 ... bench: 0.72 ns/iter (+/- 0.01)
test add4 ... bench: 0.96 ns/iter (+/- 0.01)
test empty ... bench: 0.24 ns/iter (+/- 0.00)
```
Document tests in the `run-make` directory (A to C)
Part of the #121876 project.
This PR adds comments to some `run-make` tests which lack one, explaining _what_ is being tested. If possible, a link to the relevant PR or Issue responsible for the test is also provided.
This will help the porting efforts to `rmake.rs`, and will also allow maintainers to focus efforts on tests which are more pertinent to port. For example, [this test](https://github.com/rust-lang/rust/blob/master/tests/run-make/cat-and-grep-sanity-check/Makefile) will become useless after all tests containing `CGREP` are successfully ported.
In order to simplify review and at the suggestion of Kobzol on the rust-lang #gsoc Zulip, only the first 23 comments are part of this PR. If it is merged, future PRs will ensue commenting the rest of the tests.
Could be an UI test:
- `dep-info-doesnt-run-much`
Allow fmt to run on rmake.rs test files
As discussed with `@jieyouxu,` `rmake.rs` from the `run-make` testsuite would benefit from being formatted as well.
Only thing needed to be done for it to work: allow support for `!` in our `rustfmt.toml` file parsing.
r? `@onur-ozkan`
Add support for inputing via stdin with run-make-support
This PR adds the facility to set a input bytes that will be passed via the standard input.
This is useful for testing `rustc -` (and soon `rustdoc -`).
In #124611 took the approach of having a dedicated `run` method but it is not very convenient to use and would necessitate many functions, one for success, one for fail, ...
Instead this PR takes a different approach and allows setting the input bytes as if it were a parameter and when calling the (now custom) `output` function, we write the input bytes into stdin. I think this gives us maximum flexibility in the implementation and a simple interface for users.
To test this new logic I ported `tests/run-make/stdin-non-utf8/` to an `rmake.rs` one.
r? `@jieyouxu`
Cleanup: Rid the `rmake` test runners of `extern crate run_make_support;`
`run_make_support` is part of the *extern prelude* of `rmake` test runners rendering `extern crate run_make_support` redundant:
80451a485b/src/tools/compiletest/src/runtest.rs (L3826-L3827)
~~Contains some fmt'ing changes because I've enabled format-on-save in my editor and because we don't run `x fmt` for `rmake` test runners yet (this gets addressed by #124613). I can revert those if you'd like me to.~~ (reverted)
r? jieyouxu or testing-devex(?) or boostrap(?)
Port repr128-dwarf run-make test to rmake
This PR ports the repr128-dwarf run-make test to rmake, using the `gimli` crate instead of the `llvm-dwarfdump` command.
Note that this PR changes `rmake.rs` files to be compiled with the 2021 edition (previously no edition was passed to `rustc`, meaning they were compiled with the 2015 edition). This means that `panic!("{variable}")` will now work as expected in `rmake.rs` files (there's already a usage in the [wasm-symbols-not-exported test](aca749eefc/tests/run-make/wasm-symbols-not-exported/rmake.rs (L34)) that this will fix).
Tracking issue: #121876
Rewrite the `no-input-file.stderr` test in Rust and support diff
Rewrite the `no-input-file.stderr` test from #121876.
Use the `similar` lib to replace the `diff` command.
Fix ICE when there is a non-Unicode entry in the incremental crate directory
Fix the ICE that occurs when there is a non-Unicode entry in the incremental crate directory by replacing uses of `to_string_lossy` + `assert_no_characters_lost` with `to_str`. The added test would cause the compiler to ICE before this PR.
Codegen ZSTs without an allocation
This makes sure that &[] is equivalent to unsafe code (from_raw_parts(dangling, 0)). No new stable guarantee is intended about whether or not we do this, this is just an optimization.
This regressed in #67000 (no comments I can see about that regression in the PR, though it did change the test modified here). We had previously performed this optimization since #63635.
Port the 2 `rust-lld` run-make tests to `rmake`
In preparation for finalizing most of the `rust-lld` work, this PR ports the following tests to `rmake`:
- `tests/run-make/rust-lld`
- `tests/run-make/rust-lld-custom-target`
As they use `$(CGREP) -e` I added `regex` as an exported dependency to the `run_make_support` library.
Unfortunately, the most recent versions depend on `memchr` 2.6.0 but it's currently pinned at 2.5.0 in the workspace, and therefore had to settle for the older `regex-1.8.0`.
r? `@jieyouxu`
Allow workproducts without object files.
This pull request partially reverts changes from e16c3b4a44
Original motivation for this assert was described with "A WorkProduct without a saved file is useless"
which was true at the time but now it is possible to have work products with other types of files
(llvm-ir, asm, etc) and there are bugreports for this failure:
For example: https://github.com/rust-lang/rust/issues/123695
Fixes https://github.com/rust-lang/rust/issues/123234
Now existing `assert` and `.unwrap_or_else` are unified into a single
check that emits slightly more user friendly error message if an object
files was meant to be produced but it's missing
This pull request partially reverts changes from e16c3b4a44
Original motivation for this assert was described with "A WorkProduct without a saved file is useless"
which was true at the time but now it is possible to have work products with other types of files
(llvm-ir, asm, etc) and there are bugreports for this failure:
For example: https://github.com/rust-lang/rust/issues/123695
Fixes https://github.com/rust-lang/rust/issues/123234
Now existing `assert` and `.unwrap_or_else` are unified into a single
check that emits slightly more user friendly error message if an object
files was meant to be produced but it's missing
