Add support for inputing via stdin with run-make-support
This PR adds the facility to set a input bytes that will be passed via the standard input.
This is useful for testing `rustc -` (and soon `rustdoc -`).
In #124611 took the approach of having a dedicated `run` method but it is not very convenient to use and would necessitate many functions, one for success, one for fail, ...
Instead this PR takes a different approach and allows setting the input bytes as if it were a parameter and when calling the (now custom) `output` function, we write the input bytes into stdin. I think this gives us maximum flexibility in the implementation and a simple interface for users.
To test this new logic I ported `tests/run-make/stdin-non-utf8/` to an `rmake.rs` one.
r? `@jieyouxu`
Cleanup: Rid the `rmake` test runners of `extern crate run_make_support;`
`run_make_support` is part of the *extern prelude* of `rmake` test runners rendering `extern crate run_make_support` redundant:
80451a485b/src/tools/compiletest/src/runtest.rs (L3826-L3827)
~~Contains some fmt'ing changes because I've enabled format-on-save in my editor and because we don't run `x fmt` for `rmake` test runners yet (this gets addressed by #124613). I can revert those if you'd like me to.~~ (reverted)
r? jieyouxu or testing-devex(?) or boostrap(?)
Port repr128-dwarf run-make test to rmake
This PR ports the repr128-dwarf run-make test to rmake, using the `gimli` crate instead of the `llvm-dwarfdump` command.
Note that this PR changes `rmake.rs` files to be compiled with the 2021 edition (previously no edition was passed to `rustc`, meaning they were compiled with the 2015 edition). This means that `panic!("{variable}")` will now work as expected in `rmake.rs` files (there's already a usage in the [wasm-symbols-not-exported test](aca749eefc/tests/run-make/wasm-symbols-not-exported/rmake.rs (L34)) that this will fix).
Tracking issue: #121876
Rewrite the `no-input-file.stderr` test in Rust and support diff
Rewrite the `no-input-file.stderr` test from #121876.
Use the `similar` lib to replace the `diff` command.
Fix ICE when there is a non-Unicode entry in the incremental crate directory
Fix the ICE that occurs when there is a non-Unicode entry in the incremental crate directory by replacing uses of `to_string_lossy` + `assert_no_characters_lost` with `to_str`. The added test would cause the compiler to ICE before this PR.
Codegen ZSTs without an allocation
This makes sure that &[] is equivalent to unsafe code (from_raw_parts(dangling, 0)). No new stable guarantee is intended about whether or not we do this, this is just an optimization.
This regressed in #67000 (no comments I can see about that regression in the PR, though it did change the test modified here). We had previously performed this optimization since #63635.
Port the 2 `rust-lld` run-make tests to `rmake`
In preparation for finalizing most of the `rust-lld` work, this PR ports the following tests to `rmake`:
- `tests/run-make/rust-lld`
- `tests/run-make/rust-lld-custom-target`
As they use `$(CGREP) -e` I added `regex` as an exported dependency to the `run_make_support` library.
Unfortunately, the most recent versions depend on `memchr` 2.6.0 but it's currently pinned at 2.5.0 in the workspace, and therefore had to settle for the older `regex-1.8.0`.
r? `@jieyouxu`
Allow workproducts without object files.
This pull request partially reverts changes from e16c3b4a44
Original motivation for this assert was described with "A WorkProduct without a saved file is useless"
which was true at the time but now it is possible to have work products with other types of files
(llvm-ir, asm, etc) and there are bugreports for this failure:
For example: https://github.com/rust-lang/rust/issues/123695
Fixes https://github.com/rust-lang/rust/issues/123234
Now existing `assert` and `.unwrap_or_else` are unified into a single
check that emits slightly more user friendly error message if an object
files was meant to be produced but it's missing
This pull request partially reverts changes from e16c3b4a44
Original motivation for this assert was described with "A WorkProduct without a saved file is useless"
which was true at the time but now it is possible to have work products with other types of files
(llvm-ir, asm, etc) and there are bugreports for this failure:
For example: https://github.com/rust-lang/rust/issues/123695
Fixes https://github.com/rust-lang/rust/issues/123234
Now existing `assert` and `.unwrap_or_else` are unified into a single
check that emits slightly more user friendly error message if an object
files was meant to be produced but it's missing
Linker flavors next steps: linker features
This is my understanding of the first step towards `@petrochenkov's` vision for the future of linker flavors, described in https://github.com/rust-lang/rust/pull/119906#issuecomment-1895693162 and the discussion that followed.
To summarize: having `Cc` and `Lld` embedded in linker flavors creates tension about naming, and a combinatorial explosion of flavors for each new linker feature we'd want to use. Linker features are an extension mechanism that is complementary to principal flavors, with benefits described in #119906.
The most immediate use of this flag would be to turn self-contained linking on and off via features instead of flavors. For example, `-Clinker-features=+/-lld` would toggle using lld instead of selecting a precise flavor, and would be "generic" and work cross-platform (whereas linker flavors are currently more tied to targets). Under this scheme, MCP510 is expected to be `-Clink-self-contained=+linker -Zlinker-features=+lld -Zunstable-options` (though for the time being, the original flags using lld-cc flavors still work).
I purposefully didn't add or document CLI support for `+/-cc`, as it would be a noop right now. I only expect that we'd initially want to stabilize `+/-lld` to begin with.
r? `@petrochenkov`
You had requested that minimal churn would be done to the 230 target specs and this does none yet: the linker features are inferred from the flavor since they're currently isomorphic. We of course expect this to change sooner rather than later.
In the future, we can allow targets to define linker features independently from their flavor, and remove the cc and lld components from the flavors to use the features instead, this actually doesn't need to block stabilization, as we discussed.
(Best reviewed per commit)
Set the host library path in run-make v2
When the build is configured with `[rust] rpath = false`, we need to set
`LD_LIBRARY_PATH` (or equivalent) to what would have been the `RPATH`,
so the compiler can find its own libraries. The old `tools.mk` code has
this environment prefixed in the `$(BARE_RUSTC)` variable, so we just
need to wire up something similar for run-make v2.
This is now set while building each `rmake.rs` itself, as well as in the
`rust-make-support` helpers for `rustc` and `rustdoc` commands. This is
also available in a `set_host_rpath` function for manual commands, like
in the `compiler-builtins` test.
When the build is configured with `[rust] rpath = false`, we need to set
`LD_LIBRARY_PATH` (or equivalent) to what would have been the `RPATH`,
so the compiler can find its own libraries. The old `tools.mk` code has
this environment prefixed in the `$(BARE_RUSTC)` variable, so we just
need to wire up something similar for run-make v2.
This is now set while building each `rmake.rs` itself, as well as in the
`rust-make-support` helpers for `rustc` and `rustdoc` commands. This is
also available in a `set_host_rpath` function for manual commands, like
in the `compiler-builtins` test.
Save/restore more items in cache with incremental compilation
Right now they don't play very well together, consider a simple example:
```
$ export RUSTFLAGS="--emit asm"
$ cargo new --lib foo
Created library `foo` package
$ cargo build -q
$ touch src/lib.rs
$ cargo build
error: could not copy
"/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.4qbzn9k8mosu50a5.rcgu.s"
to "/path/to/foo/target/debug/deps/foo-e307cc7fa7b6d64f.s":
No such file or directory (os error 2)
```
Touch triggers the rebuild, incremental compilation detects no changes (yay) and everything explodes while trying to copy files were they should go.
This pull request fixes it by copying and restoring more files in the incremental compilation cache
Fixes https://github.com/rust-lang/rust/issues/89149
Fixes https://github.com/rust-lang/rust/issues/88829
Related: https://internals.rust-lang.org/t/interaction-between-incremental-compilation-and-emit/20551
Port `run-make/issue-7349` to a codegen test
The test does not need to be a run-make test, it can use the codegen test infrastructure.
Also took the opportunity to rename the test to `no-redundant-item-monomorphization` so it's not just some opaque issue number.
Part of #121876.
Rewrite `core-no-fp-fmt-parse` test in Rust
Claiming the simple "core-no-fp-fmt-parse" test from #121876. `run_make_support` was altered with `arg_path` written in #121918 by `@abhay-51,` with additional doc comment.
Preliminary GSoC contribution for the project proposal mentored by `@jieyouxu.`
Simplify trim-paths feature by merging all debuginfo options together
This PR simplifies the trim-paths feature by merging all debuginfo options together, as described in https://github.com/rust-lang/rust/issues/111540#issuecomment-1994010274.
And also do some correctness fixes found during the review.
cc `@weihanglo`
r? `@michaelwoerister`
Rework rmake support library API
### Take 1: Strongly-typed API
Context: https://github.com/rust-lang/rust/pull/122448#discussion_r1523774427
> My 2 cents: from my experience with writing similar "test DSLs", I would suggest to create these helpers as soon as possible in the process (basically the first time someone needs them, not only after N similar usages), and basically treat any imperative code in these high-level tests as a maintenance burden, basically making them as declarative as possible. Otherwise it might be a bit annoying to keep refactoring the tests later once such helpers are available.
>
> I would even discourage the arg method and create explicit methods for setting things like unpretty, the output file etc., but this might be more controversial, as it will make the invoked command-line arguments more opaque.
cc `@Kobzol` for the testing DSL suggestion.
Example:
```rs
let output = Rustc::new()
.input_file("main.rs")
.emit(&[EmitKind::Metadata])
.extern_("stable", &stable_path)
.output();
```
### Take 2: xshell-based macro API
Example:
```rs
let sh = Shell::new()?;
let stable_path = stable_path.to_string_lossy();
let output = cmd!(sh, "rustc main.rs --emit=metadata --extern stable={stable_path}").output()?;
```
### Take 3: Weakly-typed API with a few helper methods
```rs
let output = Rustc::new()
.input("main.rs")
.emit("metadata")
.extern_("stable", &stable_path)
.output();
```
Print a backtrace in const eval if interrupted
Demo:
```rust
#![feature(const_eval_limit)]
#![const_eval_limit = "0"]
const OW: u64 = {
let mut res: u64 = 0;
let mut i = 0;
while i < u64::MAX {
res = res.wrapping_add(i);
i += 1;
}
res
};
fn main() {
println!("{}", OW);
}
```
```
╭ ➜ ben@archlinux:~/rust
╰ ➤ rustc +stage1 spin.rs
^Cerror[E0080]: evaluation of constant value failed
--> spin.rs:8:33
|
8 | res = res.wrapping_add(i);
| ^ Compilation was interrupted
note: erroneous constant used
--> spin.rs:15:20
|
15 | println!("{}", OW);
| ^^
note: erroneous constant used
--> spin.rs:15:20
|
15 | println!("{}", OW);
| ^^
|
= note: this note originates in the macro `$crate::format_args_nl` which comes from the expansion of the macro `println` (in Nightly builds, run with -Z macro-backtrace for more info)
error: aborting due to previous error
For more information about this error, try `rustc --explain E0080`.
```
"Handle" calls to upstream monomorphizations in compiler_builtins
This is pretty cooked, but I think it works.
compiler-builtins has a long-standing problem that at link time, its rlib cannot contain any calls to `core`. And yet, in codegen we _love_ inserting calls to symbols in `core`, generally from various panic entrypoints.
I intend this PR to attack that problem as completely as possible. When we generate a function call, we now check if we are generating a function call from `compiler_builtins` and whether the callee is a function which was not lowered in the current crate, meaning we will have to link to it.
If those conditions are met, actually generating the call is asking for a linker error. So we don't. If the callee diverges, we lower to an abort with the same behavior as `core::intrinsics::abort`. If the callee does not diverge, we produce an error. This means that compiler-builtins can contain panics, but they'll SIGILL instead of panicking. I made non-diverging calls a compile error because I'm guessing that they'd mostly get into compiler-builtins by someone making a mistake while working on the crate, and compile errors are better than linker errors. We could turn such calls into aborts as well if that's preferred.
fix `long-linker-command-lines` failure caused by `rust.rpath=false`
Fixes `long-linker-command-lines` test failure (which happens when `rust.rpath` is set to `false`) by adjusting `LD_LIBRARY_PATH`.
Fixes https://github.com/rust-lang/rust/issues/90921
This commit rewrites a number of `run-make` tests centered around wasm
to instead use `rmake.rs` and additionally use the `wasm32-wasip1`
target instead of `wasm32-unknown-unknown`. Testing no longer requires
Node.js and additionally uses the `wasmparser` crate from crates.io to
parse outputs and power assertions.
Expose the Freeze trait again (unstably) and forbid implementing it manually
non-emoji version of https://github.com/rust-lang/rust/pull/121501
cc #60715
This trait is useful for generic constants (associated consts of generic traits). See the test (`tests/ui/associated-consts/freeze.rs`) added in this PR for a usage example. The builtin `Freeze` trait is the only way to do it, users cannot work around this issue.
It's also a useful trait for building some very specific abstrations, as shown by the usage by the `zerocopy` crate: https://github.com/google/zerocopy/issues/941
cc ```@RalfJung```
T-lang signed off on reexposing this unstably: https://github.com/rust-lang/rust/pull/121501#issuecomment-1969827742
Fixing shellcheck comments on lvi test script
Running `shellcheck` on `tests/run-make/x86_64-fortanix-unknown-sgx-lvi/script.sh` gives plenty of warnings. This PR fixes those issues. For completeness: #121683 fixes another warning as well
Fix LVI tests after frame pointers are enabled by default
#121203 enables frame pointers by default. This affects LVI mitigations for the `x86_64-fortanix-unknown-sgx` target. LVI remained mitigated correctly, but the tests were too strict.
``@nshyrei`` , ``@jethrogb``
Introduce `run-make` V2 infrastructure, a `run_make_support` library and port over 2 tests as example
## Preface
See [issue #40713: Switch run-make tests from Makefiles to rust](https://github.com/rust-lang/rust/issues/40713) for more context.
## Basic Description of `run-make` V2
`run-make` V2 aims to eliminate the dependency on `make` and `Makefile`s for building `run-make`-style tests. Makefiles are replaced by *recipes* (`rmake.rs`). The current implementation runs `run-make` V2 tests in 3 steps:
1. We build the support library `run_make_support` which the `rmake.rs` recipes depend on as a tool lib.
2. We build the recipe `rmake.rs` and link in the support library.
3. We run the recipe to build and run the tests.
`rmake.rs` is basically a replacement for `Makefile`, and allows running arbitrary Rust code. The support library is built using cargo, and so can depend on external crates if desired.
The infrastructure implemented by this PR is very barebones, and is the minimally required infrastructure needed to build, run and pass the two example `run-make` tests ported over to the new infrastructure.
### Example `run-make` V2 test
```rs
// ignore-tidy-linelength
extern crate run_make_support;
use std::path::PathBuf;
use run_make_support::{aux_build, rustc};
fn main() {
aux_build()
.arg("--emit=metadata")
.arg("stable.rs")
.run();
let mut stable_path = PathBuf::from(env!("TMPDIR"));
stable_path.push("libstable.rmeta");
let output = rustc()
.arg("--emit=metadata")
.arg("--extern")
.arg(&format!("stable={}", &stable_path.to_string_lossy()))
.arg("main.rs")
.run();
let stderr = String::from_utf8_lossy(&output.stderr);
let version = include_str!(concat!(env!("S"), "/src/version"));
let expected_string = format!("stable since {}", version.trim());
assert!(stderr.contains(&expected_string));
}
```
## Follow Up Work
- [ ] Adjust rustc-dev-guide docs
test: enable `unpacked-lto` tests
This enables the correct `unpacked-lto` tests.
Not sure whether `.o` should be removed.
They are bitcode for linker-plugin-lto, though there might be some `.o` for `#[no_builtins]`?
rename 'try' intrinsic to 'catch_unwind'
The intrinsic has nothing to do with `try` blocks, and corresponds to the stable `catch_unwind` function, so this makes a lot more sense IMO.
Also rename Miri's special function while we are at it, to reflect the level of abstraction it works on: it's an unwinding mechanism, on which Rust implements panics.
Allow tests to specify a `//@ filecheck-flags:` header
This allows individual codegen/assembly/mir-opt tests to pass extra flags to the LLVM `filecheck` tool as needed.
---
The original motivation was noticing that `tests/run-make/instrument-coverage` was very close to being an ordinary codegen test, except that it needs some extra logic to set up platform-specific variables to be passed into filecheck.
I then saw the comment in `verify_with_filecheck` indicating that a `filecheck-flags` header might be useful for other purposes as well.
Always use WaitOnAddress on Win10+
`WaitOnAddress` and `WakeByAddressSingle` are always available since Windows 8 so they can now be used without needing to delay load. I've also moved the Windows 7 thread parking fallbacks into a separate sub-module.
This test was already very close to being an ordinary codegen test, except that
it needed some extra logic to set a few variables based on (target) platform
characteristics.
Now that we have support for `//@ filecheck-flags:`, we can instead set those
variables using the normal test revisions mechanism.
llvm: change data layout bug to an error and make it trigger more
Fixes#33446.
Don't skip the inconsistent data layout check for custom LLVMs or non-built-in targets.
With #118708, all targets will have a simple test that would trigger this error if LLVM's data layouts do change - so data layouts would be corrected during the LLVM upgrade. Therefore, with builtin targets, this error won't happen with our LLVM because each target will have been confirmed to work. With non-builtin targets, this error is probably useful to have because you can change the data layout in your target and if it is wrong then that could lead to bugs.
When using a custom LLVM, the same justification makes sense for non-builtin targets as with our LLVM, the user can update their target to match their LLVM and that's probably a good thing to do. However, with a custom LLVM, the user cannot change the builtin target data layouts if they don't match - though given that the compiler's data layout is used for layout computation and a bunch of other things - you could get some bugs because of the mismatch and probably want to know about that. I'm not sure if this is something that people do and is okay, but I doubt it?
`CFG_LLVM_ROOT` was also always set during local development with `download-ci-llvm` so this bug would never trigger locally.
In #33446, two points are raised:
- In the issue itself, changing this from a `bug!` to a proper error is what is suggested, by using `isCompatibleDataLayout` from LLVM, but that function still just does the same thing that we do and check for equality, so I've avoided the additional code necessary to do that FFI call.
- `@Mark-Simulacrum` suggests a different check is necessary to maintain backwards compatibility with old LLVM versions. I don't know how often this comes up, but we can do that with some simple string manipulation + LLVM version checks as happens already for LLVM 17 just above this diff.
