Add host triples to CI builders
This is a follow-up to #76415, which changed how x.py interprets cross-compilation target/host flags. This should fix the known cases, but I'm still working through CI logs before/after that PR to identify if anything else is missing.
It is really painful to inspect differences in what was built in CI if things
are appearing and disappearing randomly as they hover around the 100ms mark. No
matter what we choose there's always going to be quite a bit of variability on
CI in timing, so we're going to see things appear and vanish.
rustc is a natively cross-compiling compiler, and generally none of our steps
should care whether they are using a compiler built of triple A or B, just the
--target directive being passed to the running compiler. e.g., when building for
some target C, you don't generally want to build two stds: one with a host A
compiler and the other with a host B compiler. Just one std is sufficient.
rustbuild: Build tests with LLD if `use-lld = true` was passed
Addresses https://github.com/rust-lang/rust/pull/76127#discussion_r479932392.
Our test suite is generally ready to run with an explicitly specified linker (https://github.com/rust-lang/rust/pull/45191),
so LLD specified with `use-lld = true` works as well.
Only 4 tests fail (on `x86_64-pc-windows-msvc`):
```
ui/panic-runtime/lto-unwind.rs
run-make-fulldeps/debug-assertions
run-make-fulldeps/foreign-exceptions
run-make-fulldeps/test-harness
```
All of them are legitimate issues with LLD (or at least with combination Rust+LLD) and manifest in segfaults on access to TLS (https://github.com/rust-lang/rust/pull/76127#issuecomment-683473325). UPD: These issues are caused by https://github.com/rust-lang/rust/issues/72145 and appear because I had `-Ctarget-cpu=native` set.
UPD: Further commits build tests with LLD for non-MSVC targets and propagate LLD to more places when `use-lld` is enabled.
This currently includes libLLVM, llvm-config, and FileCheck, but will perhaps
expand to more tooling overtime. It should be considered entirely unstable and
may change at any time.
Often when modifying compiler code you'll miss that you've changed an API used
by unit tests, since x.py check didn't previously catch that.
It's also useful to have this for editing with rust-analyzer and similar tooling
where editing tests previously didn't notify you of errors in test files.
Add option to use the new symbol mangling in rustc/std
I don't know if this causes problems in some cases -- maybe it should be on by default for at least rustc. I've never encountered problems with it other than tools not supporting it, though.
cc @nnethercote
r? @eddyb
- Use stage 2 for makefile
- Move assert to builder
- Don't add an assert for --help
- Allow --stage 0 if passed explicitly
- Don't assert defaults during tests
Otherwise it's impossible to test the defaults!
### x.py build/test: stage 1
I've seen very few people who actually use full stage 2 builds on purpose. These compile rustc and libstd twice and don't give you much more information than a stage 1 build (except in rare cases like https://github.com/rust-lang/rust/pull/68692#discussion_r376392145). For new contributors, this makes the build process even more daunting than it already is. As long as CI is changed to use `--stage 2` I see no downside here.
### x.py bench/dist/install: stage 2
These commands have to do with a finished, optimized version of rustc. It seems very rare to want to use these with a stage 1 build.
### x.py doc: stage 0
Normally when you document things you're just fixing a typo. In this case there is no need to build the whole rust compiler, since the documentation will usually be the same when generated with the beta compiler or with stage 1.
Note that for this release cycle only there will be a significant different between stage0 and stage1 docs: https://github.com/rust-lang/rust/pull/73101. However most of the time this will not be the case.
Stabilize control-flow-guard codegen option
This is the stabilization PR discussed in #68793. It converts the `-Z control-flow-guard` debugging option into a codegen option (`-C control-flow-guard`), and changes the associated tests.
Generating the coverage map
@tmandry @wesleywiser
rustc now generates the coverage map and can support (limited)
coverage report generation, at the function level.
Example commands to generate a coverage report:
```shell
$ BUILD=$HOME/rust/build/x86_64-unknown-linux-gnu
$ $BUILD/stage1/bin/rustc -Zinstrument-coverage \
$HOME/rust/src/test/run-make-fulldeps/instrument-coverage/main.rs
$ LLVM_PROFILE_FILE="main.profraw" ./main
called
$ $BUILD/llvm/bin/llvm-profdata merge -sparse main.profraw -o main.profdata
$ $BUILD/llvm/bin/llvm-cov show --instr-profile=main.profdata main
```
r? @wesleywiser
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage instrumentation
Teach bootstrap about target files vs target triples
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
The current plan is that submodule tracks the `release` branch of
rust-analyzer, which is updated once a week.
rust-analyzer is a workspace (with a virtual manifest), the actual
binary is provide by `crates/rust-analyzer` package.
Note that we intentionally don't add rust-analyzer to `Kind::Test`,
for two reasons.
*First*, at the moment rust-analyzer's test suite does a couple of
things which might not work in the context of rust repository. For
example, it shells out directly to `rustup` and `rustfmt`. So, making
this work requires non-trivial efforts.
*Second*, it seems unlikely that running tests in rust-lang/rust repo
would provide any additional guarantees. rust-analyzer builds with
stable and does not depend on the specifics of the compiler, so
changes to compiler can't break ra, unless they break stability
guarantee. Additionally, rust-analyzer itself is gated on bors, so we
are pretty confident that test suite passes.
This commit moves the compiler-builtins-specific build logic from
`src/bootstrap/bin/rustc.rs` into the workspace `Cargo.toml`'s
`[profile]` configuration. Now that rust-lang/cargo#7253 is fixed we can
ensure that Cargo knows about debug assertions settings, and it can also
be configured to specifically disable debug assertions unconditionally
for compiler-builtins. This should improve rebuild logic when
debug-assertions settings change and also improve build-std integration
where Cargo externally now has an avenue to learn how to build
compiler-builtins as well.
Speed up bootstrap a little.
The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4.
An overview of the changes:
- Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config).
- Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior.
- Remove check for `_shim`, I believe all the `_shim` crates are now gone.
- Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree.
- Use relative paths in `./x.py test -h -v` output.
- Some code cleanup (remove unnecessary `find_compiler_crates`, etc.).
- Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output.
- Some doc comments.
Remove vestigial CI job msvc-aux.
This CI job isn't really doing anything, so it seems prudent to remove it.
For some history:
* This was introduced in #48809 when the msvc job was split in two to keep it under 2 hours (oh the good old days). At the time, this check-aux job did a bunch of things:
* tidy
* src/test/pretty
* src/test/run-pass/pretty
* src/test/run-fail/pretty
* src/test/run-pass-valgrind/pretty
* src/test/run-pass-fulldeps/pretty
* src/test/run-fail-fulldeps/pretty
* Tidy was removed in #60777.
* run-pass and run-pass-fulldeps moved to UI in #63029
* src/test/pretty removed in #58140
* src/test/run-fail moved to UI in #71185
* run-fail-fulldeps removed in #51285
Over time through attrition, the job was left with one lonely thing: `src/test/run-pass-valgrind/pretty`. And of course, this wasn't actually running the "pretty" tests. The normal `run-pass-valgrind` tests ran, and then when it tried to run in "pretty" mode, all the tests were ignored because compiletest thought nothing had changed (apparently compiletest isn't fingerprinting the mode? Needs more investigation…). `run-pass-valgrind` is already being run as part of `x86_64-msvc-1`, so there's no need to run it here.
I've taken the liberty of removing `src/test/run-pass-valgrind/pretty` as a distinct test. I'm guessing from the other PR's that the pretty tests should now live in `src/test/pretty`, and that the team has moved away from doing pretty tests on other parts of the `src/test` tree.
This should run much faster.
There are also some drive-by cleanups here to try to simplify things.
Also, the paths for in-tree crates are now displayed as relative
in `x.py test -h -v`.
