rustc_codegen_llvm: add support for writing summary bitcode
Typical uses of ThinLTO don't have any use for this as a standalone file, but distributed ThinLTO uses this to make the linker phase more efficient. With clang you'd do something like `clang -flto=thin -fthin-link-bitcode=foo.indexing.o -c foo.c` and then get both foo.o (full of bitcode) and foo.indexing.o (just the summary or index part of the bitcode). That's then usable by a two-stage linking process that's more friendly to distributed build systems like bazel, which is why I'm working on this area.
I talked some to `@teresajohnson` about naming in this area, as things seem to be a little confused between various blog posts and build systems. "bitcode index" and "bitcode summary" tend to be a little too ambiguous, and she tends to use "thin link bitcode" and "minimized bitcode" (which matches the descriptions in LLVM). Since the clang option is thin-link-bitcode, I went with that to try and not add a new spelling in the world.
Per `@dtolnay,` you can work around the lack of this by using `lld --thinlto-index-only` to do the indexing on regular .o files of bitcode, but that is a bit wasteful on actions when we already have all the information in rustc and could just write out the matching minimized bitcode. I didn't test that at all in our infrastructure, because by the time I learned that I already had this patch largely written.
rust-lld: fallback to rustc's sysroot if there's no path to the linker in the target sysroot
As seen in #125246, some sysroots don't expect to contain `rust-lld` and want to keep it that way, so we fallback to the default rustc sysroot if there is no path to the linker in any of the sysroot tools search paths. This is how we locate codegen-backends' dylibs already.
People also have requested an error if none of these search paths contain the self-contained linker directory, so there's also an error in that case.
r? `@petrochenkov` cc `@ehuss` `@RalfJung`
I'm not sure where we check for `rust-lld`'s existence on the targets where we use it by default, and if we just ignore it when missing or emit a warning (as I assume we don't emit an error), so I just checked for the existence of `gcc-ld`, where `cc` will look for the lld-wrapper binaries.
<sub>*Feel free to point out better ways to do this, it's the middle of the night here.*</sub>
Fixes#125246
Typical uses of ThinLTO don't have any use for this as a standalone
file, but distributed ThinLTO uses this to make the linker phase more
efficient. With clang you'd do something like `clang -flto=thin
-fthin-link-bitcode=foo.indexing.o -c foo.c` and then get both foo.o
(full of bitcode) and foo.indexing.o (just the summary or index part of
the bitcode). That's then usable by a two-stage linking process that's
more friendly to distributed build systems like bazel, which is why I'm
working on this area.
I talked some to @teresajohnson about naming in this area, as things
seem to be a little confused between various blog posts and build
systems. "bitcode index" and "bitcode summary" tend to be a little too
ambiguous, and she tends to use "thin link bitcode" and "minimized
bitcode" (which matches the descriptions in LLVM). Since the clang
option is thin-link-bitcode, I went with that to try and not add a new
spelling in the world.
Per @dtolnay, you can work around the lack of this by using `lld
--thinlto-index-only` to do the indexing on regular .o files of
bitcode, but that is a bit wasteful on actions when we already have all
the information in rustc and could just write out the matching minimized
bitcode. I didn't test that at all in our infrastructure, because by the
time I learned that I already had this patch largely written.
Translation of the lint message happens when the actual diagnostic is
created, not when the lint is buffered. Generating the message from
BuiltinLintDiag ensures that all required data to construct the message
is preserved in the LintBuffer, eventually allowing the messages to be
moved to fluent.
Remove the `msg` field from BufferedEarlyLint, it is either generated
from the data in the BuiltinLintDiag or stored inside
BuiltinLintDiag::Normal.
Relax restrictions on multiple sanitizers
Most combinations of LLVM sanitizers are legal-enough to enable simultaneously. This change will allow simultaneously enabling ASAN and shadow call stacks on supported platforms.
I used this python script to generate the mutually-exclusive sanitizer combinations:
```python
#!/usr/bin/python3
import subprocess
flags = [
["-fsanitize=address"],
["-fsanitize=leak"],
["-fsanitize=memory"],
["-fsanitize=thread"],
["-fsanitize=hwaddress"],
["-fsanitize=cfi", "-flto", "-fvisibility=hidden"],
["-fsanitize=memtag", "--target=aarch64-linux-android", "-march=armv8a+memtag"],
["-fsanitize=shadow-call-stack"],
["-fsanitize=kcfi", "-flto", "-fvisibility=hidden"],
["-fsanitize=kernel-address"],
["-fsanitize=safe-stack"],
["-fsanitize=dataflow"],
]
for i in range(len(flags)):
for j in range(i):
command = ["clang++"] + flags[i] + flags[j] + ["-o", "main.o", "-c", "main.cpp"]
completed = subprocess.run(command, stderr=subprocess.DEVNULL)
if completed.returncode != 0:
first = flags[i][0][11:].replace('-', '').upper()
second = flags[j][0][11:].replace('-', '').upper()
print(f"(SanitizerSet::{first}, SanitizerSet::{second}),")
```
Most combinations of LLVM sanitizers are legal-enough to enable
simultaneously. This change will allow simultaneously enabling ASAN and
shadow call stacks on supported platforms.
Change `SIGPIPE` ui from `#[unix_sigpipe = "..."]` to `-Zon-broken-pipe=...`
In the stabilization [attempt](https://github.com/rust-lang/rust/pull/120832) of `#[unix_sigpipe = "sig_dfl"]`, a concern was [raised ](https://github.com/rust-lang/rust/pull/120832#issuecomment-2007394609) related to using a language attribute for the feature: Long term, we want `fn lang_start()` to be definable by any crate, not just libstd. Having a special language attribute in that case becomes awkward.
So as a first step towards the next stabilization attempt, this PR changes the `#[unix_sigpipe = "..."]` attribute to a compiler flag `-Zon-broken-pipe=...` to remove that concern, since now the language is not "contaminated" by this feature.
Another point was [also raised](https://github.com/rust-lang/rust/pull/120832#issuecomment-1987023484), namely that the ui should not leak **how** it does things, but rather what the **end effect** is. The new flag uses the proposed naming. This is of course something that can be iterated on further before stabilization.
Tracking issue: https://github.com/rust-lang/rust/issues/97889
In the stabilization attempt of `#[unix_sigpipe = "sig_dfl"]`, a concern
was raised related to using a language attribute for the feature: Long
term, we want `fn lang_start()` to be definable by any crate, not just
libstd. Having a special language attribute in that case becomes
awkward.
So as a first step towards towards the next stabilization attempt, this
PR changes the `#[unix_sigpipe = "..."]` attribute to a compiler flag
`-Zon-broken-pipe=...` to remove that concern, since now the language
is not "contaminated" by this feature.
Another point was also raised, namely that the ui should not leak
**how** it does things, but rather what the **end effect** is. The new
flag uses the proposed naming. This is of course something that can be
iterated on further before stabilization.
Remove many `#[macro_use] extern crate foo` items
This requires the addition of more `use` items, which often make the code more verbose. But they also make the code easier to read, because `#[macro_use]` obscures where macros are defined.
r? `@fee1-dead`
`-Z debug-macros` is "stabilized" by enabling it by default and removing.
`-Z collapse-macro-debuginfo` is stabilized as `-C collapse-macro-debuginfo`.
It now supports all typical boolean values (`parse_opt_bool`) in addition to just yes/no.
Default value of `collapse_debuginfo` was changed from `false` to `external` (i.e. collapsed if external, not collapsed if local).
`#[collapse_debuginfo]` attribute without a value is no longer supported to avoid guessing the default.
Improve diagnostic for unknown `--print` request
This PR improves the diagnostic when encountering a unknown `--print` request.
It also moves the run-make test to a simple UI test.
Enable `CrateNum` query feeding via `TyCtxt`
Instead of having a magic function that violates some `TyCtxtFeed` invariants, add a `create_def` equivalent for `CrateNum`s.
Note that this still isn't tracked by the query system (unlike `create_def`), and that feeding most `CrateNum` queries for crates other than the local one will likely cause performance regressions.
These things should be attempted on their own separately, but this PR should stand on its own
Implement Modified Condition/Decision Coverage
This is an implementation based on llvm backend support (>= 18) by `@evodius96` and branch coverage support by `@Zalathar.`
### Major changes:
* Add -Zcoverage-options=mcdc as switch. Now coverage options accept either `no-branch`, `branch`, or `mcdc`. `mcdc` also enables `branch` because it is essential to work.
* Add coverage mapping for MCDCBranch and MCDCDecision. Note that MCDCParameter evolves from llvm 18 to llvm 19. The mapping in rust side mainly references to 19 and is casted to 18 types in llvm wrapper.
* Add wrapper for mcdc instrinc functions from llvm. And inject associated statements to mir.
* Add BcbMappingKind::Decision, I'm not sure is it proper but can't find a better way temporarily.
* Let coverage-dump support parsing MCDCBranch and MCDCDecision from llvm ir.
* Add simple tests to check whether mcdc works.
* Same as clang, currently rustc does not generate instrument for decision with more than 6 condtions or only 1 condition due to considerations of resource.
### Implementation Details
1. To get information about conditions and decisions, `MCDCState` in `BranchInfoBuilder` is used during hir lowering to mir. For expressions with logical op we call `Builder::visit_coverage_branch_operation` to record its sub conditions, generate condition ids for them and save their spans (to construct the span of whole decision). This process mainly references to the implementation in clang and is described in comments over `MCDCState::record_conditions`. Also true marks and false marks introduced by branch coverage are used to detect where the decision evaluation ends: the next id of the condition == 0.
2. Once the `MCDCState::decision_stack` popped all recorded conditions, we can ensure that the decision is checked over and push it into `decision_spans`. We do not manually insert decision span to avoid complexity from then_else_break in nested if scopes.
3. When constructing CoverageSpans, add condition info to BcbMappingKind::Branch and decision info to BcbMappingKind::Decision. If the branch mapping has non-zero condition id it will be transformed to MCDCBranch mapping and insert `CondBitmapUpdate` statements to its evaluated blocks. While decision bcb mapping will insert `TestVectorBitmapUpdate` in all its end blocks.
### Usage
```bash
echo "[build]\nprofiler=true" >> config.toml
./x build --stage 1
./x test tests/coverage/mcdc_if.rs
```
to build the compiler and run tests.
```shell
export PATH=path/to/llvm-build:$PATH
rustup toolchain link mcdc build/host/stage1
cargo +mcdc rustc --bin foo -- -Cinstrument-coverage -Zcoverage-options=mcdc
cd target/debug
LLVM_PROFILE_FILE="foo.profraw" ./foo
llvm-profdata merge -sparse foo.profraw -o foo.profdata
llvm-cov show ./foo -instr-profile=foo.profdata --show-mcdc
```
to check "foo" code.
### Problems to solve
For now decision mapping will insert statements to its all end blocks, which may be optimized by inserting a final block of the decision. To do this we must also trace the evaluated value at each end of the decision and join them separately.
This implementation is not heavily tested so there should be some unrevealed issues. We are going to check our rust products in the next. Please let me know if you had any suggestions or comments.
Introduce perma-unstable `wasm-c-abi` flag
Now that `wasm-bindgen` v0.2.88 supports the spec-compliant C ABI, the idea is to switch to that in a future version of Rust. In the meantime it would be good to let people test and play around with it.
This PR introduces a new perma-unstable `-Zwasm-c-abi` compiler flag, which switches to the new spec-compliant C ABI when targeting `wasm32-unknown-unknown`.
Alternatively, we could also stabilize this and then deprecate it when we switch. I will leave this to the Rust maintainers to decide.
This is a companion PR to #117918, but they could be merged independently.
MCP: https://github.com/rust-lang/compiler-team/issues/703
Tracking issue: https://github.com/rust-lang/rust/issues/122532
Stabilize checking of cfgs at compile-time: `--check-cfg` option
This PR stabilize the `--check-cfg` CLI option of `rustc` (and `rustdoc`) 🎉.
In particular this PR does two things:
1. it makes the `--check-cfg` option stable
2. and it moves the documentation to the stable books
FCP: https://github.com/rust-lang/rust/issues/82450#issuecomment-1965328542Resolves#82450
``@rustbot`` labels +S-blocked +F-check-cfg
r? ``@petrochenkov``
Currently it's a method on `EarlyDiagCtxt`, which is not the right place
for it at all -- `EarlyDiagCtxt` is used to issue diagnostics, but
shouldn't be doing any of the actual checking.
This commit moves it into a standalone function that takes an
`EarlyDiagCtxt` as an argument, which is more sensible. This does
require adding `EarlyDiagCtxt::early_struct_warn`, so a warning can be
returned and then modified with a note. (And that likely explains why
somebody put `initialize_checked_jobserver` into `EarlyDiagCtxt` in the
first place.)
Currently `SourceMap` is constructed slightly later than
`SessionGlobals`, and inserted. This commit changes things so they are
done at the same time.
Benefits:
- `SessionGlobals::source_map` changes from
`Lock<Option<Lrc<SourceMap>>>` to `Option<Lrc<SourceMap>>`. It's still
optional, but mutability isn't required because it's initialized at
construction.
- `set_source_map` is removed, simplifying `run_compiler`, which is
good because that's a critical function and it's nice to make it
simpler.
This requires moving things around a bit, so the necessary inputs are
available when `SessionGlobals` is created, in particular the `loader`
and `hash_kind`, which are no longer computed by `build_session`. These
inputs are captured by the new `SourceMapInputs` type, which is threaded
through various places.
Linker flavors next steps: linker features
This is my understanding of the first step towards `@petrochenkov's` vision for the future of linker flavors, described in https://github.com/rust-lang/rust/pull/119906#issuecomment-1895693162 and the discussion that followed.
To summarize: having `Cc` and `Lld` embedded in linker flavors creates tension about naming, and a combinatorial explosion of flavors for each new linker feature we'd want to use. Linker features are an extension mechanism that is complementary to principal flavors, with benefits described in #119906.
The most immediate use of this flag would be to turn self-contained linking on and off via features instead of flavors. For example, `-Clinker-features=+/-lld` would toggle using lld instead of selecting a precise flavor, and would be "generic" and work cross-platform (whereas linker flavors are currently more tied to targets). Under this scheme, MCP510 is expected to be `-Clink-self-contained=+linker -Zlinker-features=+lld -Zunstable-options` (though for the time being, the original flags using lld-cc flavors still work).
I purposefully didn't add or document CLI support for `+/-cc`, as it would be a noop right now. I only expect that we'd initially want to stabilize `+/-lld` to begin with.
r? `@petrochenkov`
You had requested that minimal churn would be done to the 230 target specs and this does none yet: the linker features are inferred from the flavor since they're currently isomorphic. We of course expect this to change sooner rather than later.
In the future, we can allow targets to define linker features independently from their flavor, and remove the cc and lld components from the flavors to use the features instead, this actually doesn't need to block stabilization, as we discussed.
(Best reviewed per commit)
Replace some `CrateStore` trait methods with hooks.
Just like with the `CrateStore` trait, this avoids the cyclic definition issues with `CStore` being
defined after TyCtxt, but needing to be used in TyCtxt.
