Support tail calls in mir via `TerminatorKind::TailCall`
This is one of the interesting bits in tail call implementation — MIR support.
This adds a new `TerminatorKind` which represents a tail call:
```rust
TailCall {
func: Operand<'tcx>,
args: Vec<Operand<'tcx>>,
fn_span: Span,
},
```
*Structurally* this is very similar to a normal `Call` but is missing a few fields:
- `destination` — tail calls don't write to destination, instead they pass caller's destination to the callee (such that eventual `return` will write to the caller of the function that used tail call)
- `target` — similarly to `destination` tail calls pass the caller's return address to the callee, so there is nothing to do
- `unwind` — I _think_ this is applicable too, although it's a bit confusing
- `call_source` — `become` forbids operators and is not created as a lowering of something else; tail calls always come from HIR (at least for now)
It might be helpful to read the interpreter implementation to understand what `TailCall` means exactly, although I've tried documenting it too.
-----
There are a few `FIXME`-questions still left, ideally we'd be able to answer them during review ':)
-----
r? `@oli-obk`
cc `@scottmcm` `@DrMeepster` `@JakobDegen`
Make jump threading state sparse
Continuation of https://github.com/rust-lang/rust/pull/127024
Both dataflow const-prop and jump threading involve cloning the state vector a lot. This PR replaces the data structure by a sparse vector, considering:
- that jump threading state is typically very sparse (at most 1 or 2 set entries);
- that dataflow const-prop is disabled by default;
- that place/value map is very eager, and prone to creating an overly large state.
The first commit is shared with the previous PR to avoid needless conflicts.
r? `@oli-obk`
Re-implement a type-size based limit
r? lcnr
This PR reintroduces the type length limit added in #37789, which was accidentally made practically useless by the caching changes to `Ty::walk` in #72412, which caused the `walk` function to no longer walk over identical elements.
Hitting this length limit is not fatal unless we are in codegen -- so it shouldn't affect passes like the mir inliner which creates potentially very large types (which we observed, for example, when the new trait solver compiles `itertools` in `--release` mode).
This also increases the type length limit from `1048576 == 2 ** 20` to `2 ** 24`, which covers all of the code that can be reached with craterbot-check. Individual crates can increase the length limit further if desired.
Perf regression is mild and I think we should accept it -- reinstating this limit is important for the new trait solver and to make sure we don't accidentally hit more type-size related regressions in the future.
Fixes#125460
Fix `FnMut::call_mut`/`Fn::call` shim for async closures that capture references
I adjusted async closures to be able to implement `Fn` and `FnMut` *even if* they capture references, as long as those references did not need to borrow data from the closure captures themselves. See #125259.
However, when I did this, I didn't actually relax an assertion in the `build_construct_coroutine_by_move_shim` shim code, which builds the `Fn`/`FnMut`/`FnOnce` implementations for async closures. Therefore, if we actually tried to *call* `FnMut`/`Fn` on async closures, it would ICE.
This PR adjusts this assertion to ensure that we only capture immutable references in closures if they implement `Fn`/`FnMut`. It also adds a bunch of tests and makes more of the async-closure tests into `build-pass` since we often care about these tests actually generating the right closure shims and stuff. I think it might be excessive to *always* use build-pass here, but 🤷 it's not that big of a deal.
Fixes#127019Fixes#127012
r? oli-obk
In 126578 we ended up with more binary size increases than expected.
This change attempts to avoid inlining large things into small things, to avoid that kind of increase, in cases when top-down inlining will still be able to do that inlining later.
Rollup of 7 pull requests
Successful merges:
- #126923 (test: dont optimize to invalid bitcasts)
- #127090 (Reduce merge conflicts from rustfmt's wrapping)
- #127105 (Only update `Eq` operands in GVN if it can update both sides)
- #127150 (Fix x86_64 code being produced for bare-metal LoongArch targets' `compiler_builtins`)
- #127181 (Introduce a `rustc_` attribute to dump all the `DefId` parents of a `DefId`)
- #127182 (Fix error in documentation for IpAddr::to_canonical and Ipv6Addr::to_canonical)
- #127191 (Ensure `out_of_scope_macro_calls` lint is registered)
r? `@ghost`
`@rustbot` modify labels: rollup
Automatically taint InferCtxt when errors are emitted
r? `@nnethercote`
Basically `InferCtxt::dcx` now returns a `DiagCtxt` that refers back to the `Cell<Option<ErrorGuaranteed>>` of the `InferCtxt` and thus when invoking `Diag::emit`, and the diagnostic is an error, we taint the `InferCtxt` directly.
That change on its own has no effect at all, because `InferCtxt` already tracks whether errors have been emitted by recording the global error count when it gets opened, and checking at the end whether the count changed. So I removed that error count check, which had a bit of fallout that I immediately fixed by invoking `InferCtxt::dcx` instead of `TyCtxt::dcx` in a bunch of places.
The remaining new errors are because an error was reported in another query, and never bubbled up. I think they are minor enough for this to be ok, and sometimes it actually improves diagnostics, by not silencing useful diagnostics anymore.
fixes#126485 (cc `@olafes)`
There are more improvements we can do (like tainting in hir ty lowering), but I would rather do that in follow up PRs, because it requires some refactorings.
coverage: Avoid getting extra unexpansion info when we don't need it
Several callers of `unexpand_into_body_span_with_visible_macro` would immediately discard the additional macro-related information, which is wasteful. We can avoid this by having them instead call a simpler method that just returns the span they care about.
This PR also moves the relevant functions out of `coverage::spans::from_mir` and into a new submodule `coverage::unexpand`, so that calling them from `coverage::mappings` is less awkward.
There should be no actual changes to coverage-instrumentation output, as demonstrated by the absence of test updates.
Avoid cloning jump threading state when possible
The current implementation of jump threading passes most of its time cloning its state. This PR attempts to avoid such clones by special-casing the last predecessor when recursing through a terminator.
This is not optimal, but a first step while I refactor the state data structure to be sparse.
The two other commits are drive-by.
Fixes https://github.com/rust-lang/rust/issues/116721
r? `@oli-obk`
These particular callers don't actually use the returned macro information, so
they can use a simpler span-unexpansion function that doesn't return it.
coverage: Make `#[coverage(..)]` apply recursively to nested functions
This PR makes the (currently-unstable) `#[coverage(off)]` and `#[coverage(on)]` attributes apply recursively to all nested functions/closures, instead of just the function they are directly attached to.
Those attributes can now also be applied to modules and to impl/impl-trait blocks, where they have no direct effect, but will be inherited by all enclosed functions/closures/methods that don't override the inherited value.
---
Fixes#126625.
Remove more `PtrToPtr` casts in GVN
This addresses two things I noticed in MIR:
1. `NonNull::<T>::eq` does `(a as *mut T) == (b as *mut T)`, but it could just compare the `*const T`s, so this removes `PtrToPtr` casts that are on both sides of a pointer comparison, so long as they're not fat-to-thin casts.
2. `NonNull::<T>::addr` does `transmute::<_, usize>(p as *const ())`, but so long as `T: Thin` that cast doesn't do anything, and thus we can directly transmute the `*const T` instead.
r? mir-opt
Save 2 pointers in `TerminatorKind` (96 → 80 bytes)
These things don't need to be `Vec`s; boxed slices are enough.
The frequent one here is call arguments, but MIR building knows the number of arguments from the THIR, so the collect is always getting the allocation right in the first place, and thus this shouldn't ever add the shrink-in-place overhead.
These things don't need to be `Vec`s; boxed slices are enough.
The frequent one here is call arguments, but MIR building knows the number of arguments from the THIR, so the collect is always getting the allocation right in the first place, and thus this shouldn't ever add the shrink-in-place overhead.
`PtrMetadata` doesn't care about `*const`/`*mut`/`&`/`&mut`, so GVN away those casts in its argument.
This includes updating MIR to allow calling PtrMetadata on references too, not just raw pointers. That means that `[T]::len` can be just `_0 = PtrMetadata(_1)`, for example.
# Conflicts:
# tests/mir-opt/pre-codegen/slice_index.slice_get_unchecked_mut_range.PreCodegen.after.panic-abort.mir
# tests/mir-opt/pre-codegen/slice_index.slice_get_unchecked_mut_range.PreCodegen.after.panic-unwind.mir
Account for things that optimize out in inlining costs
This updates the MIR inlining `CostChecker` to have both bonuses and penalties, rather than just penalties.
That lets us add bonuses for some things where we want to encourage inlining without risking wrapping into a gigantic cost. For example, `switchInt(const …)` we give an inlining bonus because codegen will actually eliminate the branch (and associated dead blocks) once it's monomorphized, so measuring both sides of the branch gives an unrealistically-high cost to it. Similarly, an `unreachable` terminator gets a small bonus, because whatever branch leads there doesn't actually exist post-codegen.
Clean up some comments near `use` declarations
#125443 will reformat all `use` declarations in the repository. There are a few edge cases involving comments on `use` declarations that require care. This PR cleans up some clumsy comment cases, taking us a step closer to #125443 being able to merge.
r? ``@lqd``
Stabilise `c_unwind`
Fix#74990Fix#115285 (that's also where FCP is happening)
Marking as draft PR for now due to `compiler_builtins` issues
r? `@Amanieu`
Most modules have such a blank line, but some don't. Inserting the blank
line makes it clearer that the `//!` comments are describing the entire
module, rather than the `use` declaration(s) that immediately follows.
Apparently MIR borrowck cares about at least one of these for checking variance.
In runtime MIR, though, there's no need for them as `PtrToPtr` does the same thing.
(Banning them simplifies passes like GVN that no longer need to handle multiple cast possibilities.)
Replace all `&DiagCtxt` with a `DiagCtxtHandle<'_>` wrapper type
r? `@davidtwco`
This paves the way for tracking more state (e.g. error tainting) in the diagnostic context handle
Basically I will add a field to the `DiagCtxtHandle` that refers back to the `InferCtxt`'s (and others) `Option<ErrorHandled>`, allowing us to immediately taint these contexts when emitting an error and not needing manual tainting anymore (which is easy to forget and we don't do in general anyway)
coverage: Add debugging flag `-Zcoverage-options=no-mir-spans`
When set, this flag skips the code that normally extracts coverage spans from MIR statements and terminators. That sometimes makes it easier to debug branch coverage and MC/DC coverage instrumentation, because the coverage output is less noisy.
For internal debugging only. If future code changes would make it hard to keep supporting this flag, it should be removed at that time.
`@rustbot` label +A-code-coverage
Rename `InstanceDef` -> `InstanceKind`
Renames `InstanceDef` to `InstanceKind`. The `Def` here is confusing, and makes it hard to distinguish `Instance` and `InstanceDef`. `InstanceKind` makes this more obvious, since it's really just describing what *kind* of instance we have.
Not sure if this is large enough to warrant a types team MCP -- it's only 53 files. I don't personally think it does, but happy to write one if anyone disagrees. cc ``@rust-lang/types``
r? types
When set, this flag skips the code that normally extracts coverage spans from
MIR statements and terminators. That sometimes makes it easier to debug branch
coverage and MC/DC coverage, because the coverage output is less noisy.
For internal debugging only. If other code changes would make it hard to keep
supporting this flag, remove it.
coverage: Several small improvements to graph code
This PR combines a few small improvements to coverage graph handling code:
- Remove some low-value implementation tests that were getting in the way of other changes.
- Clean up `pub` visibility.
- Flatten some code using let-else.
- Prefer `.copied()` over `.cloned()`.
`@rustbot` label +A-code-coverage
These tests might have originally been useful as an implementation aid, but now
they don't provide enough value to justify the burden of updating them as the
underlying code changes.
The code they test is still exercised by the main end-to-end coverage tests.
smir: merge identical Constant and ConstOperand types
The first commit renames the const operand visitor functions on regular MIR to match the type name, that was forgotten in the original rename.
The second commit changes stable MIR, fixing https://github.com/rust-lang/project-stable-mir/issues/71. Previously there were two different smir types for the MIR type `ConstOperand`, one used in `Operand` and one in `VarDebugInfoContents`.
Maybe we should have done this with https://github.com/rust-lang/rust/pull/125967, so there's only a single breaking change... but I saw that PR too late.
Fixes https://github.com/rust-lang/project-stable-mir/issues/71
Use `Variance` glob imported variants everywhere
Fully commit to using the globbed variance. Could be convinced the other way, and change this PR to not use the globbed variants anywhere, but I'd rather we do one or the other.
r? lcnr
coverage: Replace the old span refiner with a single function
As more and more of the span refiner's functionality has been pulled out into separate early passes, it has finally reached the point where we can remove the rest of the old `SpansRefiner` code, and replace it with a single modestly-sized function.
~~There should be no change to the resulting coverage mappings, as demonstrated by the lack of changes to test output.~~
There is *almost* no change to the resulting coverage mappings. There are some minor changes to `loop` that on inspection appear to be neutral in terms of accuracy, with the old behaviour being a slightly-horrifying implementation detail of the old code, so I think they're acceptable.
Previous work in this direction includes:
- #125921
- #121019
- #119208
As more and more of the span refiner's functionality has been pulled out into
separate early passes, it has finally reached the point where we can remove the
rest of the old `SpansRefiner` code, and replace it with a single
modestly-sized function.
We already do this for a number of crates, e.g. `rustc_middle`,
`rustc_span`, `rustc_metadata`, `rustc_span`, `rustc_errors`.
For the ones we don't, in many cases the attributes are a mess.
- There is no consistency about order of attribute kinds (e.g.
`allow`/`deny`/`feature`).
- Within attribute kind groups (e.g. the `feature` attributes),
sometimes the order is alphabetical, and sometimes there is no
particular order.
- Sometimes the attributes of a particular kind aren't even grouped
all together, e.g. there might be a `feature`, then an `allow`, then
another `feature`.
This commit extends the existing sorting to all compiler crates,
increasing consistency. If any new attribute line is added there is now
only one place it can go -- no need for arbitrary decisions.
Exceptions:
- `rustc_log`, `rustc_next_trait_solver` and `rustc_type_ir_macros`,
because they have no crate attributes.
- `rustc_codegen_gcc`, because it's quasi-external to rustc (e.g. it's
ignored in `rustfmt.toml`).
Add `SingleUseConsts` mir-opt pass
The goal here is to make a pass that can be run in debug builds to simplify the common case of constants that are used just once -- that doesn't need SSA handling and avoids any potential downside of multi-use constants. In particular, to simplify the `if T::IS_ZST` pattern that's common in the standard library.
By also handling the case of constants that are *never* actually used this fully replaces the `ConstDebugInfo` pass, since it has all the information needed to do that naturally from the traversal it needs to do anyway.
This is roughly a wash on instructions on its own (a couple regressions, a few improvements https://github.com/rust-lang/rust/pull/125910#issuecomment-2144963361), with a bunch of size improvements. So I'd like to land it as its own PR, then do follow-ups to take more advantage of it (in the inliner, cg_ssa, etc).
r? `@saethlin`
Enable GVN for `AggregateKind::RawPtr`
Looks like I was worried for nothing; this seems like it's much easier than I was originally thinking it would be.
r? `@cjgillot`
This should be useful for `x[..4]`-like things, should those start inlining enough to expose the lengths.
Revert "Use the HIR instead of mir_keys for determining whether something will have a MIR body."
This reverts commit e5cba17b84.
turns out SMIR still needs it (https://github.com/model-checking/kani/issues/3218). I'll create a full plan and MCP for what I intended this to be a part of. Maybe my plan is nonsense anyway.
Remove the `ty` field from type system `Const`s
Fixes#125556Fixes#122908
Part of the work on `adt_const_params`/`generic_const_param_types`/`min_generic_const_exprs`/generally making the compiler nicer. cc rust-lang/project-const-generics#44
Please review commit-by-commit otherwise I wasted a lot of time not just squashing this into a giant mess (and also it'll be SO much nicer because theres a lot of fluff changes mixed in with other more careful changes if looking via File Changes
---
Why do this?
- The `ty` field keeps causing ICEs and weird behaviour due to it either being treated as "part of the const" or it being forgotten about leading to ICEs.
- As we move forward with `adt_const_params` and a potential `min_generic_const_exprs` it's going to become more complex to actually lower the correct `Ty<'tcx>`
- It muddles the idea behind how we check `Const` arguments have the correct type. By having the `ty` field it may seem like we ought to be relating it when we relate two types, or that its generally important information about the `Const`.
- Brings the compiler more in line with `a-mir-formality` as that also tracks the type of type system `Const`s via `ConstArgHasType` bounds in the env instead of on the `Const` itself.
- A lot of stuff is a lot nicer when you dont have to pass around the type of a const lol. Everywhere we construct `Const` is now significantly nicer 😅
See #125671's description for some more information about the `ty` field
---
General summary of changes in this PR:
- Add `Ty` to `ConstKind::Value` as otherwise there is no way to implement `ConstArgHasType` to ensure that const arguments are correctly typed for the parameter when we stop creating anon consts for all const args. It's also just incredibly difficult/annoying to thread the correct `Ty` around to a bunch of ctfe functions otherwise.
- Fully implement `ConstArgHasType` in both the old and new solver. Since it now has no reliance on the `ty` field it serves its originally intended purpose of being able to act as a double check that trait vs impls have correctly typed const parameters. It also will now be able to be responsible for checking types of const arguments to parameters under `min_generic_const_exprs`.
- Add `Ty` to `mir::Const::Ty`. I dont have a great understanding of why mir constants are setup like this to be honest. Regardless they need to be able to determine the type of the const and the easiest way to make this happen was to simply store the `Ty` along side the `ty::Const`. Maybe we can do better here in the future but I'd have to spend way more time looking at everywhere we use `mir::Const`.
- rustdoc has its own `Const` which also has a `ty` field. It was relatively easy to remove this.
---
r? `@lcnr` `@compiler-errors`
Rollup of 9 pull requests
Successful merges:
- #124840 (resolve: mark it undetermined if single import is not has any bindings)
- #125622 (Winnow private method candidates instead of assuming any candidate of the right name will apply)
- #125648 (Remove unused(?) `~/rustsrc` folder from docker script)
- #125672 (Add more ABI test cases to miri (RFC 3391))
- #125800 (Fix `mut` static task queue in SGX target)
- #125871 (Orphanck[old solver]: Consider opaque types to never cover type parameters)
- #125893 (Handle all GVN binops in a single place.)
- #126008 (Port `tests/run-make-fulldeps/issue-19371` to ui-fulldeps)
- #126032 (Update description of the `IsTerminal` example)
r? `@ghost`
`@rustbot` modify labels: rollup
Handle all GVN binops in a single place.
<!--
If this PR is related to an unstable feature or an otherwise tracked effort,
please link to the relevant tracking issue here. If you don't know of a related
tracking issue or there are none, feel free to ignore this.
This PR will get automatically assigned to a reviewer. In case you would like
a specific user to review your work, you can assign it to them by using
r? <reviewer name>
-->
Addresses https://github.com/rust-lang/rust/pull/125359/files#r1608185319
r? ``@oli-obk``
Show files produced by `--emit foo` in json artifact notifications
Right now it is possible to ask `rustc` to save some intermediate representation into one or more files with `--emit=foo`, but figuring out what exactly was produced is difficult. This pull request adds information about `llvm_ir` and `asm` intermediate files into notifications produced by `--json=artifacts`.
Related discussion: https://internals.rust-lang.org/t/easier-access-to-files-generated-by-emit-foo/20477
Motivation - `cargo-show-asm` parses those intermediate files and presents them in a user friendly way, but right now I have to apply some dirty hacks. Hacks make behavior confusing: https://github.com/hintron/computer-enhance/issues/35
This pull request introduces a new behavior: now `rustc` will emit a new artifact notification for every artifact type user asked to `--emit`, for example for `--emit asm` those will include all the `.s` files.
Most users won't notice this behavior, to be affected by it all of the following must hold:
- user must use `rustc` binary directly (when `cargo` invokes `rustc` - it consumes artifact notifications and doesn't emit anything)
- user must specify both `--emit xxx` and `--json artifacts`
- user must refuse to handle unknown artifact types
- user must disable incremental compilation (or deal with it better than cargo does, or use a workaround like `save-temps`) in order not to hit #88829 / #89149
The `mir!` macro has multiple parts:
- An optional return type annotation.
- A sequence of zero or more local declarations.
- A mandatory starting anonymous basic block, which is brace-delimited.
- A sequence of zero of more additional named basic blocks.
Some `mir!` invocations use braces with a "block" style, like so:
```
mir! {
let _unit: ();
{
let non_copy = S(42);
let ptr = std::ptr::addr_of_mut!(non_copy);
// Inside `callee`, the first argument and `*ptr` are basically
// aliasing places!
Call(_unit = callee(Move(*ptr), ptr), ReturnTo(after_call), UnwindContinue())
}
after_call = {
Return()
}
}
```
Some invocations use parens with a "block" style, like so:
```
mir!(
let x: [i32; 2];
let one: i32;
{
x = [42, 43];
one = 1;
x = [one, 2];
RET = Move(x);
Return()
}
)
```
And some invocations uses parens with a "tighter" style, like so:
```
mir!({
SetDiscriminant(*b, 0);
Return()
})
```
This last style is generally used for cases where just the mandatory
starting basic block is present. Its braces are placed next to the
parens.
This commit changes all `mir!` invocations to use braces with a "block"
style. Why?
- Consistency is good.
- The contents of the invocation is a block of code, so it's odd to use
parens. They are more normally used for function-like macros.
- Most importantly, the next commit will enable rustfmt for
`tests/mir-opt/`. rustfmt is more aggressive about formatting macros
that use parens than macros that use braces. Without this commit's
changes, rustfmt would break a couple of `mir!` macro invocations that
use braces within `tests/mir-opt` by inserting an extraneous comma.
E.g.:
```
mir!(type RET = (i32, bool);, { // extraneous comma after ';'
RET.0 = 1;
RET.1 = true;
Return()
})
```
Switching those `mir!` invocations to use braces avoids that problem,
resulting in this, which is nicer to read as well as being valid
syntax:
```
mir! {
type RET = (i32, bool);
{
RET.0 = 1;
RET.1 = true;
Return()
}
}
```
Check index `value <= 0xFFFF_FF00`
<!--
If this PR is related to an unstable feature or an otherwise tracked effort,
please link to the relevant tracking issue here. If you don't know of a related
tracking issue or there are none, feel free to ignore this.
This PR will get automatically assigned to a reviewer. In case you would like
a specific user to review your work, you can assign it to them by using
r? <reviewer name>
-->
fixes#121126
check `idx <= FieldIdx::MAX_AS_U32` before calling `FieldIdx::from_u32` to avoid panic.
Implement `needs_async_drop` in rustc and optimize async drop glue
This PR expands on #121801 and implements `Ty::needs_async_drop` which works almost exactly the same as `Ty::needs_drop`, which is needed for #123948.
Also made compiler's async drop code to look more like compiler's regular drop code, which enabled me to write an optimization where types which do not use `AsyncDrop` can simply forward async drop glue to `drop_in_place`. This made size of the async block from the [async_drop test](67980dd6fb/tests/ui/async-await/async-drop.rs) to decrease by 12%.
Enable DestinationPropagation by default.
~~Based on https://github.com/rust-lang/rust/pull/115291.~~
This PR proposes to enable the destination propagation pass by default.
This pass is meant to reduce the amount of copies present in MIR.
At the same time, this PR removes the `RenameReturnPlace` pass, as it is currently unsound.
`DestinationPropagation` is not limited to `_0`, but does not handle borrowed locals.
coverage: Rename MC/DC `conditions_num` to `num_conditions`
Updated version of #124571, without the other changes that were split out into #125108 and #125700.
This value represents a quantity of conditions, not an ID, so the new spelling is more appropriate.
Some of the code touched by this PR could perhaps use some other changes, but I would prefer to keep this PR as a simple renaming and avoid scope creep.
`@rustbot` label +A-code-coverage
rustfmt fixes
The `rmake.rs` entries in `rustfmt.toml` are causing major problems for `x fmt`. This PR removes them and does some minor related cleanups.
r? ``@GuillaumeGomez``
It's reasonable to want to, but in the current implementation this
causes multiple problems.
- All the `rmake.rs` files are formatted every time even when they
haven't changed. This is because they get whitelisted unconditionally
in the `OverrideBuilder`, before the changed files get added.
- The way `OverrideBuilder` works, if any files gets whitelisted then no
unmentioned files will get traversed. This is surprising, and means
that the `rmake.rs` entries broke the use of explicit paths to `x
fmt`, and also broke `GITHUB_ACTIONS=true git check --fmt`.
The commit removes the `rmake.rs` entries, fixes the formatting of a
couple of files that were misformatted (not previously caught due to the
`GITHUB_ACTIONS` breakage), and bans `!`-prefixed entries in
`rustfmt.toml` because they cause all these problems.
interpret: get rid of 'mir lifetime
I realized our MIR bodies are actually at lifetime `'tcx`, so we don't need to carry around this other lifetime everywhere.
r? `@oli-obk`
[perf] Delay the construction of early lint diag structs
Attacks some of the perf regressions from https://github.com/rust-lang/rust/pull/124417#issuecomment-2123700666.
See individual commits for details. The first three commits are not strictly necessary.
However, the 2nd one (06bc4fc671, *Remove `LintDiagnostic::msg`*) makes the main change way nicer to implement.
It's also pretty sweet on its own if I may say so myself.
* instead simply set the primary message inside the lint decorator functions
* it used to be this way before [#]101986 which introduced `msg` to prevent
good path delayed bugs (which no longer exist) from firing under certain
circumstances when lints were suppressed / silenced
* this is no longer necessary for various reasons I presume
* it shaves off complexity and makes further changes easier to implement
coverage: Memoize and simplify counter expressions
When creating coverage counter expressions as part of coverage instrumentation, we often end up creating obviously-redundant expressions like `c1 + (c0 - c1)`, which is equivalent to just `c0`.
To avoid doing so, this PR checks when we would create an expression matching one of 5 patterns, and uses the simplified form instead:
- `(a - b) + b` → `a`.
- `(a + b) - b` → `a`.
- `(a + b) - a` → `b`.
- `a + (b - a)` → `b`.
- `a - (a - b)` → `b`.
Of all the different ways to combine 3 operands and 2 operators, these are the patterns that allow simplification.
(Some of those patterns currently don't occur in practice, but are included anyway for completeness, to avoid having to add them later as branch coverage and MC/DC coverage support expands.)
---
This PR also adds memoization for newly-created (or newly-simplified) counter expressions, to avoid creating duplicates.
This currently makes no difference to the final mappings, but is expected to be useful for MC/DC coverage of match expressions, as proposed by https://github.com/rust-lang/rust/pull/124278#issuecomment-2106754753.
This code for recalculating `mcdc_bitmap_bytes` doesn't provide any benefit,
because its result won't have changed from the value in `FunctionCoverageInfo`
that was computed during the MIR instrumentation pass.
Some of these cases currently don't occur in practice, but are included for
completeness, and to avoid having to add them later as branch coverage and
MC/DC coverage start building more complex expressions.
Split out `ty::AliasTerm` from `ty::AliasTy`
Splitting out `AliasTerm` (for use in project and normalizes goals) and `AliasTy` (for use in `ty::Alias`)
r? lcnr
coverage: Further simplify extraction of mapping info from MIR
This is another round of rearrangement and simplification that builds on top of the changes made to mapping-extraction by #124603.
The overall theme is to take the computation of `bcb_has_mappings` and `test_vector_bitmap_bytes` out of the main body of `generate_coverage_spans`, which then lets us perform a few other small changes that had previously been held up by the need to work around those computations.
The code in `extract_mcdc_mappings` that allocates these bytes already knows
how many are needed in total, so there's no need to immediately recompute that
value in the calling function.
Now that branch and MC/DC mappings have been split out into separate types and
vectors, this enum is no longer needed, since it only represents ordinary
"code" regions.
(We can revisit this decision if we ever add support for other region kinds,
such as skipped regions or expansion regions. But at that point, we might just
add new structs/vectors for those kinds as well.)
Account for immutably borrowed locals in MIR copy-prop and GVN
For the most part, we consider that immutably borrowed `Freeze` locals still fulfill SSA conditions. As the borrow is immutable, any use of the local will have the value given by the single assignment, and there can be no surprise.
This allows copy-prop to merge a non-borrowed local with a borrowed local. We chose to keep copy-classes heads unborrowed, as those may be easier to optimize in later passes.
This also allows to GVN the value behind an immutable borrow. If a SSA local is borrowed, dereferencing that borrow is equivalent to copying the local's value: re-executing the assignment between the borrow and the dereference would be UB.
r? `@ghost` for perf
coverage: Clean up creation of MC/DC condition bitmaps
This PR improves the code for creating and initializing [MC/DC](https://en.wikipedia.org/wiki/Modified_condition/decision_coverage) condition bitmap variables, as introduced by #123409 and modified by #124255.
- The condition bitmap variables are now created eagerly at the start of per-function codegen, via a new `init_coverage` method in `CoverageInfoBuilderMethods`. This avoids having to retroactively create the bitmaps while doing codegen for an individual coverage statement.
- As a result, we can now create and initialize those bitmaps using existing safe APIs, instead of having to perform our own unsafe call to `llvm::LLVMBuildAlloca`.
- This PR also tweaks the way we count the number of condition bitmaps needed, by tracking the total number of bitmaps needed (max depth + 1), instead of only tracking the maximum depth. This reduces the potential for subtle off-by-one confusion.
Use `tcx.types.unit` instead of `Ty::new_unit(tcx)`
I don't think there is any need for the function, given that we can just access the `.types`, similarly to all other primitives?
because we are already marking unions `NoPropagation` in
`CanConstProp::check()`. That is enough to prevent any attempts
at const propagating unions and this second check is not needed.
Also improve a comment in `CanConstProp::check()`
Mark unions non-const-propagatable in `KnownPanicsLint` without calling layout
Fixes#123710
The ICE occurs during the layout calculation of the union `InvalidTag` in #123710 because the following assert fails:5fe8b697e7/compiler/rustc_abi/src/layout.rs (L289-L292)
The layout calculation is invoked by `KnownPanicsLint` when it is trying to figure out which locals it can const prop. Since `KnownPanicsLint` is never actually going to const props unions thanks to PR https://github.com/rust-lang/rust/pull/121628 there's no point calling layout to check if it can. So in this fix I skip the call to layout and just mark the local non-const propagatable if it is a union.
MCDC coverage: support nested decision coverage
#123409 provided the initial MCDC coverage implementation.
As referenced in #124144, it does not currently support "nested" decisions, like the following example :
```rust
fn nested_if_in_condition(a: bool, b: bool, c: bool) {
if a && if b || c { true } else { false } {
say("yes");
} else {
say("no");
}
}
```
Note that there is an if-expression (`if b || c ...`) embedded inside a boolean expression in the decision of an outer if-expression.
This PR proposes a workaround for this cases, by introducing a Decision context stack, and by handing several `temporary condition bitmaps` instead of just one.
When instrumenting boolean expressions, if the current node is a leaf condition (i.e. not a `||`/`&&` logical operator nor a `!` not operator), we insert a new decision context, such that if there are more boolean expressions inside the condition, they are handled as separate expressions.
On the codegen LLVM side, we allocate as many `temp_cond_bitmap`s as necessary to handle the maximum encountered decision depth.
Add decision_depth field to TVBitmapUpdate/CondBitmapUpdate statements
Add decision_depth field to BcbMappingKinds MCDCBranch and MCDCDecision
Add decision_depth field to MCDCBranchSpan and MCDCDecisionSpan
Do `check_coroutine_obligations` once per typeck root
We only need to do `check_coroutine_obligations` once per typeck root, especially since the new solver can't really (easily) associate which obligations correspond to which coroutines.
This requires us to move the checks for sized coroutine fields into `mir_coroutine_witnesses`, but that's fine imo.
r? lcnr
deref patterns: lower deref patterns to MIR
This lowers deref patterns to MIR. This is a bit tricky because this is the first kind of pattern that requires storing a value in a temporary. Thanks to https://github.com/rust-lang/rust/pull/123324 false edges are no longer a problem.
The thing I'm not confident about is the handling of fake borrows. This PR ignores any fake borrows inside a deref pattern. We are guaranteed to at least fake borrow the place of the first pointer value, which could be enough, but I'm not certain.
weak lang items are not allowed to be #[track_caller]
For instance the panic handler will be called via this import
```rust
extern "Rust" {
#[lang = "panic_impl"]
fn panic_impl(pi: &PanicInfo<'_>) -> !;
}
```
A `#[track_caller]` would add an extra argument and thus make this the wrong signature.
The 2nd commit is a consistency rename; based on the docs [here](https://doc.rust-lang.org/unstable-book/language-features/lang-items.html) and [here](https://rustc-dev-guide.rust-lang.org/lang-items.html) I figured "lang item" is more widely used. (In the compiler output, "lang item" and "language item" seem to be pretty even.)
Add simple async drop glue generation
This is a prototype of the async drop glue generation for some simple types. Async drop glue is intended to behave very similar to the regular drop glue except for being asynchronous. Currently it does not execute synchronous drops but only calls user implementations of `AsyncDrop::async_drop` associative function and awaits the returned future. It is not complete as it only recurses into arrays, slices, tuples, and structs and does not have same sensible restrictions as the old `Drop` trait implementation like having the same bounds as the type definition, while code assumes their existence (requires a future work).
This current design uses a workaround as it does not create any custom async destructor state machine types for ADTs, but instead uses types defined in the std library called future combinators (deferred_async_drop, chain, ready_unit).
Also I recommend reading my [explainer](https://zetanumbers.github.io/book/async-drop-design.html).
This is a part of the [MCP: Low level components for async drop](https://github.com/rust-lang/compiler-team/issues/727) work.
Feature completeness:
- [x] `AsyncDrop` trait
- [ ] `async_drop_in_place_raw`/async drop glue generation support for
- [x] Trivially destructible types (integers, bools, floats, string slices, pointers, references, etc.)
- [x] Arrays and slices (array pointer is unsized into slice pointer)
- [x] ADTs (enums, structs, unions)
- [x] tuple-like types (tuples, closures)
- [ ] Dynamic types (`dyn Trait`, see explainer's [proposed design](https://github.com/zetanumbers/posts/blob/main/async-drop-design.md#async-drop-glue-for-dyn-trait))
- [ ] coroutines (https://github.com/rust-lang/rust/pull/123948)
- [x] Async drop glue includes sync drop glue code
- [x] Cleanup branch generation for `async_drop_in_place_raw`
- [ ] Union rejects non-trivially async destructible fields
- [ ] `AsyncDrop` implementation requires same bounds as type definition
- [ ] Skip trivially destructible fields (optimization)
- [ ] New [`TyKind::AdtAsyncDestructor`](https://github.com/zetanumbers/posts/blob/main/async-drop-design.md#adt-async-destructor-types) and get rid of combinators
- [ ] [Synchronously undroppable types](https://github.com/zetanumbers/posts/blob/main/async-drop-design.md#exclusively-async-drop)
- [ ] Automatic async drop at the end of the scope in async context
This clears the way for larger changes to how branches are handled by the
coverage instrumentor, in order to support branch coverage for more language
constructs.
