Added "copy" to Debug fmt for copy operands
In MIR's debug mode (--emit mir) the printing for Operands is slightly inconsistent.
The RValues - values on the right side of an Assign - are usually printed with their Operand when they are Places.
Example:
_2 = move _3
But for arguments, the operand is omitted.
_2 = _1
I propose a change be made, to display the place with the operand.
_2 = copy _1
Move and copy have different semantics, meaning this difference is important and helpful to the user. It also adds consistency to the pretty printing.
-- EDIT --
Consider this example Rust program and its MIR output with the **updated pretty printer.**
This was generated with the arguments --emit mir --crate-type lib -Zmir-opt-level=0 (Otherwise, it's optimised away since it's a junk program).
```rust
fn main(foo: i32) {
let v = 10;
if v == 20 {
foo;
}
else {
v;
}
}
```
```MIR
// WARNING: This output format is intended for human consumers only
// and is subject to change without notice. Knock yourself out.
fn main(_1: i32) -> () {
debug foo => _1;
let mut _0: ();
let _2: i32;
let mut _3: bool;
let mut _4: i32;
let _5: i32;
let _6: i32;
scope 1 {
debug v => _2;
}
bb0: {
StorageLive(_2);
_2 = const 10_i32;
StorageLive(_3);
StorageLive(_4);
_4 = copy _2;
_3 = Eq(move _4, const 20_i32);
switchInt(move _3) -> [0: bb2, otherwise: bb1];
}
bb1: {
StorageDead(_4);
StorageLive(_5);
_5 = copy _1;
StorageDead(_5);
_0 = const ();
goto -> bb3;
}
bb2: {
StorageDead(_4);
StorageLive(_6);
_6 = copy _2;
StorageDead(_6);
_0 = const ();
goto -> bb3;
}
bb3: {
StorageDead(_3);
StorageDead(_2);
return;
}
}
```
In this example program, we can see that when we move a place, it is preceded by "move". e.g. ``` _3 = Eq(move _4, const 20_i32);```. However, when we copy a place such as ```_5 = _1;```, it is not preceded by the operand in the original printout. I propose to change the print to include the copy ```_5 = copy _1``` as in this example.
Regarding the arguments part. When I originally submitted this PR, I was under the impression this only affected the print for arguments to a function, but actually, it affects anything that uses a copy. This is preferable anyway with regard to consistency. The PR is about making ```copy``` explicit.
Apply "polymorphization at home" to RawVec
The idea here is to move all the logic in RawVec into functions with explicit size and alignment parameters. This should eliminate all the fussing about how tweaking RawVec code produces large swings in compile times.
This uncovered https://github.com/rust-lang/rust-clippy/issues/12979, so I've modified the relevant test in a way that tries to preserve the spirit of the test without tripping the ICE.
Fix `ElaborateBoxDerefs` on debug varinfo
Slightly simplifies the `ElaborateBoxDerefs` pass to fix cases where it was applying the wrong projections to debug var infos containing places that deref boxes.
From what I can tell[^1], we don't actually have any tests (or code anywhere, really) that exercise `debug x => *(...: Box<T>)`, and it's very difficult to trigger this in surface Rust, so I wrote a custom MIR test.
What happens is that the pass was turning `*(SOME_PLACE: Box<T>)` into `*(*((((SOME_PLACE).0: Unique<T>).0: NonNull<T>).0: *const T))` in debug var infos. In particular, notice the *double deref*, which was wrong.
This is the root cause of #128554, so this PR fixes#128554 as well. The reason that async closures was affected is because of the way that we compute the [`ByMove` body](https://github.com/rust-lang/rust/blob/master/compiler/rustc_mir_transform/src/coroutine/by_move_body.rs), which resulted in `*(...: Box<T>)` in debug var info. But this really has nothing to do with async closures.
[^1]: Validated by literally replacing the `if elem == PlaceElem::Deref && base_ty.is_box() { ... }` innards with a `panic!()`, which compiled all of stage2 without panicking.
match lowering: Hide `Candidate` from outside the lowering algorithm
The internals of `Candidate` are tricky and a source of confusion. This PR makes it so we don't expose `Candidate`s outside the lowering algorithm. Now:
- false edges are handled in `lower_match_tree`;
- `lower_match_tree` takes a list of patterns as input;
- `lower_match_tree` returns a flat datastructure that contains only the necessary information.
r? ```@matthewjasper```
Disable jump threading of float equality
Jump threading stores values as `u128` (`ScalarInt`) and does its comparisons for equality as integer comparisons.
This works great for integers. Sadly, not everything is an integer.
Floats famously have wonky equality semantcs, with `NaN!=NaN` and `0.0 == -0.0`. This does not match our beautiful integer bitpattern equality and therefore causes things to go horribly wrong.
While jump threading could be extended to support floats by remembering that they're floats in the value state and handling them properly, it's signficantly easier to just disable it for now.
fixes#128243
Jump threading stores values as `u128` (`ScalarInt`) and does its
comparisons for equality as integer comparisons.
This works great for integers. Sadly, not everything is an integer.
Floats famously have wonky equality semantcs, with `NaN!=NaN` and
`0.0 == -0.0`. This does not match our beautiful integer bitpattern
equality and therefore causes things to go horribly wrong.
While jump threading could be extended to support floats by remembering
that they're floats in the value state and handling them properly,
it's signficantly easier to just disable it for now.
Forbid borrows and unsized types from being used as the type of a const generic under `adt_const_params`
Fixes#112219Fixes#112124Fixes#112125
### Motivation
Currently the `adt_const_params` feature allows writing `Foo<const N: [u8]>` this is entirely useless as it is not possible to write an expression which evaluates to a type that is not `Sized`. In order to actually use unsized types in const generics they are typically written as `const N: &[u8]` which *is* possible to provide a value of.
Unfortunately allowing the types of const parameters to contain references is non trivial (#120961) as it introduces a number of difficult questions about how equality of references in the type system should behave. References in the types of const generics is largely only useful for using unsized types in const generics.
This PR introduces a new feature gate `unsized_const_parameters` and moves support for `const N: [u8]` and `const N: &...` from `adt_const_params` into it. The goal here hopefully is to experiment with allowing `const N: [u8]` to work without references and then eventually completely forbid references in const generics.
Splitting this out into a new feature gate means that stabilization of `adt_const_params` does not have to resolve#120961 which is the only remaining "big" blocker for the feature. Remaining issues after this are a few ICEs and naming bikeshed for `ConstParamTy`.
### Implementation
The implementation is slightly subtle here as we would like to ensure that a stabilization of `adt_const_params` is forwards compatible with any outcome of `unsized_const_parameters`. This is inherently tricky as we do not support unstable trait implementations and we determine whether a type is valid as the type of a const parameter via a trait bound.
There are a few constraints here:
- We would like to *allow for the possibility* of adding a `Sized` supertrait to `ConstParamTy` in the event that we wind up opting to not support unsized types and instead requiring people to write the 'sized version', e.g. `const N: [u8; M]` instead of `const N: [u8]`.
- Crates should be able to enable `unsized_const_parameters` and write trait implementations of `ConstParamTy` for `!Sized` types without downstream crates that only enable `adt_const_params` being able to observe this (required for std to be able to `impl<T> ConstParamTy for [T]`
Ultimately the way this is accomplished is via having two traits (sad), `ConstParamTy` and `UnsizedConstParamTy`. Depending on whether `unsized_const_parameters` is enabled or not we change which trait is used to check whether a type is allowed to be a const parameter.
Long term (when stabilizing `UnsizedConstParamTy`) it should be possible to completely merge these traits (and derive macros), only having a single `trait ConstParamTy` and `macro ConstParamTy`.
Under `adt_const_params` it is now illegal to directly refer to `ConstParamTy` it is only used as an internal impl detail by `derive(ConstParamTy)` and checking const parameters are well formed. This is necessary in order to ensure forwards compatibility with all possible future directions for `feature(unsized_const_parameters)`.
Generally the intuition here should be that `ConstParamTy` is the stable trait that everything uses, and `UnsizedConstParamTy` is that plus unstable implementations (well, I suppose `ConstParamTy` isn't stable yet :P).
Ignore allocation bytes in one more mir-opt test
Following on PR #126502, add `rustc -Zdump-mir-exclude-alloc-bytes` to tests/mir-opt/dataflow-const-prop/aggregate_copy.rs as well to skip writing allocation bytes in MIR dumps.
Fixes#126261
Move a few intrinsics to Rust abi
Move a few more intrinsic functions to the convention added in #121192. In the second commit, I added documentation about their safety requirements. Let me know if you would like me to move the second commit to a different PR.
Note: I kept the same signature of `pref_align_of`, but I was wondering why this function is considered unsafe?
Following on PR #126502, add `rustc -Zdump-mir-exclude-alloc-bytes`
to tests/mir-opt/dataflow-const-prop/aggregate_copy.rs as well
to skip writing allocation bytes in MIR dumps.
Fixes#126261
Only track mentioned places for jump threading
This PR aims to reduce the state space size in jump threading and dataflow const-prop opts.
The current implementation walks the types of all locals, and creates a place for each possible projection. This can easily lead to a large number of places and tracked values, most being useless to the actual pass.
With this PR, we instead collect places that appear syntactically in the MIR (first commit). However, this is not sufficient (second commit), and we miss places that we could track in aggregate assignments. The third commit tracks such assignments to mirror place projections, see the inline comment.
This is complementary to https://github.com/rust-lang/rust/pull/127036
r? `@oli-obk`
Add FileCheck annotations to mir-opt/dest-prop tests
Part of https://github.com/rust-lang/rust/issues/116971, adds FileCheck annotations to MIR-opt tests in tests/mir-opt/dest-prop.
I would like some feedback. Also, I don't know how to approach `union.rs`. I couldn't figure out what it is testing.
r? cjgillot
Ignore allocation bytes in some mir-opt tests
This adds `rustc -Zdump-mir-exclude-alloc-bytes` to skip writing allocation bytes in MIR dumps, and applies it to tests that were failing on s390x due to its big-endian byte order.
Fixes#126261
Fix regression in the MIR lowering of or-patterns
In https://github.com/rust-lang/rust/pull/126553 I made a silly indexing mistake and regressed the MIR lowering of or-patterns. This fixes it.
r? `@compiler-errors` because I'd like this to be merged quickly 🙏
Fix `FnMut::call_mut`/`Fn::call` shim for async closures that capture references
I adjusted async closures to be able to implement `Fn` and `FnMut` *even if* they capture references, as long as those references did not need to borrow data from the closure captures themselves. See #125259.
However, when I did this, I didn't actually relax an assertion in the `build_construct_coroutine_by_move_shim` shim code, which builds the `Fn`/`FnMut`/`FnOnce` implementations for async closures. Therefore, if we actually tried to *call* `FnMut`/`Fn` on async closures, it would ICE.
This PR adjusts this assertion to ensure that we only capture immutable references in closures if they implement `Fn`/`FnMut`. It also adds a bunch of tests and makes more of the async-closure tests into `build-pass` since we often care about these tests actually generating the right closure shims and stuff. I think it might be excessive to *always* use build-pass here, but 🤷 it's not that big of a deal.
Fixes#127019Fixes#127012
r? oli-obk
In 126578 we ended up with more binary size increases than expected.
This change attempts to avoid inlining large things into small things, to avoid that kind of increase, in cases when top-down inlining will still be able to do that inlining later.
Remove more `PtrToPtr` casts in GVN
This addresses two things I noticed in MIR:
1. `NonNull::<T>::eq` does `(a as *mut T) == (b as *mut T)`, but it could just compare the `*const T`s, so this removes `PtrToPtr` casts that are on both sides of a pointer comparison, so long as they're not fat-to-thin casts.
2. `NonNull::<T>::addr` does `transmute::<_, usize>(p as *const ())`, but so long as `T: Thin` that cast doesn't do anything, and thus we can directly transmute the `*const T` instead.
r? mir-opt
`PtrMetadata` doesn't care about `*const`/`*mut`/`&`/`&mut`, so GVN away those casts in its argument.
This includes updating MIR to allow calling PtrMetadata on references too, not just raw pointers. That means that `[T]::len` can be just `_0 = PtrMetadata(_1)`, for example.
# Conflicts:
# tests/mir-opt/pre-codegen/slice_index.slice_get_unchecked_mut_range.PreCodegen.after.panic-abort.mir
# tests/mir-opt/pre-codegen/slice_index.slice_get_unchecked_mut_range.PreCodegen.after.panic-unwind.mir
Account for things that optimize out in inlining costs
This updates the MIR inlining `CostChecker` to have both bonuses and penalties, rather than just penalties.
That lets us add bonuses for some things where we want to encourage inlining without risking wrapping into a gigantic cost. For example, `switchInt(const …)` we give an inlining bonus because codegen will actually eliminate the branch (and associated dead blocks) once it's monomorphized, so measuring both sides of the branch gives an unrealistically-high cost to it. Similarly, an `unreachable` terminator gets a small bonus, because whatever branch leads there doesn't actually exist post-codegen.
Clean up some comments near `use` declarations
#125443 will reformat all `use` declarations in the repository. There are a few edge cases involving comments on `use` declarations that require care. This PR cleans up some clumsy comment cases, taking us a step closer to #125443 being able to merge.
r? ``@lqd``
Apparently MIR borrowck cares about at least one of these for checking variance.
In runtime MIR, though, there's no need for them as `PtrToPtr` does the same thing.
(Banning them simplifies passes like GVN that no longer need to handle multiple cast possibilities.)
coverage: Replace the old span refiner with a single function
As more and more of the span refiner's functionality has been pulled out into separate early passes, it has finally reached the point where we can remove the rest of the old `SpansRefiner` code, and replace it with a single modestly-sized function.
~~There should be no change to the resulting coverage mappings, as demonstrated by the lack of changes to test output.~~
There is *almost* no change to the resulting coverage mappings. There are some minor changes to `loop` that on inspection appear to be neutral in terms of accuracy, with the old behaviour being a slightly-horrifying implementation detail of the old code, so I think they're acceptable.
Previous work in this direction includes:
- #125921
- #121019
- #119208
As more and more of the span refiner's functionality has been pulled out into
separate early passes, it has finally reached the point where we can remove the
rest of the old `SpansRefiner` code, and replace it with a single
modestly-sized function.
Enable GVN for `AggregateKind::RawPtr`
Looks like I was worried for nothing; this seems like it's much easier than I was originally thinking it would be.
r? `@cjgillot`
This should be useful for `x[..4]`-like things, should those start inlining enough to expose the lengths.
The only non-obvious changes:
- `building/storage_live_dead_in_statics.rs` has a `#[rustfmt::skip]`
attribute to avoid reformating a table of data.
- Two `.mir` files have slight changes involving line numbers.
- In `unusual_item_types.rs` an `EMIT_MIR` annotation is moved to
outside a function, which is the usual spot, because `tidy` complains
if such a comment is indented.
The commit also tweaks the comments in `rustfmt.toml`.
The `mir!` macro has multiple parts:
- An optional return type annotation.
- A sequence of zero or more local declarations.
- A mandatory starting anonymous basic block, which is brace-delimited.
- A sequence of zero of more additional named basic blocks.
Some `mir!` invocations use braces with a "block" style, like so:
```
mir! {
let _unit: ();
{
let non_copy = S(42);
let ptr = std::ptr::addr_of_mut!(non_copy);
// Inside `callee`, the first argument and `*ptr` are basically
// aliasing places!
Call(_unit = callee(Move(*ptr), ptr), ReturnTo(after_call), UnwindContinue())
}
after_call = {
Return()
}
}
```
Some invocations use parens with a "block" style, like so:
```
mir!(
let x: [i32; 2];
let one: i32;
{
x = [42, 43];
one = 1;
x = [one, 2];
RET = Move(x);
Return()
}
)
```
And some invocations uses parens with a "tighter" style, like so:
```
mir!({
SetDiscriminant(*b, 0);
Return()
})
```
This last style is generally used for cases where just the mandatory
starting basic block is present. Its braces are placed next to the
parens.
This commit changes all `mir!` invocations to use braces with a "block"
style. Why?
- Consistency is good.
- The contents of the invocation is a block of code, so it's odd to use
parens. They are more normally used for function-like macros.
- Most importantly, the next commit will enable rustfmt for
`tests/mir-opt/`. rustfmt is more aggressive about formatting macros
that use parens than macros that use braces. Without this commit's
changes, rustfmt would break a couple of `mir!` macro invocations that
use braces within `tests/mir-opt` by inserting an extraneous comma.
E.g.:
```
mir!(type RET = (i32, bool);, { // extraneous comma after ';'
RET.0 = 1;
RET.1 = true;
Return()
})
```
Switching those `mir!` invocations to use braces avoids that problem,
resulting in this, which is nicer to read as well as being valid
syntax:
```
mir! {
type RET = (i32, bool);
{
RET.0 = 1;
RET.1 = true;
Return()
}
}
```
Enable DestinationPropagation by default.
~~Based on https://github.com/rust-lang/rust/pull/115291.~~
This PR proposes to enable the destination propagation pass by default.
This pass is meant to reduce the amount of copies present in MIR.
At the same time, this PR removes the `RenameReturnPlace` pass, as it is currently unsound.
`DestinationPropagation` is not limited to `_0`, but does not handle borrowed locals.
[ACP 362] genericize `ptr::from_raw_parts`
This implements https://github.com/rust-lang/libs-team/issues/362
As such, it can partially undo https://github.com/rust-lang/rust/pull/124795 , letting `slice_from_raw_parts` just call `from_raw_parts` again without re-introducing the unnecessary cast to MIR.
By doing this it also removes a spurious cast from `str::from_raw_parts`. And I think it does a good job of showing the value of the ACP, since the only thing that needed new turbofishing because of this is inside `ptr::null(_mut)`, but only because `ptr::without_provenance(_mut)` doesn't support pointers to extern types, which it absolutely could (without even changing the implementation).
don't inhibit random field reordering on repr(packed(1))
`inhibit_struct_field_reordering_opt` being false means we exclude this type from random field shuffling. However, `packed(1)` types can still be shuffled! The logic was added in https://github.com/rust-lang/rust/pull/48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit `-Zrandomize-layout` (which did not exist at the time).
We could add an optimization elsewhere to not bother sorting the fields for `repr(packed)` types, but I don't think that's worth the effort.
This *does* change the behavior in that we may now reorder fields of `packed(1)` structs (e.g. if there are niches, we'll try to move them to the start/end, according to `NicheBias`). We were always allowed to do that but so far we didn't. Quoting the [reference](https://doc.rust-lang.org/reference/type-layout.html):
> On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.
coverage: Memoize and simplify counter expressions
When creating coverage counter expressions as part of coverage instrumentation, we often end up creating obviously-redundant expressions like `c1 + (c0 - c1)`, which is equivalent to just `c0`.
To avoid doing so, this PR checks when we would create an expression matching one of 5 patterns, and uses the simplified form instead:
- `(a - b) + b` → `a`.
- `(a + b) - b` → `a`.
- `(a + b) - a` → `b`.
- `a + (b - a)` → `b`.
- `a - (a - b)` → `b`.
Of all the different ways to combine 3 operands and 2 operators, these are the patterns that allow simplification.
(Some of those patterns currently don't occur in practice, but are included anyway for completeness, to avoid having to add them later as branch coverage and MC/DC coverage support expands.)
---
This PR also adds memoization for newly-created (or newly-simplified) counter expressions, to avoid creating duplicates.
This currently makes no difference to the final mappings, but is expected to be useful for MC/DC coverage of match expressions, as proposed by https://github.com/rust-lang/rust/pull/124278#issuecomment-2106754753.
