MIR-OPT: Pass to deduplicate blocks
This pass finds basic blocks that are completely equal,
and replaces all uses with just one of them.
```bash
$ RUSTC_LOG=rustc_mir::transform::deduplicate_blocks ./x.py build --stage 2 | grep "SUCCESS: Replacing: " > log
...
$ cat log | wc -l
23875
```
Suggest to create a new `const` item if the `fn` in the array is a `const fn`
Fixes#73734. If the `fn` in the array repeat expression is a `const fn`, suggest creating a new `const` item. On nightly, suggest creating an inline `const` block. This PR also removes the `suggest_const_in_array_repeat_expressions` as it is no longer necessary.
Example:
```rust
fn main() {
// Should not compile but hint to create a new const item (stable) or an inline const block (nightly)
let strings: [String; 5] = [String::new(); 5];
println!("{:?}", strings);
}
```
Gives this error:
```
error[E0277]: the trait bound `std::string::String: std::marker::Copy` is not satisfied
--> $DIR/const-fn-in-vec.rs:3:32
|
2 | let strings: [String; 5] = [String::new(); 5];
| ^^^^^^^^^^^^^^^^^^ the trait `std::marker::Copy` is not implemented for `String`
|
= note: the `Copy` trait is required because the repeated element will be copied
```
With this change, this is the error message:
```
error[E0277]: the trait bound `String: Copy` is not satisfied
--> $DIR/const-fn-in-vec.rs:3:32
|
LL | let strings: [String; 5] = [String::new(); 5];
| ^^^^^^^^^^^^^^^^^^ the trait `Copy` is not implemented for `String`
|
= help: moving the function call to a new `const` item will resolve the error
```
The information logged here is of limited general interest, while at the
same times makes it impractical to simply enable logging and share the
resulting logs due to the amount of the output produced.
Reduce log level from info to debug for developer oriented information.
For example, when building cargo, this reduces the amount of logs
generated by `RUSTC_LOG=info cargo build` from 265 MB to 79 MB.
Continuation of changes from 81350.
Reduce log level used by tracing instrumentation from info to debug
Restore log level to debug to avoid make info log level overly verbose (the uses of instrument attribute modified there, were for the most part a replacement for `debug!`; one use was novel).
Remove const_in_array_repeat
Fixes#80371. Fixes#81315. Fixes#80767. Fixes#75682.
I thought there might be some issue with `Repeats(_, 0)`, but if you increase the items in the array it still ICEs. I'm not sure if this is the best fix but it does fix the given issue.
Use `reachable_as_bitset` to reuse a bitset from the traversal rather
than allocating it seprately. Additionally check if there are any
unreachable blocks before proceeding.
Prevent query cycles in the MIR inliner
r? `@eddyb` `@wesleywiser`
cc `@rust-lang/wg-mir-opt`
The general design is that we have a new query that is run on the `validated_mir` instead of on the `optimized_mir`. That query is forced before going into the optimization pipeline, so as to not try to read from a stolen MIR.
The query should not be cached cross crate, as you should never call it for items from other crates. By its very design calls into other crates can never cause query cycles.
This is a pessimistic approach to inlining, since we strictly have more calls in the `validated_mir` than we have in `optimized_mir`, but that's not a problem imo.
avoid promoting division, modulo and indexing operations that could fail
For division, `x / y` will still be promoted if `y` is a non-zero integer literal; however, `1/(1+1)` will not be promoted any more.
While at it, also see if we can reject promoting floating-point arithmetic (which are [complicated](https://github.com/rust-lang/unsafe-code-guidelines/issues/237) so maybe we should not promote them).
This will need a crater run to see if there's code out there that relies on these things being promoted.
If we can land this, promoteds in `fn`/`const fn` cannot fail to evaluate any more, which should let us do some simplifications in codegen/Miri!
Cc https://github.com/rust-lang/rfcs/pull/3027
Fixes https://github.com/rust-lang/rust/issues/61821
r? `@oli-obk`
Fix `unused_unsafe` label with `unsafe_block_in_unsafe_fn
Previously, the following code:
```rust
#![feature(unsafe_block_in_unsafe_fn)]
unsafe fn foo() {
unsafe { unsf() }
}
unsafe fn unsf() {}
```
Would give the following warning:
```
warning: unnecessary `unsafe` block
--> src/lib.rs:4:5
|
4 | unsafe { unsf() }
| ^^^^^^ unnecessary `unsafe` block
|
= note: `#[warn(unused_unsafe)]` on by default
```
which doesn't point out that the block is in an `unsafe fn`.
Tracking issue: #71668
cc #79208
Stability oddity with const intrinsics
cc `@RalfJung`
In https://github.com/rust-lang/rust/pull/80699#discussion_r551495670 `@usbalbin` realized we accepted some intrinsics as `const` without a `#[rustc_const_(un)stable]` attribute. I did some digging, and that example works because intrinsics inherit their stability from their parents... including `#[rustc_const_(un)stable]` attributes. While we may want to fix that (not sure, wasn't there just a MCPed PR that caused this on purpose?), we definitely want tests for it, thus this PR adding tests and some fun tracing statements.
Use PlaceRef projection abstractions more consistently in rustc_mir
PlaceRef contains abstractions for dealing with the `projections` array. This PR uses these abstractions more consistently within the `rustc_mir` crate.
See associated issue: rust-lang/rust#80647.
r? `@RalfJung`
implement ptr::write without dedicated intrinsic
This makes `ptr::write` more consistent with `ptr::write_unaligned`, `ptr::read`, `ptr::read_unaligned`, all of which are implemented in terms of `copy_nonoverlapping`.
This means we can also remove `move_val_init` implementations in codegen and Miri, and its special handling in the borrow checker.
Also see [this Zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/ptr.3A.3Aread.20vs.20ptr.3A.3Awrite).
Make CTFE able to check for UB...
... by not doing any optimizations on the `const fn` MIR used in CTFE. This means we duplicate all `const fn`'s MIR now, once for CTFE, once for runtime. This PR is for checking the perf effect, so we have some data when talking about https://github.com/rust-lang/const-eval/blob/master/rfcs/0000-const-ub.md
To do this, we now have two queries for obtaining mir: `optimized_mir` and `mir_for_ctfe`. It is now illegal to invoke `optimized_mir` to obtain the MIR of a const/static item's initializer, an array length, an inline const expression or an enum discriminant initializer. For `const fn`, both `optimized_mir` and `mir_for_ctfe` work, the former returning the MIR that LLVM should use if the function is called at runtime. Similarly it is illegal to invoke `mir_for_ctfe` on regular functions.
This is all checked via appropriate assertions and I don't think it is easy to get wrong, as there should be no `mir_for_ctfe` calls outside the const evaluator or metadata encoding. Almost all rustc devs should keep using `optimized_mir` (or `instance_mir` for that matter).
Add ABI argument for called function in `find_mir_or_eval_fn` and
`call_extra_fn`. Useful for comparing with expected ABI in interpreters.
Related to [miri/1631](https://github.com/rust-lang/miri/issues/1631)
MIR Inline is incompatible with coverage
Fixes: #80060
Fixed by disabling inlining if `-Zinstrument-coverage` is set.
The PR also adds additional use cases to the coverage test for doctests.
r? `@wesleywiser`
cc: `@tmandry`
Make `ExpnData` fields `krate` and `orig_id` private
These fields are only used by hygiene serialized, and should not be
accessed by anything outside of `rustc_span`.
Some Promotion Refactoring
Clean up promotion a bit:
* factor out some common code
* more exhaustive matches
This *should* not break anything... the only potentially-breaking change is that `BorrowKind::Shallow | BorrowKind::Unique` are now rejected for internal references.
r? ``@oli-obk``
Remap instrument-coverage line numbers in doctests
This uses the `SourceMap::doctest_offset_line` method to re-map line
numbers from doctests. Remapping columns is not yet done, and rustdoc
still does not output the correct filename when running doctests in a
workspace.
Part of #79417 although I dont consider that fixed until both filenames
and columns are mapped correctly.
r? `@richkadel`
I might jump on zulip the comming days. Still need to figure out how to properly write tests for this, and deal with other doctest issues in the meantime.
Always run intrinsics lowering pass
Move intrinsics lowering pass from the optimization phase (where it
would not run if -Zmir-opt-level=0), to the drop lowering phase where it
runs unconditionally.
The implementation of those intrinsics in code generation and
interpreter is unnecessary. Remove it.
Fixed conflict with drop elaboration and coverage
See
https://github.com/rust-lang/rust/issues/80045#issuecomment-745733339
Coverage statements are moved to the beginning of the BCB. This does
also affect what's counted before a panic, changing some results, but I
think these results may even be preferred? In any case, there are no
guarantees about what's counted when a panic occurs (by design).
r? `@tmandry`
FYI `@wesleywiser` `@ecstatic-morse`
Fix issue #78496
EarlyOtherwiseBranch finds MIR structures like:
```
bb0: {
...
_2 = discriminant(X)
...
switchInt(_2) -> [1_isize: bb1, otherwise: bb3]
}
bb1: {
...
_3 = discriminant(Y)
...
switchInt(_3) -> [1_isize: bb2, otherwise: bb3]
}
bb2: {...}
bb3: {...}
```
And transforms them into something like:
```
bb0: {
...
_2 = discriminant(X)
_3 = discriminant(Y)
_4 = Eq(_2, _3)
switchInt(_4) -> [true: bb4, otherwise: bb3]
}
bb2: {...} // unchanged
bb3: {...} // unchanged
bb4: {
switchInt(_2) -> [1_isize: bb2, otherwise: bb3]
}
```
But that is not always a safe thing to do -- sometimes the early `otherwise` branch is necessary so the later block could assume the value of `discriminant(X)`.
I am not totally sure what's the best way to detect that, but fixing #78496 should be easy -- we just check if `X` is a sub-expression of `Y`. A more precise test might be to check if `Y` contains a `Downcast(1)` of `X`, but I think this might be good enough.
Fix#78496
See
https://github.com/rust-lang/rust/issues/80045#issuecomment-745733339
Coverage statements are moved to the beginning of the BCB. This does
also affect what's counted before a panic, changing some results, but I
think these results may even be preferred? In any case, there are no
guarantees about what's counted when a panic occurs (by design).
Fixes reported bugs in Rust Coverage
Fixes: #79569Fixes: #79566Fixes: #79565
For the first issue (#79569), I got hit a `debug_assert!()` before
encountering the reported error message (because I have `debug = true`
enabled in my config.toml).
The assertion showed me that some `SwitchInt`s can have more than one
target pointing to the same `BasicBlock`.
I had thought that was invalid, but since it seems to be possible, I'm
allowing this now.
I added a new test for this.
----
In the last two cases above, both tests (intentionally) fail to compile,
but the `InstrumentCoverage` pass is invoked anyway.
The MIR starts with an `Unreachable` `BasicBlock`, which I hadn't
encountered before. (I had assumed the `InstrumentCoverage` pass
would only be invoked with MIRs from successful compilations.)
I don't have test infrastructure set up to test coverage on files that
fail to compile, so I didn't add a new test.
r? `@tmandry`
FYI: `@wesleywiser`
consider assignments of union field of ManuallyDrop type safe
Assigning to `Copy` union fields is safe because that assignment will never drop anything. However, with https://github.com/rust-lang/rust/pull/77547, unions may also have `ManuallyDrop` fields, and their assignments are currently still unsafe. That seems unnecessary though, as assigning `ManuallyDrop` does not drop anything either, and is thus safe even for union fields.
I assume this will at least require FCP.
[mir-opt] Allow debuginfo to be generated for a constant or a Place
Prior to this commit, debuginfo was always generated by mapping a name
to a Place. This has the side-effect that `SimplifyLocals` cannot remove
locals that are only used for debuginfo because their other uses have
been const-propagated.
To allow these locals to be removed, we now allow debuginfo to point to
a constant value. The `ConstProp` pass detects when debuginfo points to
a local with a known constant value and replaces it with the value. This
allows the later `SimplifyLocals` pass to remove the local.
Move intrinsics lowering pass from the optimization phase (where it
would not run if -Zmir-opt-level=0), to the drop lowering phase where it
runs unconditionally.
The implementation of those intrinsics in code generation and
interpreter is unnecessary. Remove it.
Adds checks for:
* `no_core` attribute
* explicitly-enabled `legacy` symbol mangling
* mir_opt_level > 1 (which enables inlining)
I removed code from the `Inline` MIR pass that forcibly disabled
inlining if `-Zinstrument-coverage` was set. The default `mir_opt_level`
does not enable inlining anyway. But if the level is explicitly set and
is greater than 1, I issue a warning.
The new warnings show up in tests, which is much better for diagnosing
potential option conflicts in these cases.
Remove an unused dependency that made `rustdoc` crash
Whilst struggling with https://github.com/rust-lang/rust/issues/79980 I discovered that this dependency was unused, and that made rustdoc crash. This PR removes it.
Fixes: #79569Fixes: #79566Fixes: #79565
For the first issue (#79569), I got hit a `debug_assert!()` before
encountering the reported error message (because I have `debug = true`
enabled in my config.toml).
The assertion showed me that some `SwitchInt`s can have more than one
target pointing to the same `BasicBlock`.
I had thought that was invalid, but since it seems to be possible, I'm
allowing this now.
I added a new test for this.
----
In the last two cases above, both tests (intentionally) fail to compile,
but the `InstrumentCoverage` pass is invoked anyway.
The MIR starts with an `Unreachable` `BasicBlock`, which I hadn't
encountered before. (I had assumed the `InstrumentCoverage` pass
would only be invoked with MIRs from successful compilations.)
I don't have test infrastructure set up to test coverage on files that
fail to compile, so I didn't add a new test.
Dogfood `str_split_once()`
Part of https://github.com/rust-lang/rust/issues/74773.
Beyond increased clarity, this fixes some instances of a common confusion with how `splitn(2)` behaves: the first element will always be `Some()`, regardless of the delimiter, and even if the value is empty.
Given this code:
```rust
fn main() {
let val = "...";
let mut iter = val.splitn(2, '=');
println!("Input: {:?}, first: {:?}, second: {:?}", val, iter.next(), iter.next());
}
```
We get:
```
Input: "no_delimiter", first: Some("no_delimiter"), second: None
Input: "k=v", first: Some("k"), second: Some("v")
Input: "=", first: Some(""), second: Some("")
```
Using `str_split_once()` makes more clear what happens when the delimiter is not found.
Fixes to Rust coverage
Fixes: #79725
Some macros can create a situation where `fn_sig_span` and `body_span`
map to different files.
New documentation on coverage tests incorrectly assumed multiple test
binaries could just be listed at the end of the `llvm-cov` command,
but it turns out each binary needs a `--object` prefix.
This PR fixes the bug and updates the documentation to correct that
issue. It also fixes a few other minor issues in internal implementation
comments, and adds documentation on getting coverage results for doc
tests.
Fixes: #79725
Some macros can create a situation where `fn_sig_span` and `body_span`
map to different files.
New documentation on coverage tests incorrectly assumed multiple test
binaries could just be listed at the end of the `llvm-cov` command,
but it turns out each binary needs a `--object` prefix.
This PR fixes the bug and updates the documentation to correct that
issue. It also fixes a few other minor issues in internal implementation
comments, and adds documentation on getting coverage results for doc
tests.
Prior to this commit, debuginfo was always generated by mapping a name
to a Place. This has the side-effect that `SimplifyLocals` cannot remove
locals that are only used for debuginfo because their other uses have
been const-propagated.
To allow these locals to be removed, we now allow debuginfo to point to
a constant value. The `ConstProp` pass detects when debuginfo points to
a local with a known constant value and replaces it with the value. This
allows the later `SimplifyLocals` pass to remove the local.
Added one more test (two files) showing coverage of generics and unused
functions across crates.
Created and referenced new Issues, as requested.
Added comments.
Added a note about the possible effects of compiler options on LLVM
coverage maps.
Fixes multiple issue with counters, with simplification
Includes a change to the implicit else span in ast_lowering, so coverage
of the implicit else no longer spans the `then` block.
Adds coverage for unused closures and async function bodies.
Fixes: #78542
Adding unreachable regions for known MIR missing from coverage map
Cleaned up PR commits, and removed link-dead-code requirement and tests
Coverage no longer depends on Issue #76038 (`-C link-dead-code` is
no longer needed or enforced, so MSVC can use the same tests as
Linux and MacOS now)
Restrict adding unreachable regions to covered files
Improved the code that adds coverage for uncalled functions (with MIR
but not-codegenned) to avoid generating coverage in files not already
included in the files with covered functions.
Resolved last known issue requiring --emit llvm-ir workaround
Fixed bugs in how unreachable code spans were added.
Allow using generic trait methods in `const fn`
Next step for https://github.com/rust-lang/rust/issues/67792, this now also allows code like the following:
```rust
struct S;
impl const PartialEq for S {
fn eq(&self, _: &S) -> bool {
true
}
}
const fn equals_self<T: PartialEq>(t: &T) -> bool {
*t == *t
}
pub const EQ: bool = equals_self(&S);
```
This works by threading const-ness of trait predicates through trait selection, in particular through `ParamCandidate`, and exposing it in the resulting `ImplSource`.
Since this change makes two bounds `T: Trait` and `T: ?const Trait` that only differ in their const-ness be treated like different bounds, candidate winnowing has been changed to drop the `?const` candidate in favor of the const candidate, to avoid ambiguities when both a const and a non-const bound is present.
Never inline naked functions
The `#[naked]` attribute disabled prologue / epilogue emission for the
function and it is responsibility of a developer to provide them. The
compiler is no position to inline such functions correctly.
Disable inlining of naked functions at LLVM and MIR level.
Closes#60919.
The `#[naked]` attribute disabled prologue / epilogue emission for the
function and it is responsibility of a developer to provide them. The
compiler is no position to inline such functions correctly.
Disable inlining of naked functions at LLVM and MIR level.
add optimization fuel checks to some mir passes
Fixes#77402
Inserts a bunch of calls to `consider_optimizing`. Note that `consider_optimizing` is the method that actually decrements the fuel count, so the point at which it's called is when the optimization takes place, from a fuel perspective. This means that where we call it has some thought behind it:
1. We probably don't want to decrement the fuel count before other simple checks, otherwise we count an optimization as being performed even if nothing was mutated (ie. it returned early).
2. In cases like `InstCombine`, where we gather optimizations in a pass and then mutate values, we probably would rather skip the gathering pass for performance reasons rather than skip the mutations afterwards.
Allow making `RUSTC_BOOTSTRAP` conditional on the crate name
Motivation: This came up in the [Zulip stream](https://rust-lang.zulipchat.com/#narrow/stream/233931-t-compiler.2Fmajor-changes/topic/Require.20users.20to.20confirm.20they.20know.20RUSTC_.E2.80.A6.20compiler-team.23350/near/208403962) for https://github.com/rust-lang/compiler-team/issues/350.
See also https://github.com/rust-lang/cargo/pull/6608#issuecomment-458546258; this implements https://github.com/rust-lang/cargo/issues/6627.
The goal is for this to eventually allow prohibiting setting `RUSTC_BOOTSTRAP` in build.rs (https://github.com/rust-lang/cargo/issues/7088).
## User-facing changes
- `RUSTC_BOOTSTRAP=1` still works; there is no current plan to remove this.
- Things like `RUSTC_BOOTSTRAP=0` no longer activate nightly features. In practice this shouldn't be a big deal, since `RUSTC_BOOTSTRAP` is the opposite of stable and everyone uses `RUSTC_BOOTSTRAP=1` anyway.
- `RUSTC_BOOTSTRAP=x` will enable nightly features only for crate `x`.
- `RUSTC_BOOTSTRAP=x,y` will enable nightly features only for crates `x` and `y`.
## Implementation changes
The main change is that `UnstableOptions::from_environment` now requires
an (optional) crate name. If the crate name is unknown (`None`), then the new feature is not available and you still have to use `RUSTC_BOOTSTRAP=1`. In practice this means the feature is only available for `--crate-name`, not for `#![crate_name]`; I'm interested in supporting the second but I'm not sure how.
Other major changes:
- Added `Session::is_nightly_build()`, which uses the `crate_name` of
the session
- Added `nightly_options::match_is_nightly_build`, a convenience method
for looking up `--crate-name` from CLI arguments.
`Session::is_nightly_build()`should be preferred where possible, since
it will take into account `#![crate_name]` (I think).
- Added `unstable_features` to `rustdoc::RenderOptions`
I'm not sure whether this counts as T-compiler or T-lang; _technically_ RUSTC_BOOTSTRAP is an implementation detail, but it's been used so much it seems like this counts as a language change too.
r? `@joshtriplett`
cc `@Mark-Simulacrum` `@hsivonen`
Normalize function type during validation
During inlining, the callee body is normalized and has types revealed,
but some of locals corresponding to the arguments might come from the
caller body which is not. As a result the caller body does not pass
validation without additional normalization.
Closes#78442.
Lower intrinsics calls: forget, size_of, unreachable, wrapping_*
This allows constant propagation to evaluate `size_of` and `wrapping_*`,
and unreachable propagation to propagate a call to `unreachable`.
The lowering is performed as a MIR optimization, rather than during MIR
building to preserve the special status of intrinsics with respect to
unsafety checks and promotion.
Currently enabled by default to determine the performance impact (no
significant impact expected). In practice only useful when combined with
inlining since intrinsics are rarely used directly (with exception of
`unreachable` and `discriminant_value` used by built-in derive macros).
Closes#32716.
This allows constant propagation to evaluate `size_of` and `wrapping_*`,
and unreachable propagation to propagate a call to `unreachable`.
The lowering is performed as a MIR optimization, rather than during MIR
building to preserve the special status of intrinsics with respect to
unsafety checks and promotion.
During inlining, the callee body is normalized and has types revealed,
but some of locals corresponding to the arguments might come from the
caller body which is not. As a result the caller body does not pass
validation without additional normalization.
The inliner looks if a sanitizer is enabled before considering
`no_sanitize` attribute as possible source of incompatibility.
The MIR inlining could happen in a crate with sanitizer disabled, but
code generation in a crate with sanitizer enabled, thus the attribute
would be incorrectly ignored.
To avoid the issue never inline functions with different `no_sanitize`
attributes.
Support inlining diverging function calls
The existing heuristic does penalize diverging calls to some degree, but since
it never inlined them previously it might need some further modifications.
Additionally introduce storage markers for all temporaries created by
the inliner. The temporary introduced for destination rebrorrow, didn't
use them previously.
Add flags customizing behaviour of MIR inlining
* `-Zinline-mir-threshold` to change the default threshold.
* `-Zinline-mir-hint-threshold` to change the threshold used by
functions with inline hint.
Having those as configurable flags makes it possible to experiment with with
different inlining thresholds and substantially increase test coverage of MIR
inlining when used with increased thresholds (for example, necessary to test
#78844).
Assert that a return place is not used for indexing during integration
The inliner integrates call destination place with callee return place
by remapping the local and adding extra projections as necessary.
If a call destination place contains any projections (which is already
possible) and a return place is used in an indexing projection (most
likely doesn't happen yet) the end result would be incorrect.
Add an assertion to ensure that potential issue won't go unnoticed in
the presence of more sophisticated copy propagation scheme.
* `-Zinline-mir-threshold` to change the default threshold.
* `-Zinline-mir-hint-threshold` to change the threshold used by
functions with inline hint.
Additionally introduce storage markers for all temporaries created by
the inliner. The temporary introduced for destination rebrorrow, didn't
use them previously.
When examining candidates for inlining, reject those that are determined
to be recursive either because of self-recursive calls or calls to any
instances already inlined.
