These assertions detect situations where a BCB node would have both a physical
counter and one or more in-edge counters/expressions.
For most BCBs that situation would indicate an implementation bug. However,
it's perfectly fine in the case of a BCB having an edge that loops back to
itself.
Given the complexity and risk involved in fixing the assertions, and the fact
that nothing relies on them actually being true, this patch just removes them
instead.
Use hir::Node helper methods instead of repeating the same impl multiple times
I wanted to do something entirely different and stumbled upon a bunch of cleanups
add_retag: ensure box-to-raw-ptr casts are preserved for Miri
In https://github.com/rust-lang/rust/pull/122233 I added `retag_box_to_raw` not realizing that we can already do `addr_of_mut!(*bx)` to turn a box into a raw pointer without an intermediate reference. We just need to ensure this information is preserved past the ElaborateBoxDerefs pass.
r? ``@oli-obk``
simplify_cfg: rename some passes so that they make more sense
I was extremely confused by `SimplifyCfg::ElaborateDrops`, since it runs way later than drop elaboration. It is used e.g. in `mir-opt/retag.rs` even though that pass doesn't care about drop elaboration at all.
"Early opt" is also very confusing since that makes it sounds like it runs early during optimizations, i.e. on runtime MIR, but actually it runs way before that.
So I decided to rename
- early-opt -> post-analysis
- elaborate-drops -> pre-optimizations
I am open to other suggestions.
preserve span when evaluating mir::ConstOperand
This lets us show to the user where they were using the faulty const (which can be quite relevant when generics are involved).
I wonder if we should change "erroneous constant encountered" to something like "the above error was encountered while evaluating this constant" or so, to make this more similar to what the collector emits when showing a "backtrace" of where things get monomorphized? It seems a bit strange to rely on the order of emitted diagnostics for that but it seems the collector already [does that](da8a8c9223/compiler/rustc_monomorphize/src/collector.rs (L472-L475)).
coverage: Initial support for branch coverage instrumentation
(This is a review-ready version of the changes that were drafted in #118305.)
This PR adds support for branch coverage instrumentation, gated behind the unstable flag value `-Zcoverage-options=branch`. (Coverage instrumentation must also be enabled with `-Cinstrument-coverage`.)
During THIR-to-MIR lowering (MIR building), if branch coverage is enabled, we collect additional information about branch conditions and their corresponding then/else blocks. We inject special marker statements into those blocks, so that the `InstrumentCoverage` MIR pass can reliably identify them even after the initially-built MIR has been simplified and renumbered.
The rest of the changes are mostly just plumbing needed to gather up the information that was collected during MIR building, and include it in the coverage metadata that we embed in the final binary.
Note that `llvm-cov show` doesn't print branch coverage information in its source views by default; that needs to be explicitly enabled with `--show-branches=count` or similar.
---
The current implementation doesn't have any support for instrumenting `if let` or let-chains. I think it's still useful without that, and adding it would be non-trivial, so I'm happy to leave that for future work.
interpret: ensure that Place is never used for a different frame
We store the address where the stack frame stores its `locals`. The idea is that even if we pop and push, or switch to a different thread with a larger number of frames, then the `locals` address will most likely change so we'll notice that problem. This is made possible by some recent changes by `@WaffleLapkin,` where we no longer use `Place` across things that change the number of stack frames.
I made these debug assertions for now, just to make sure this can't cost us any perf.
The first commit is unrelated but it's a one-line comment change so it didn't warrant a separate PR...
r? `@oli-obk`
This will allow MIR building to check whether a function is eligible for
coverage instrumentation, and avoid collecting branch coverage info if it is
not.
Distinguish between library and lang UB in assert_unsafe_precondition
As described in https://github.com/rust-lang/rust/pull/121583#issuecomment-1963168186, `assert_unsafe_precondition` now explicitly distinguishes between language UB (conditions we explicitly optimize on) and library UB (things we document you shouldn't do, and maybe some library internals assume you don't do).
`debug_assert_nounwind` was originally added to avoid the "only at runtime" aspect of `assert_unsafe_precondition`. Since then the difference between the macros has gotten muddied. This totally revamps the situation.
Now _all_ preconditions shall be checked with `assert_unsafe_precondition`. If you have a precondition that's only checkable at runtime, do a `const_eval_select` hack, as done in this PR.
r? RalfJung
Add asm goto support to `asm!`
Tracking issue: #119364
This PR implements asm-goto support, using the syntax described in "future possibilities" section of [RFC2873](https://rust-lang.github.io/rfcs/2873-inline-asm.html#asm-goto).
Currently I have only implemented the `label` part, not the `fallthrough` part (i.e. fallthrough is implicit). This doesn't reduce the expressive though, since you can use label-break to get arbitrary control flow or simply set a value and rely on jump threading optimisation to get the desired control flow. I can add that later if deemed necessary.
r? ``@Amanieu``
cc ``@ojeda``
interpret: avoid a long-lived PlaceTy in stack frames
`PlaceTy` uses a representation that's not very stable under changes to the stack. I'd feel better if we didn't have one in the long-term machine state.
r? `@oli-obk`
Fix `async Fn` confirmation for `FnDef`/`FnPtr`/`Closure` types
Fixes three issues:
1. The code in `extract_tupled_inputs_and_output_from_async_callable` was accidentally getting the *future* type and the *output* type (returned by the future) messed up for fnptr/fndef/closure types. :/
2. We have a (class of) bug(s) in the old solver where we don't really support higher ranked built-in `Future` goals for generators. This is not possible to hit on stable code, but [can be hit with `unboxed_closures`](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=e935de7181e37e13515ad01720bcb899) (#121653).
* I'm opting not to fix that in this PR. Instead, I just instantiate placeholders when confirming `async Fn` goals.
4. Fixed a bug when generating `FnPtr` shims for `async Fn` trait goals.
r? oli-obk
Add `#[rustc_no_mir_inline]` for standard library UB checks
should help with #121110 and also with #120848
Because the MIR inliner cannot know whether the checks are enabled or not, so inlining is an unnecessary compile time pessimization when debug assertions are disabled. LLVM knows whether they are enabled or not, so it can optimize accordingly without wasting time.
r? `@saethlin`
coverage: Rename `is_closure` to `is_hole`
Extracted from #121433, since I was having second thoughts about some of the other changes bundled in that PR, but these changes are still fine.
---
When refining covspans, we don't specifically care which ones represent closures; we just want to know which ones represent "holes" that should be carved out of other spans and then discarded.
(Closures are currently the only source of hole spans, but in the future we might want to also create hole spans for nested items and inactive `#[cfg(..)]` regions.)
``@rustbot`` label +A-code-coverage
When refining covspans, we don't specifically care which ones represent
closures; we just want to know which ones represent "holes" that should be
carved out of other spans and then discarded.
(Closures are currently the only source of hole spans, but in the future we
might want to also create hole spans for nested items and inactive `#[cfg(..)]`
regions.)
Make intrinsic fallback bodies cross-crate inlineable
This change was prompted by the stage1 compiler spending 4% of its time when compiling the polymorphic-recursion MIR opt test in `unlikely`.
Intrinsic fallback bodies like `unlikely` should always be inlined, it's very silly if they are not. To do this, we enable the fallback bodies to be cross-crate inlineable. Not that this matters for our workloads since the compiler never actually _uses_ the "fallback bodies", it just uses whatever was cfg(bootstrap)ped, so I've also added `#[inline]` to those.
See the comments for more information.
r? oli-obk
coverage: Remove `pending_dups` from the span refiner
When extracting coverage spans from a function's MIR, we need to decide how to handle spans that are associated with more than one node (BCB) in the coverage control flow graph.
The existing code for managing those duplicate spans is very subtle and difficult to modify. But by eagerly deduplicating those extracted spans in a much simpler way, we can remove a massive chunk of complexity from the span refiner.
There is a tradeoff here, in that we no longer try to retain *all* nondominating BCBs that have the same span, only the last one in the (semi-arbitrary) dominance ordering. But in practice, this produces very little difference in our coverage tests, and the simplification is so significant that I think it's worthwhile.
``@rustbot`` label +A-code-coverage
I have a suspicion that quite a few delayed bug paths are impossible to
reach, so I did an experiment.
I converted every `delayed_bug` to a `bug`, ran the full test suite,
then converted back every `bug` that was hit. A surprising number were
never hit.
The next commit will convert some more back, based on human judgment.
Rename `ConstPropLint` to `KnownPanicsLint`
`OverflowLint` is a clearer name because it communicates what the lint does instead of the underlying mechanism it uses (const propagation) which should be of secondary concern.
`OverflowLint` isn't the most accurate name because the lint looks for other errors as well such as division by zero not just overflows, but I couldn't think of another equally succinct name.
As a part of this change. I've also added/updated some of the comments.
cc ```@RalfJung``` ```@oli-obk``` for visibility in case you go looking for the lint using the old name.
Edit:
Changed the name from `OverflowLint` to `KnownPanicsLint`
Overhaul `Diagnostic` and `DiagnosticBuilder`
Implements the first part of https://github.com/rust-lang/compiler-team/issues/722, which moves functionality and use away from `Diagnostic`, onto `DiagnosticBuilder`.
Likely follow-ups:
- Move things around, because this PR was written to minimize diff size, so some things end up in sub-optimal places. E.g. `DiagnosticBuilder` has impls in both `diagnostic.rs` and `diagnostic_builder.rs`.
- Rename `Diagnostic` as `DiagInner` and `DiagnosticBuilder` as `Diag`.
r? `@davidtwco`
Currently many diagnostic modifier methods are available on both
`Diagnostic` and `DiagnosticBuilder`. This commit removes most of them
from `Diagnostic`. To minimize the diff size, it keeps them within
`diagnostic.rs` but changes the surrounding `impl Diagnostic` block to
`impl DiagnosticBuilder`. (I intend to move things around later, to give
a more sensible code layout.)
`Diagnostic` keeps a few methods that it still needs, like `sub`,
`arg`, and `replace_args`.
The `forward!` macro, which defined two additional methods per call
(e.g. `note` and `with_note`), is replaced by the `with_fn!` macro,
which defines one additional method per call (e.g. `with_note`). It's
now also only used when necessary -- not all modifier methods currently
need a `with_*` form. (New ones can be easily added as necessary.)
All this also requires changing `trait AddToDiagnostic` so its methods
take `DiagnosticBuilder` instead of `Diagnostic`, which leads to many
mechanical changes. `SubdiagnosticMessageOp` gains a type parameter `G`.
There are three subdiagnostics -- `DelayedAtWithoutNewline`,
`DelayedAtWithNewline`, and `InvalidFlushedDelayedDiagnosticLevel` --
that are created within the diagnostics machinery and appended to
external diagnostics. These are handled at the `Diagnostic` level, which
means it's now hard to construct them via `derive(Diagnostic)`, so
instead we construct them by hand. This has no effect on what they look
like when printed.
There are lots of new `allow` markers for `untranslatable_diagnostics`
and `diagnostics_outside_of_impl`. This is because
`#[rustc_lint_diagnostics]` annotations were present on the `Diagnostic`
modifier methods, but missing from the `DiagnosticBuilder` modifier
methods. They're now present.
This change was prompted by the stage1 compiler spending 4% of its time
when compiling the polymorphic-recursion MIR opt test in `unlikely`.
Intrinsic fallback bodies like `unlikely` should always be inlined, it's
very silly if they are not. To do this, we enable the fallback bodies to
be cross-crate inlineable. Not that this matters for our workloads since
the compiler never actually _uses_ the "fallback bodies", it just uses
whatever was cfg(bootstrap)ped, so I've also added `#[inline]` to those.
errors: only eagerly translate subdiagnostics
Subdiagnostics don't need to be lazily translated, they can always be eagerly translated. Eager translation is slightly more complex as we need to have a `DiagCtxt` available to perform the translation, which involves slightly more threading of that context.
This slight increase in complexity should enable later simplifications - like passing `DiagCtxt` into `AddToDiagnostic` and moving Fluent messages into the diagnostic structs rather than having them in separate files (working on that was what led to this change).
r? ```@nnethercote```
coverage: Discard spans that fill the entire function body
While debugging some other coverage changes, I discovered a frustrating inconsistency that occurs in functions containing closures, if they end with an implicit `()` return instead of an explicit trailing-expression.
This turns out to have been caused by the corresponding node in MIR having a span that covers the entire function body. When preparing coverage spans, any span that fills the whole body tends to cause more harm than good, so this PR detects and discards those spans.
(This isn't the first time whole-body spans have caused problems; we also eliminated some of them in #118525.)
This fixes the issue wherein the lint didn't fire for promoteds
in the case of SHL/SHR operators in non-optimized builds
and all arithmetic operators in optimized builds
Implement intrinsics with fallback bodies
fixes#93145 (though we can port many more intrinsics)
cc #63585
The way this works is that the backend logic for generating custom code for intrinsics has been made fallible. The only failure path is "this intrinsic is unknown". The `Instance` (that was `InstanceDef::Intrinsic`) then gets converted to `InstanceDef::Item`, which represents the fallback body. A regular function call to that body is then codegenned. This is currently implemented for
* codegen_ssa (so llvm and gcc)
* codegen_cranelift
other backends will need to adjust, but they can just keep doing what they were doing if they prefer (though adding new intrinsics to the compiler will then require them to implement them, instead of getting the fallback body).
cc `@scottmcm` `@WaffleLapkin`
### todo
* [ ] miri support
* [x] default intrinsic name to name of function instead of requiring it to be specified in attribute
* [x] make sure that the bodies are always available (must be collected for metadata)
When we try to extract coverage-relevant spans from MIR, sometimes we see MIR
statements/terminators whose spans cover the entire function body. Those spans
tend to be unhelpful for coverage purposes, because they often represent
compiler-inserted code, e.g. the implicit return value of `()`.
Subdiagnostics don't need to be lazily translated, they can always be
eagerly translated. Eager translation is slightly more complex as we need
to have a `DiagCtxt` available to perform the translation, which involves
slightly more threading of that context.
This slight increase in complexity should enable later simplifications -
like passing `DiagCtxt` into `AddToDiagnostic` and moving Fluent messages
into the diagnostic structs rather than having them in separate files
(working on that was what led to this change).
Signed-off-by: David Wood <david@davidtw.co>
Store static initializers in metadata instead of the MIR of statics.
This means that adding generic statics would be even more difficult, as we can't evaluate statics from other crates anymore, but the subtle issue I have encountered make me think that having this be an explicit problem is better.
The issue is that
```rust
static mut FOO: &mut u32 = &mut 42;
static mut BAR = unsafe { FOO };
```
gets different allocations, instead of referring to the same one. This is also true for non-static mut, but promotion makes `static FOO: &u32 = &42;` annoying to demo.
Fixes https://github.com/rust-lang/rust/issues/61345
## Why is this being done?
In order to ensure all crates see the same nested allocations (which is the last issue that needs fixing before we can stabilize [`const_mut_refs`](https://github.com/rust-lang/rust/issues/57349)), I am working on creating anonymous (from the Rust side, to LLVM it's like a regular static item) static items for the nested allocations in a static. If we evaluate the static item in a downstream crate again, we will end up duplicating its nested allocations (and in some cases, like the `match` case, even duplicate the main allocation).
Use fewer delayed bugs.
For some cases where it's clear that an error has already occurred, e.g.:
- there's a comment stating exactly that, or
- things like HIR lowering, where we are lowering an error kind
The commit also tweaks some comments around delayed bug sites.
r? `@oli-obk`
For some cases where it's clear that an error has already occurred,
e.g.:
- there's a comment stating exactly that, or
- things like HIR lowering, where we are lowering an error kind
The commit also tweaks some comments around delayed bug sites.
If we only check for duplicate spans when `prev` is unmodified, we reduce the
number of situations that `update_pending_dups` needs to handle.
This could potentially change the coverage spans we produce in some unknown
corner cases, but none of our current coverage tests indicate any change.
Dejargonize `subst`
In favor of #110793, replace almost every occurence of `subst` and `substitution` from rustc codes, but they still remains in subtrees under `src/tools/` like clippy and test codes (I'd like to replace them after this)
Fix async closures in CTFE
First commit renames `is_coroutine_or_closure` into `is_closure_like`, because `is_coroutine_or_closure_or_coroutine_closure` seems confusing and long.
Second commit fixes some forgotten cases where we want to handle `TyKind::CoroutineClosure` the same as closures and coroutines.
The test exercises the change to `ValidityVisitor::aggregate_field_path_elem` which is the source of #120946, but not the change to `UsedParamsNeedSubstVisitor`, though I feel like it's not that big of a deal. Let me know if you'd like for me to look into constructing a test for the latter, though I have no idea what it'd look like (we can't assert against `TooGeneric` anywhere?).
Fixes#120946
r? oli-obk cc ``@RalfJung``
Check that the ABI of the instance we are inlining is correct
When computing the `CallSite` in the mir inliner, double check that the instance of the function that we are inlining is compatible with the signature from the trait definition that we acquire from the MIR.
Fixes#120940
r? ``@oli-obk`` or ``@cjgillot``
Remove a bunch of dead parameters in functions
Found this kind of issue when working on https://github.com/rust-lang/rust/pull/119650
I wrote a trivial toy lint and manual review to find these.
Assert that params with the same *index* have the same *name*
Found this bug when trying to build libcore with the new solver, since it will canonicalize two params with the same index into *different* placeholders if those params differ by name.
Fold pointer operations in GVN
This PR proposes 2 combinations of cast operations in MIR GVN:
- a chain of `PtrToPtr` or `MutToConstPointer` casts can be folded together into a single `PtrToPtr` cast;
- we attempt to evaluate more ptr ops when there is no provenance.
In particular, this allows to read from static slices.
This is not yet sufficient to see through slice operations that use `PtrComponents` (because that's a union), but still a step forward.
r? `@ghost`
These crates all needed specialization for `newtype_index!`, which will no
longer be necessary when the current nightly eventually becomes the next
bootstrap compiler.
Fix more `ty::Error` ICEs in MIR passes
Fixes#120791 - Add a check for `ty::Error` in the `ByMove` coroutine pass
Fixes#120816 - Add a check for `ty::Error` in the MIR validator
Also a drive-by fix for a FIXME I had asked oli to add
r? oli-obk
Invert diagnostic lints.
That is, change `diagnostic_outside_of_impl` and `untranslatable_diagnostic` from `allow` to `deny`, because more than half of the compiler has been converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow` attributes, which proves that this change is warranted.
r? ````@davidtwco````
Toggle assert_unsafe_precondition in codegen instead of expansion
The goal of this PR is to make some of the unsafe precondition checks in the standard library available in debug builds. Some UI tests are included to verify that it does that.
The diff is large, but most of it is blessing mir-opt tests and I've also split up this PR so it can be reviewed commit-by-commit.
This PR:
1. Adds a new intrinsic, `debug_assertions` which is lowered to a new MIR NullOp, and only to a constant after monomorphization
2. Rewrites `assume_unsafe_precondition` to check the new intrinsic, and be monomorphic.
3. Skips codegen of the `assume` intrinsic in unoptimized builds, because that was silly before but with these checks it's *very* silly
4. The checks with the most overhead are `ptr::read`/`ptr::write` and `NonNull::new_unchecked`. I've simply added `#[cfg(debug_assertions)]` to the checks for `ptr::read`/`ptr::write` because I was unable to come up with any (good) ideas for decreasing their impact. But for `NonNull::new_unchecked` I found that the majority of callers can use a different function, often a safe one.
Yes, this PR slows down the compile time of some programs. But in our benchmark suite it's never more than 1% icount, and the average icount change in debug-full programs is 0.22%. I think that is acceptable for such an improvement in developer experience.
https://github.com/rust-lang/rust/issues/120539#issuecomment-1922687101
Fix mir pass ICE in the presence of other errors
fixes#120779
it is impossible to add a ui test for this, because it only reproduces in build-fail, but a test that also has errors in check-fail mode can't be made build-fail 🙃
I would have to add a run-make test or sth, which is overkill for such a tiny thing imo.
Remove unused args from functions
`#[instrument]` suppresses the unused arguments from a function, *and* suppresses unused methods too! This PR removes things which are only used via `#[instrument]` calls, and fixes some other errors (privacy?) that I will comment inline.
It's possible that some of these arguments were being passed in for the purposes of being instrumented, but I am unconvinced by most of them.
