Simplify and consolidate the way we handle construct `OutlivesEnvironment` for lexical region resolution
This is best reviewed commit-by-commit. I tried to consolidate the API for lexical region resolution *first*, then change the API when it was finally behind a single surface.
r? lcnr or reassign
Properly check that array length is valid type during built-in unsizing in index
This results in duplicated errors, but this class of errors is not new; in general, we aren't really equipped to detect cases where a WF error due to a field type would be shadowed by the parent struct of that field also not being WF.
This also adds a note for these types of mismatches to make it clear that this is due to an array type.
Fixes#134352
r? boxyuwu
Rollup of 7 pull requests
Successful merges:
- #135625 ([cfg_match] Document the use of expressions.)
- #135902 (Do not consider child bound assumptions for rigid alias)
- #135943 (Rename `Piece::String` to `Piece::Lit`)
- #136104 (Add mermaid graphs of NLL regions and SCCs to polonius MIR dump)
- #136143 (Update books)
- #136147 (ABI-required target features: warn when they are missing in base CPU)
- #136164 (Refactor FnKind variant to hold &Fn)
r? `@ghost`
`@rustbot` modify labels: rollup
Do not consider child bound assumptions for rigid alias
r? lcnr
See first commit for the important details. For second commit, I also stacked a somewhat opinionated name change, though I can separate that if needed.
Fixes https://github.com/rust-lang/trait-system-refactor-initiative/issues/149
Use `structurally_normalize` instead of manual `normalizes-to` goals in alias relate errors
r? `@lcnr`
I added `structurally_normalize_term` so that code that is generic over ty or const can use the structurally normalize helpers. See `tests/ui/traits/next-solver/diagnostics/alias_relate_error_uses_structurally_normalize.rs` for a description of the reason for the (now fixed) ICEs
This CL makes a number of small changes to dyn compatibility errors:
- "object safety" has been renamed to "dyn-compatibility" throughout
- "Convert to enum" suggestions are no longer generated when there
exists a type-generic impl of the trait or an impl for `dyn OtherTrait`
- Several error messages are reorganized for user readability
Additionally, the dyn compatibility error creation code has been
split out into functions.
cc #132713
cc #133267
Rework dyn trait lowering to stop being so intertwined with trait alias expansion
This PR reworks the trait object lowering code to stop handling trait aliases so funky, and removes the `TraitAliasExpander` in favor of a much simpler design. This refactoring is important for making the code that I'm writing in https://github.com/rust-lang/rust/pull/133397 understandable and easy to maintain, so the diagnostics regressions are IMO inevitable.
In the old trait object lowering code, we used to be a bit sloppy with the lists of traits in their unexpanded and expanded forms. This PR largely rewrites this logic to expand the trait aliases *once* and handle them more responsibly throughout afterwards.
Please review this with whitespace disabled.
r? lcnr
new solver: prefer trivial builtin impls
As discussed [on zulip](https://rust-lang.zulipchat.com/#narrow/channel/364551-t-types.2Ftrait-system-refactor/topic/needs_help.3A.20trivial.20builtin.20impls), this PR:
- adds a new `BuiltinImplSource::Trivial` source, and marks the `Sized` builtin impls as trivial
- prefers these trivial builtin impls in `merge_trait_candidates`
The comments can likely be wordsmithed a bit better, and I ~stole~ was inspired by the old solver ones. Let me know how you want them improved.
When enabling the new solver for tests, 3 UI tests now pass:
- `regions/issue-26448-1.rs` and its sibling `regions/issue-26448-2.rs` were rejected by the new solver but accepted by the old one
- and `issues/issue-42796.rs` where the old solver emitted some overflow errors in addition to the expected error
(For some reason one of these tests is run-pass, but I can take care of that another day)
r? lcnr
Prefer lower `TraitUpcasting` candidates in selection
Fixes#135463. The underlying cause is this ambiguity, but it's more clear (and manifests as a coercion error, rather than a MIR validation error) when it's written the way I did in the UI test.
Sorry this is cursed r? lcnr
handle member constraints directly in the mir type checker
cleaner, faster, easier to change going forward :> fixes#109654
r? `@oli-obk` `@compiler-errors`
Re-export more `rustc_span::symbol` things from `rustc_span`.
`rustc_span::symbol` defines some things that are re-exported from `rustc_span`, such as `Symbol` and `sym`. But it doesn't re-export some closely related things such as `Ident` and `kw`. So you can do `use rustc_span::{Symbol, sym}` but you have to do `use rustc_span::symbol::{Ident, kw}`, which is inconsistent for no good reason.
This commit re-exports `Ident`, `kw`, and `MacroRulesNormalizedIdent`, and changes many `rustc_span::symbol::` qualifiers to `rustc_span::`. This is a 300+ net line of code reduction, mostly because many files with two `use rustc_span` items can be reduced to one.
r? `@jieyouxu`
`rustc_span::symbol` defines some things that are re-exported from
`rustc_span`, such as `Symbol` and `sym`. But it doesn't re-export some
closely related things such as `Ident` and `kw`. So you can do `use
rustc_span::{Symbol, sym}` but you have to do `use
rustc_span::symbol::{Ident, kw}`, which is inconsistent for no good
reason.
This commit re-exports `Ident`, `kw`, and `MacroRulesNormalizedIdent`,
and changes many `rustc_span::symbol::` qualifiers in `compiler/` to
`rustc_span::`. This is a 200+ net line of code reduction, mostly
because many files with two `use rustc_span` items can be reduced to
one.
Add some convenience helper methods on `hir::Safety`
Makes a lot of call sites simpler and should make any refactorings needed for https://github.com/rust-lang/rust/pull/134090#issuecomment-2541332415 simpler, as fewer sites have to be touched in case we end up storing some information in the variants of `hir::Safety`
Move impl constness into impl trait header
This PR is kind of the opposite of the rejected https://github.com/rust-lang/rust/pull/134114
Instead of moving more things into the `constness` query, we want to keep them where their corresponding hir nodes are lowered. So I gave this a spin for impls, which have an obvious place to be (the impl trait header). And surprisingly it's also a perf improvement (likely just slightly better query & cache usage).
The issue was that removing anything from the `constness` query makes it just return `NotConst`, which is wrong. So I had to change it to `bug!` out if used wrongly, and only then remove the impl blocks from the `constness` query. I think this change is good in general, because it makes using `constness` more robust (as can be seen by how few sites that had to be changed, so it was almost solely used specifically for the purpose of asking for functions' constness). The main thing where this change was not great was in clippy, which was using the `constness` query as a general DefId -> constness map. I added a `DefKind` filter in front of that. If it becomes a more common pattern we can always move that helper into rustc.
Add unpolished, experimental support for AFIDT (async fn in dyn trait)
This allows us to begin messing around `async fn` in `dyn Trait`. Calling an async fn from a trait object always returns a `dyn* Future<Output = ...>`.
To make it work, Implementations are currently required to return something that can be coerced to a `dyn* Future` (see the example in `tests/ui/async-await/dyn/works.rs`). If it's not the right size, then it'll raise an error at the coercion site (see the example in `tests/ui/async-await/dyn/wrong-size.rs`). Currently the only practical way of doing this is wrapping the body in `Box::pin(async move { .. })`.
This PR does not implement a helper type like a "`Boxing`"[^boxing] adapter, and I'll probably follow-up with another PR to improve the error message for the `PointerLike` trait (something that explains in just normal prose what is happening here, rather than a trait error).
[^boxing]: https://rust-lang.github.io/async-fundamentals-initiative/explainer/user_guide_future.html#the-boxing-adapter
This PR also does not implement new trait solver support for AFIDT; I'll need to think how best to integrate it into candidate assembly, and that's a bit of a matter of taste, but I don't think it will be difficult to do.
This could also be generalized:
* To work on functions that are `-> impl Future` (soon).
* To work on functions that are `-> impl Iterator` and other "dyn rpitit safe" traits. We still need to nail down exactly what is needed for this to be okay (not soon).
Tracking:
* https://github.com/rust-lang/rust/issues/133119
As a rule, the application of `unsafe` to a declaration requires that use-sites
of that declaration also require `unsafe`. For example, a field declared
`unsafe` may only be read in the lexical context of an `unsafe` block.
For nearly all safe traits, the safety obligations of fields are explicitly
discharged when they are mentioned in method definitions. For example,
idiomatically implementing `Clone` (a safe trait) for a type with unsafe fields
will require `unsafe` to clone those fields.
Prior to this commit, `Copy` violated this rule. The trait is marked safe, and
although it has no explicit methods, its implementation permits reads of `Self`.
This commit resolves this by making `Copy` conditionally safe to implement. It
remains safe to implement for ADTs without unsafe fields, but unsafe to
implement for ADTs with unsafe fields.
Tracking: #132922
If a type has unsafe fields, its safety invariants are not simply
the conjunction of its field types' safety invariants. Consequently,
it's invalid to reason about the safety properties of these types
in a purely structural manner — i.e., the manner in which `auto`
traits are implemented.
Makes progress towards #132922.
Rollup of 10 pull requests
Successful merges:
- #118833 (Add lint against function pointer comparisons)
- #122161 (Fix suggestion when shorthand `self` has erroneous type)
- #133233 (Add context to "const in pattern" errors)
- #133761 (Update books)
- #133843 (Do not emit empty suggestion)
- #133863 (Rename `core_pattern_type` and `core_pattern_types` lib feature gates to `pattern_type_macro`)
- #133872 (No need to create placeholders for GAT args in confirm_object_candidate)
- #133874 (`fn_sig_for_fn_abi` should return a `ty::FnSig`, no need for a binder)
- #133890 (Add a new test ui/incoherent-inherent-impls/no-other-unrelated-errors to check E0116 does not cause unrelated errors)
- #133892 (Revert #133817)
r? `@ghost`
`@rustbot` modify labels: rollup
Deeply normalize when computing implied outlives bounds
r? lcnr
Unfortunately resolving regions is still slightly scuffed (though in an unrelated way). Specifically, we should be normalizing our param-env outlives when constructing the `OutlivesEnv`; otherwise, these assumptions (dd2837ec5d/compiler/rustc_infer/src/infer/outlives/env.rs (L78)) are not constructed correctly.
Let me know if you want us to track that somewhere.
check local cache even if global is usable
we store overflow errors locally, even if we can otherwise use the global cache for this goal. should fix#133616, didn't test it locally yet as diesel tends to hit an unrelated debug assertion in rustdoc.
r? types
Some minor dyn-related tweaks
Each commit should be self-explanatory, but I'm happy to explain what's going on if not. These are tweaks I pulled out of #133388, but they can be reviewed sooner than that.
r? types
finish `Reveal` removal
After #133212 changed the `TypingMode` to be the only source of truth, this entirely rips out `Reveal`.
cc #132279
r? `@compiler-errors`
Stop being so bail-y in candidate assembly
A conceptual follow-up to #132084. We gotta stop bailing so much when there are errors; it's both unnecessary, leads to weird knock-on errors, and it's messing up the vibes lol
Bail in effects in old solver if self ty is ty var
Otherwise when we try to check something like `?t: ~const Trait` we'll immediately stick it to the first param-env candidate, lol.
r? lcnr
the behavior of the type system not only depends on the current
assumptions, but also the currentnphase of the compiler. This is
mostly necessary as we need to decide whether and how to reveal
opaque types. We track this via the `TypingMode`.
Only disable cache if predicate has opaques within it
This is an alternative to https://github.com/rust-lang/rust/pull/132075.
This refines the check implemented in https://github.com/rust-lang/rust/pull/126024 to only disable the global cache if the predicate being considered has opaques in it. This is still theoretically unsound, since goals can indirectly rely on opaques in the defining scope, but we're much less likely to hit it.
It doesn't totally fix https://github.com/rust-lang/rust/issues/132064: for example, `lemmy` goes from 1:29 (on rust 1.81) to 9:53 (on nightly) to 4:07 (after this PR). But I think it's at least *more* sound than a total revert :/
r? lcnr
Remove unnecessary pub enum glob-imports from `rustc_middle::ty`
We used to have an idiom in the compiler where we'd prefix or suffix all the variants of an enum, for example `BoundRegionKind`, with something like `Br`, and then *glob-import* that enum variant directly.
`@noratrieb` brought this up, and I think that it's easier to read when we just use the normal style `EnumName::Variant`.
This PR is a bit large, but it's just naming.
The only somewhat opinionated change that this PR does is rename `BorrowKind::Imm` to `BorrowKind::Immutable` and same for the other variants. I think these enums are used sparingly enough that the extra length is fine.
r? `@noratrieb` or reassign
Rename `rustc_abi::Abi` to `BackendRepr`
Remove the confabulation of `rustc_abi::Abi` with what "ABI" actually means by renaming it to `BackendRepr`, and rename `Abi::Aggregate` to `BackendRepr::Memory`. The type never actually represented how things are passed, as that has to have `PassMode` considered, at minimum, but rather it just is how we represented some things to the backend. This conflation arose because LLVM, the primary backend at the time, would lower certain IR forms using certain ABIs. Even that only somewhat was true, as it broke down when one ventured significantly afield of what is described by the System V AMD64 ABI either by using different architectures, ABI-modifying IR annotations, the same architecture **with different ISA extensions enabled**, or other... unexpected delights.
Unfortunately both names are still somewhat of a misnomer right now, as people have written code for years based on this misunderstanding. Still, their original names are even moreso, and for better or worse, this backend code hasn't received as much maintenance as the rest of the compiler, lately. Actually arriving at a correct end-state will simply require us to disentangle a lot of code in order to fix, much of it pointlessly repeated in several places. Thus this is not an "actual fix", just a way to deflect further misunderstandings.
The initial naming of "Abi" was an awful mistake, conveying wrong ideas
about how psABIs worked and even more about what the enum meant.
It was only meant to represent the way the value would be described to
a codegen backend as it was lowered to that intermediate representation.
It was never meant to mean anything about the actual psABI handling!
The conflation is because LLVM typically will associate a certain form
with a certain ABI, but even that does not hold when the special cases
that actually exist arise, plus the IR annotations that modify the ABI.
Reframe `rustc_abi::Abi` as the `BackendRepr` of the type, and rename
`BackendRepr::Aggregate` as `BackendRepr::Memory`. Unfortunately, due to
the persistent misunderstandings, this too is now incorrect:
- Scattered ABI-relevant code is entangled with BackendRepr
- We do not always pre-compute a correct BackendRepr that reflects how
we "actually" want this value to be handled, so we leave the backend
interface to also inject various special-cases here
- In some cases `BackendRepr::Memory` is a "real" aggregate, but in
others it is in fact using memory, and in some cases it is a scalar!
Our rustc-to-backend lowering code handles this sort of thing right now.
That will eventually be addressed by lifting duplicated lowering code
to either rustc_codegen_ssa or rustc_target as appropriate.
Then we can rename the _raw functions to drop their suffix, and instead
explicitly use is_stable_const_fn for the few cases where that is really what
you want.
Represent trait constness as a distinct predicate
cc `@rust-lang/project-const-traits`
r? `@ghost` for now
Also mirrored everything that is written below on this hackmd here: https://hackmd.io/`@compiler-errors/r12zoixg1l`
# Tl;dr:
* This PR removes the bulk of the old effect desugaring.
* This PR reimplements most of the effect desugaring as a new predicate and set of a couple queries. I believe it majorly simplifies the implementation and allows us to move forward more easily on its implementation.
I'm putting this up both as a request for comments and a vibe-check, but also as a legitimate implementation that I'd like to see land (though no rush of course on that last part).
## Background
### Early days
Once upon a time, we represented trait constness in the param-env and in `TraitPredicate`. This was very difficult to implement correctly; it had bugs and was also incomplete; I don't think this was anyone's fault though, it was just the limit of experimental knowledge we had at that point.
Dealing with `~const` within predicates themselves meant dealing with constness all throughout the trait solver. This was difficult to keep track of, and afaict was not handled well with all the corners of candidate assembly.
Specifically, we had to (in various places) remap constness according to the param-env constness:
574b64a97f/compiler/rustc_trait_selection/src/traits/select/mod.rs (L1498)
This was annoying and manual and also error prone.
### Beginning of the effects desugaring
Later on, #113210 reimplemented a new desugaring for const traits via a `<const HOST: bool>` predicate. This essentially "reified" the const checking and separated it from any of the remapping or separate tracking in param-envs. For example, if I was in a const-if-const environment, but I wanted to call a trait that was non-const, this reification would turn the constness mismatch into a simple *type* mismatch of the effect parameter.
While this was a monumental step towards straightening out const trait checking in the trait system, it had its own issues, since that meant that the constness of a trait (or any item within it, like an associated type) was *early-bound*. This essentially meant that `<T as Trait>::Assoc` was *distinct* from `<T as ~const Trait>::Assoc`, which was bad.
### Associated-type bound based effects desugaring
After this, #120639 implemented a new effects desugaring. This used an associated type to more clearly represent the fact that the constness is not an input parameter of a trait, but a property that could be computed of a impl. The write-up linked in that PR explains it better than I could.
However, I feel like it really reached the limits of what can comfortably be expressed in terms of associated type and trait calculus. Also, `<const HOST: bool>` remains a synthetic const parameter, which is observable in nested items like RPITs and closures, and comes with tons of its own hacks in the astconv and middle layer.
For example, there are pieces of unintuitive code that are needed to represent semantics like elaboration, and eventually will be needed to make error reporting intuitive, and hopefully in the future assist us in implementing built-in traits (eventually we'll want something like `~const Fn` trait bounds!).
elaboration hack: 8069f8d17a/compiler/rustc_type_ir/src/elaborate.rs (L133-L195)
trait bound remapping hack for diagnostics: 8069f8d17a/compiler/rustc_trait_selection/src/error_reporting/traits/fulfillment_errors.rs (L2370-L2413)
I want to be clear that I don't think this is a issue of implementation quality or anything like that; I think it's simply a very clear sign that we're using types and traits in a way that they're not fundamentally supposed to be used, especially given that constness deserves to be represented as a first-class concept.
### What now?
This PR implements a new desugaring for const traits. Specifically, it introduces a `HostEffect` predicate to represent the obligation an impl is const, rather than using associated type bounds and the compat trait that exists for effects today.
### `HostEffect` predicate
A `HostEffect` clause has two parts -- the `TraitRef` we're trying to prove, and a `HostPolarity::{Maybe, Const}`.
`HostPolarity::Const` corresponds to `T: const Trait` bounds, which must *always* be proven as const, and which can be written in any context. These are lowered directly into the predicates of an item, since they're not "context-specific".
On the other hand, `HostPolarity::Maybe` corresponds to `T: ~const Trait` bounds which must only exist in a conditionally-const context like a method in a `#[const_trait]`, or a `const fn` free function. We do not lower these immediately into the predicates of an item; instead, we collect them into a new query called the **`const_conditions`**. These are the set of trait refs that we need to prove have const implementations for an item to be const.
Notably, they're represented as bare (poly) trait refs because they are meant to be paired back together with a `HostPolarity` when they're being registered in typeck (see next section).
For example, given:
```rust
const fn foo<T: ~const A + const B>() {}
```
`foo`'s const conditions would contain `T: A`, but not `T: B`. On the flip side, foo's predicates (`predicates_of`) query would contain `HostEffect(T: B, HostPolarity::Const)` but not `HostEffect(T: A, HostPolarity::Maybe)` since we don't need to prove that predicate in a non-const environment (and it's not even the right predicate to prove in an unconditionally const environment).
### Type checking const bodies
When type checking bodies in HIR, when we encounter a call expression, we additionally register the callee item's const conditions with the `HostPolarity` from the body we're typechecking (`Const` for unconditionally const things like `const`/`static` items, and `Maybe` for conditionally const things like const fns; and we don't register `HostPolarity` predicates for non-const bodies).
When type-checking a conditionally const body, we augment its param-env with `HostEffect(..., Maybe)` predicates.
### Checking that const impls are WF
We extend the logic in `compare_method_predicate_entailment` to also check the const-conditions of the impl method, to make sure that we error for:
```rust
#[const_trait] Bar {}
#[const_trait] trait Foo {
fn method<T: Bar>();
}
impl Foo for () {
fn method<T: ~const Bar>() {} // stronger assumption!
}
```
We also extend the WF check for impls to register the const conditions of the trait that is being implemented. This is to make sure we error for:
```rust
#[const_trait] trait Bar {}
#[const_trait] trait Foo<T> where T: ~const Bar {}
impl<T> const Foo<T> for () {}
//~^ `T: ~const Bar` is missing!
```
### Proving a `HostEffect` predicate
We have several ways of proving a `HostEffect` predicate:
1. Matching a `HostEffect` predicate from the param-env
2. From an impl - we do impl selection very similar to confirming a trait goal, except we filter for only const impls, and we additionally register the impl's const conditions (i.e. the impl's `~const` where clauses).
Later I expect that we will add more built-in implementations for things like `Fn`.
## What next?
After this PR, I'd like to split out the work more so it can proceed in parallel and probably amongst others that are not me.
* Register `HostEffect` goal for places in HIR typeck that correspond to call terminators, like autoderef.
* Make traits in libstd const again.
* Probably need to impl host effect preds in old solver.
* Implement built-in `HostEffect` rules for traits like `Fn`.
* Rip out const checking from MIR altogether.
## So what?
This ends up being super convenient basically everywhere in the compiler. Due to the design of the new trait solver, we end up having an almost parallel structure to the existing trait and projection predicates for assembling `HostEffect` predicates; adding new candidates and especially new built-in implementations is now basically trivial, and it's quite straightforward to understand the confirmation logic for these predicates.
Same with diagnostics reporting; since we have predicates which represent the obligation to prove an impl is const, we can simplify and make these diagnostics richer without having to write a ton of logic to intercept and rewrite the existing `Compat` trait errors.
Finally, it gives us a much more straightforward path for supporting the const effect on the old trait solver. I'm personally quite passionate about getting const trait support into the hands of users without having to wait until the new solver lands[^1], so I think after this PR lands we can begin to gauge how difficult it would be to implement constness in the old trait solver too. This PR will not do this yet.
[^1]: Though this is not a prerequisite or by any means the only justification for this PR.
Dont consider predicates that may hold as impossible in `is_impossible_associated_item`
Use infer vars to account for ambiguities when considering if methods are impossible to instantiate for a given self type. Also while we're at it, let's use the new trait solver instead of `evaluate` since this is used in rustdoc.
r? lcnr
Fixes#131839
Rollup of 4 pull requests
Successful merges:
- #126588 (Added more scenarios where comma to be removed in the function arg)
- #131728 (bootstrap: extract builder cargo to its own module)
- #131968 (Rip out old effects var handling code from traits)
- #131981 (Remove the `BoundConstness::NotConst` variant)
r? `@ghost`
`@rustbot` modify labels: rollup
Never emit `vptr` for empty/auto traits
Emiting `vptr`s for empty/auto traits is unnecessary (#114942) and causes unsoundness in `trait_upcasting` (#131813). This PR should ensure that we never emit vtables for such traits. See the linked issues for more details.
I'm not sure if I can add tests for the vtable layout. So this PR only adds tests for the soundness hole (i.e., the segmentation fault will disappear after this PR).
Fixes#114942Fixes#131813
Cc #65991 (tracking issue for `trait_upcasting`)
r? `@WaffleLapkin` (per https://github.com/rust-lang/rust/issues/131813#issuecomment-2419969745)
Use `ThinVec` for PredicateObligation storage
~~I noticed while profiling clippy on a project that a large amount of time is being spent allocating `Vec`s for `PredicateObligation`, and the `Vec`s are often quite small. This is an attempt to optimise this by using SmallVec to avoid heap allocations for these common small Vecs.~~
This PR turns all the `Vec<PredicateObligation>` into a single type alias while avoiding referring to `Vec` around it, then swaps the type over to `ThinVec<PredicateObligation>` and fixes the fallout. This also contains an implementation of `ThinVec::extract_if`, copied from `Vec::extract_if` and currently being upstreamed to https://github.com/Gankra/thin-vec/pull/66.
This leads to a small (0.2-0.7%) performance gain in the latest perf run.
stabilize `-Znext-solver=coherence` again
r? `@compiler-errors`
---
This PR stabilizes the use of the next generation trait solver in coherence checking by enabling `-Znext-solver=coherence` by default. More specifically its use in the *implicit negative overlap check*. The tracking issue for this is https://github.com/rust-lang/rust/issues/114862. Closes#114862.
This is a direct copy of #121848 which has been reverted due to a hang in `nalgebra`: #130056. This hang should have been fixed by #130617 and #130821. See the added section in the stabilization report containing user facing changes merged since the original FCP.
## Background
### The next generation trait solver
The new solver lives in [`rustc_trait_selection::solve`](https://github.com/rust-lang/rust/blob/master/compiler/rustc_trait_selection/src/solve/mod.rs) and is intended to replace the existing *evaluate*, *fulfill*, and *project* implementation. It also has a wider impact on the rest of the type system, for example by changing our approach to handling associated types.
For a more detailed explanation of the new trait solver, see the [rustc-dev-guide](https://rustc-dev-guide.rust-lang.org/solve/trait-solving.html). This does not stabilize the current behavior of the new trait solver, only the behavior impacting the implicit negative overlap check. There are many areas in the new solver which are not yet finalized. We are confident that their final design will not conflict with the user-facing behavior observable via coherence. More on that further down.
Please check out [the chapter](https://rustc-dev-guide.rust-lang.org/solve/significant-changes.html) summarizing the most significant changes between the existing and new implementations.
### Coherence and the implicit negative overlap check
Coherence checking detects any overlapping impls. Overlapping trait impls always error while overlapping inherent impls result in an error if they have methods with the same name. Coherence also results in an error if any other impls could exist, even if they are currently unknown. This affects impls which may get added to upstream crates in a backwards compatible way and impls from downstream crates.
Coherence failing to detect overlap is generally considered to be unsound, even if it is difficult to actually get runtime UB this way. It is quite easy to get ICEs due to bugs in coherence.
It currently consists of two checks:
The [orphan check] validates that impls do not overlap with other impls we do not know about: either because they may be defined in a sibling crate, or because an upstream crate is allowed to add it without being considered a breaking change.
The [overlap check] validates that impls do not overlap with other impls we know about. This is done as follows:
- Instantiate the generic parameters of both impls with inference variables
- Equate the `TraitRef`s of both impls. If it fails there is no overlap.
- [implicit negative]: Check whether any of the instantiated `where`-bounds of one of the impls definitely do not hold when using the constraints from the previous step. If a `where`-bound does not hold, there is no overlap.
- *explicit negative (still unstable, ignored going forward)*: Check whether the any negated `where`-bounds can be proven, e.g. a `&mut u32: Clone` bound definitely does not hold as an explicit `impl<T> !Clone for &mut T` exists.
The overlap check has to *prove that unifying the impls does not succeed*. This means that **incorrectly getting a type error during coherence is unsound** as it would allow impls to overlap: coherence has to be *complete*.
Completeness means that we never incorrectly error. This means that during coherence we must only add inference constraints if they are definitely necessary. During ordinary type checking [this does not hold](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=01d93b592bd9036ac96071cbf1d624a9), so the trait solver has to behave differently, depending on whether we're in coherence or not.
The implicit negative check only considers goals to "definitely not hold" if they could not be implemented downstream, by a sibling, or upstream in a backwards compatible way. If the goal is is "unknowable" as it may get added in another crate, we add an ambiguous candidate: [source](bea5bebf3d/compiler/rustc_trait_selection/src/solve/assembly/mod.rs (L858-L883)).
[orphan check]: fd80c02c16/compiler/rustc_trait_selection/src/traits/coherence.rs (L566-L579)
[overlap check]: fd80c02c16/compiler/rustc_trait_selection/src/traits/coherence.rs (L92-L98)
[implicit negative]: fd80c02c16/compiler/rustc_trait_selection/src/traits/coherence.rs (L223-L281)
## Motivation
Replacing the existing solver in coherence fixes soundness bugs by removing sources of incompleteness in the type system. The new solver separately strengthens coherence, resulting in more impls being disjoint and passing the coherence check. The concrete changes will be elaborated further down. We believe the stabilization to reduce the likelihood of future bugs in coherence as the new implementation is easier to understand and reason about.
It allows us to remove the support for coherence and implicit-negative reasoning in the old solver, allowing us to remove some code and simplifying the old trait solver. We will only remove the old solver support once this stabilization has reached stable to make sure we're able to quickly revert in case any unexpected issues are detected before then.
Stabilizing the use of the next-generation trait solver expresses our confidence that its current behavior is intended and our work towards enabling its use everywhere will not require any breaking changes to the areas used by coherence checking. We are also confident that we will be able to replace the existing solver everywhere, as maintaining two separate systems adds a significant maintainance burden.
## User-facing impact and reasoning
### Breakage due to improved handling of associated types
The new solver fixes multiple issues related to associated types. As these issues caused coherence to consider more types distinct, fixing them results in more overlap errors. This is therefore a breaking change.
#### Structurally relating aliases containing bound vars
Fixes https://github.com/rust-lang/rust/issues/102048. In the existing solver relating ambiguous projections containing bound variables is structural. This is *incomplete* and allows overlapping impls. These was mostly not exploitable as the same issue also caused impls to not apply when trying to use them. The new solver defers alias-relating to a nested goal, fixing this issue:
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
trait Trait {}
trait Project {
type Assoc<'a>;
}
impl Project for u32 {
type Assoc<'a> = &'a u32;
}
// Eagerly normalizing `<?infer as Project>::Assoc<'a>` is ambiguous,
// so the old solver ended up structurally relating
//
// (?infer, for<'a> fn(<?infer as Project>::Assoc<'a>))
//
// with
//
// ((u32, fn(&'a u32)))
//
// Equating `&'a u32` with `<u32 as Project>::Assoc<'a>` failed, even
// though these types are equal modulo normalization.
impl<T: Project> Trait for (T, for<'a> fn(<T as Project>::Assoc<'a>)) {}
impl<'a> Trait for (u32, fn(&'a u32)) {}
//[next]~^ ERROR conflicting implementations of trait `Trait` for type `(u32, for<'a> fn(&'a u32))`
```
A crater run did not discover any breakage due to this change.
#### Unknowable candidates for higher ranked trait goals
This avoids an unsoundness by attempting to normalize in `trait_ref_is_knowable`, fixing https://github.com/rust-lang/rust/issues/114061. This is a side-effect of supporting lazy normalization, as that forces us to attempt to normalize when checking whether a `TraitRef` is knowable: [source](47dd709bed/compiler/rustc_trait_selection/src/solve/assembly/mod.rs (L754-L764)).
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
trait IsUnit {}
impl IsUnit for () {}
pub trait WithAssoc<'a> {
type Assoc;
}
// We considered `for<'a> <T as WithAssoc<'a>>::Assoc: IsUnit`
// to be knowable, even though the projection is ambiguous.
pub trait Trait {}
impl<T> Trait for T
where
T: 'static,
for<'a> T: WithAssoc<'a>,
for<'a> <T as WithAssoc<'a>>::Assoc: IsUnit,
{
}
impl<T> Trait for Box<T> {}
//[next]~^ ERROR conflicting implementations of trait `Trait`
```
The two impls of `Trait` overlap given the following downstream crate:
```rust
use dep::*;
struct Local;
impl WithAssoc<'_> for Box<Local> {
type Assoc = ();
}
```
There a similar coherence unsoundness caused by our handling of aliases which is fixed separately in https://github.com/rust-lang/rust/pull/117164.
This change breaks the [`derive-visitor`](https://crates.io/crates/derive-visitor) crate. I have opened an issue in that repo: nikis05/derive-visitor#16.
### Evaluating goals to a fixpoint and applying inference constraints
In the old implementation of the implicit-negative check, each obligation is [checked separately without applying its inference constraints](bea5bebf3d/compiler/rustc_trait_selection/src/traits/coherence.rs (L323-L338)). The new solver instead [uses a `FulfillmentCtxt`](bea5bebf3d/compiler/rustc_trait_selection/src/traits/coherence.rs (L315-L321)) for this, which evaluates all obligations in a loop until there's no further inference progress.
This is necessary for backwards compatibility as we do not eagerly normalize with the new solver, resulting in constraints from normalization to only get applied by evaluating a separate obligation. This also allows more code to compile:
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
trait Mirror {
type Assoc;
}
impl<T> Mirror for T {
type Assoc = T;
}
trait Foo {}
trait Bar {}
// The self type starts out as `?0` but is constrained to `()`
// due to the where-clause below. Because `(): Bar` is known to
// not hold, we can prove the impls disjoint.
impl<T> Foo for T where (): Mirror<Assoc = T> {}
//[current]~^ ERROR conflicting implementations of trait `Foo` for type `()`
impl<T> Foo for T where T: Bar {}
fn main() {}
```
The old solver does not run nested goals to a fixpoint in evaluation. The new solver does do so, strengthening inference and improving the overlap check:
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
trait Foo {}
impl<T> Foo for (u8, T, T) {}
trait NotU8 {}
trait Bar {}
impl<T, U: NotU8> Bar for (T, T, U) {}
trait NeedsFixpoint {}
impl<T: Foo + Bar> NeedsFixpoint for T {}
impl NeedsFixpoint for (u8, u8, u8) {}
trait Overlap {}
impl<T: NeedsFixpoint> Overlap for T {}
impl<T, U: NotU8, V> Overlap for (T, U, V) {}
//[current]~^ ERROR conflicting implementations of trait `Foo`
```
### Breakage due to removal of incomplete candidate preference
Fixes#107887. In the old solver we incompletely prefer the builtin trait object impl over user defined impls. This can break inference guidance, inferring `?x` in `dyn Trait<u32>: Trait<?x>` to `u32`, even if an explicit impl of `Trait<u64>` also exists.
This caused coherence to incorrectly allow overlapping impls, resulting in ICEs and a theoretical unsoundness. See https://github.com/rust-lang/rust/issues/107887#issuecomment-1997261676. This compiles on stable but results in an overlap error with `-Znext-solver=coherence`:
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
struct W<T: ?Sized>(*const T);
trait Trait<T: ?Sized> {
type Assoc;
}
// This would trigger the check for overlap between automatic and custom impl.
// They actually don't overlap so an impl like this should remain possible
// forever.
//
// impl Trait<u64> for dyn Trait<u32> {}
trait Indirect {}
impl Indirect for dyn Trait<u32, Assoc = ()> {}
impl<T: Indirect + ?Sized> Trait<u64> for T {
type Assoc = ();
}
// Incomplete impl where `dyn Trait<u32>: Trait<_>` does not hold, but
// `dyn Trait<u32>: Trait<u64>` does.
trait EvaluateHack<U: ?Sized> {}
impl<T: ?Sized, U: ?Sized> EvaluateHack<W<U>> for T
where
T: Trait<U, Assoc = ()>, // incompletely constrains `_` to `u32`
U: IsU64,
T: Trait<U, Assoc = ()>, // incompletely constrains `_` to `u32`
{
}
trait IsU64 {}
impl IsU64 for u64 {}
trait Overlap<U: ?Sized> {
type Assoc: Default;
}
impl<T: ?Sized + EvaluateHack<W<U>>, U: ?Sized> Overlap<U> for T {
type Assoc = Box<u32>;
}
impl<U: ?Sized> Overlap<U> for dyn Trait<u32, Assoc = ()> {
//[next]~^ ERROR conflicting implementations of trait `Overlap<_>`
type Assoc = usize;
}
```
### Considering region outlives bounds in the `leak_check`
For details on the `leak_check`, see the FCP proposal #119820.[^leak_check]
[^leak_check]: which should get moved to the dev-guide :3
In both coherence and during candidate selection, the `leak_check` relies on the region constraints added in `evaluate`. It therefore currently does not register outlives obligations: [source](ccb1415eac/compiler/rustc_trait_selection/src/traits/select/mod.rs (L792-L810)). This was likely done as a performance optimization without considering its impact on the `leak_check`. This is the case as in the old solver, *evaluatation* and *fulfillment* are split, with evaluation being responsible for candidate selection and fulfillment actually registering all the constraints.
This split does not exist with the new solver. The `leak_check` can therefore eagerly detect errors caused by region outlives obligations. This improves both coherence itself and candidate selection:
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
trait LeakErr<'a, 'b> {}
// Using this impl adds an `'b: 'a` bound which results
// in a higher-ranked region error. This bound has been
// previously ignored but is now considered.
impl<'a, 'b: 'a> LeakErr<'a, 'b> for () {}
trait NoOverlapDir<'a> {}
impl<'a, T: for<'b> LeakErr<'a, 'b>> NoOverlapDir<'a> for T {}
impl<'a> NoOverlapDir<'a> for () {}
//[current]~^ ERROR conflicting implementations of trait `NoOverlapDir<'_>`
// --------------------------------------
// necessary to avoid coherence unknowable candidates
struct W<T>(T);
trait GuidesSelection<'a, U> {}
impl<'a, T: for<'b> LeakErr<'a, 'b>> GuidesSelection<'a, W<u32>> for T {}
impl<'a, T> GuidesSelection<'a, W<u8>> for T {}
trait NotImplementedByU8 {}
trait NoOverlapInd<'a, U> {}
impl<'a, T: GuidesSelection<'a, W<U>>, U> NoOverlapInd<'a, U> for T {}
impl<'a, U: NotImplementedByU8> NoOverlapInd<'a, U> for () {}
//[current]~^ conflicting implementations of trait `NoOverlapInd<'_, _>`
```
### Removal of `fn match_fresh_trait_refs`
The old solver tries to [eagerly detect unbounded recursion](b14fd2359f/compiler/rustc_trait_selection/src/traits/select/mod.rs (L1196-L1211)), forcing the affected goals to be ambiguous. This check is only an approximation and has not been added to the new solver.
The check is not necessary in the new solver and it would be problematic for caching. As it depends on all goals currently on the stack, using a global cache entry would have to always make sure that doing so does not circumvent this check.
This changes some goals to error - or succeed - instead of failing with ambiguity. This allows more code to compile:
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
// Need to use this local wrapper for the impls to be fully
// knowable as unknowable candidate result in ambiguity.
struct Local<T>(T);
trait Trait<U> {}
// This impl does not hold, but is ambiguous in the old
// solver due to its overflow approximation.
impl<U> Trait<U> for Local<u32> where Local<u16>: Trait<U> {}
// This impl holds.
impl Trait<Local<()>> for Local<u8> {}
// In the old solver, `Local<?t>: Trait<Local<?u>>` is ambiguous,
// resulting in `Local<?u>: NoImpl`, also being ambiguous.
//
// In the new solver the first impl does not apply, constraining
// `?u` to `Local<()>`, causing `Local<()>: NoImpl` to error.
trait Indirect<T> {}
impl<T, U> Indirect<U> for T
where
T: Trait<U>,
U: NoImpl
{}
// Not implemented for `Local<()>`
trait NoImpl {}
impl NoImpl for Local<u8> {}
impl NoImpl for Local<u16> {}
// `Local<?t>: Indirect<Local<?u>>` cannot hold, so
// these impls do not overlap.
trait NoOverlap<U> {}
impl<T: Indirect<U>, U> NoOverlap<U> for T {}
impl<T, U> NoOverlap<Local<U>> for Local<T> {}
//~^ ERROR conflicting implementations of trait `NoOverlap<Local<_>>`
```
### Non-fatal overflow
The old solver immediately emits a fatal error when hitting the recursion limit. The new solver instead returns overflow. This both allows more code to compile and is results in performance and potential future compatability issues.
Non-fatal overflow is generally desirable. With fatal overflow, changing the order in which we evaluate nested goals easily causes breakage if we have goal which errors and one which overflows. It is also required to prevent breakage due to the removal of `fn match_fresh_trait_refs`, e.g. [in `typenum`](https://github.com/rust-lang/trait-system-refactor-initiative/issues/73).
#### Enabling more code to compile
In the below example, the old solver first tried to prove an overflowing goal, resulting in a fatal error. The new solver instead returns ambiguity due to overflow for that goal, causing the implicit negative overlap check to succeed as `Box<u32>: NotImplemented` does not hold.
```rust
// revisions: current next
//[next] compile-flags: -Znext-solver=coherence
//[current] ERROR overflow evaluating the requirement
trait Indirect<T> {}
impl<T: Overflow<()>> Indirect<T> for () {}
trait Overflow<U> {}
impl<T, U> Overflow<U> for Box<T>
where
U: Indirect<Box<Box<T>>>,
{}
trait NotImplemented {}
trait Trait<U> {}
impl<T, U> Trait<U> for T
where
// T: NotImplemented, // causes old solver to succeed
U: Indirect<T>,
T: NotImplemented,
{}
impl Trait<()> for Box<u32> {}
```
#### Avoiding hangs with non-fatal overflow
Simply returning ambiguity when reaching the recursion limit can very easily result in hangs, e.g.
```rust
trait Recur {}
impl<T, U> Recur for ((T, U), (U, T))
where
(T, U): Recur,
(U, T): Recur,
{}
trait NotImplemented {}
impl<T: NotImplemented> Recur for T {}
```
This can happen quite frequently as it's easy to have exponential blowup due to multiple nested goals at each step. As the trait solver is depth-first, this immediately caused a fatal overflow error in the old solver. In the new solver we have to handle the whole proof tree instead, which can very easily hang.
To avoid this we restrict the recursion depth after hitting the recursion limit for the first time. We also **ignore all inference constraints from goals resulting in overflow**. This is mostly backwards compatible as any overflow in the old solver resulted in a fatal error.
### sidenote about normalization
We return ambiguous nested goals of `NormalizesTo` goals to the caller and ignore their impact when computing the `Certainty` of the current goal. See the [normalization chapter](https://rustc-dev-guide.rust-lang.org/solve/normalization.html) for more details.This means we apply constraints resulting from other nested goals and from equating the impl header when normalizing, even if a nested goal results in overflow. This is necessary to avoid breaking the following example:
```rust
trait Trait {
type Assoc;
}
struct W<T: ?Sized>(*mut T);
impl<T: ?Sized> Trait for W<W<T>>
where
W<T>: Trait,
{
type Assoc = ();
}
// `W<?t>: Trait<Assoc = u32>` does not hold as
// `Assoc` gets normalized to `()`. However, proving
// the where-bounds of the impl results in overflow.
//
// For this to continue to compile we must not discard
// constraints from normalizing associated types.
trait NoOverlap {}
impl<T: Trait<Assoc = u32>> NoOverlap for T {}
impl<T: ?Sized> NoOverlap for W<T> {}
```
#### Future compatability concerns
Non-fatal overflow results in some unfortunate future compatability concerns. Changing the approach to avoid more hangs by more strongly penalizing overflow can cause breakage as we either drop constraints or ignore candidates necessary to successfully compile. Weakening the overflow penalities instead allows more code to compile and strengthens inference while potentially causing more code to hang.
While the current approach is not perfect, we believe it to be good enough. We believe it to apply the necessary inference constraints to avoid breakage and expect there to not be any desirable patterns broken by our current penalities. Similarly we believe the current constraints to avoid most accidental hangs. Ignoring constraints of overflowing goals is especially useful, as it may allow major future optimizations to our overflow handling. See [this summary](https://hackmd.io/ATf4hN0NRY-w2LIVgeFsVg) and the linked documents in case you want to know more.
### changes to performance
In general, trait solving during coherence checking is not significant for performance. Enabling the next-generation trait solver in coherence does not impact our compile time benchmarks. We are still unable to compile the benchmark suite when fully enabling the new trait solver.
There are rare cases where the new solver has significantly worse performance due to non-fatal overflow, its reliance on fixpoint algorithms and the removal of the `fn match_fresh_trait_refs` approximation. We encountered such issues in [`typenum`](https://crates.io/crates/typenum) and believe it should be [pretty much as bad as it can get](https://github.com/rust-lang/trait-system-refactor-initiative/issues/73).
Due to an improved structure and far better caching, we believe that there is a lot of room for improvement and that the new solver will outperform the existing implementation in nearly all cases, sometimes significantly. We have not yet spent any time micro-optimizing the implementation and have many unimplemented major improvements, such as fast-paths for trivial goals.
### Unstable features
#### Unsupported unstable features
The new solver currently does not support all unstable features, most notably `#![feature(generic_const_exprs)]`, `#![feature(associated_const_equality)]` and `#![feature(adt_const_params)]` are not yet fully supported in the new solver. We are confident that supporting them is possible, but did not consider this to be a priority. This stabilization introduces new ICE when using these features in impl headers.
#### fixes to `#![feature(specialization)]`
- fixes#105782
- fixes#118987
#### fixes to `#![feature(type_alias_impl_trait)]`
- fixes#119272
- https://github.com/rust-lang/rust/issues/105787#issuecomment-1750112388
- fixes#124207
### Important changes since the original FCP
https://github.com/rust-lang/rust/pull/127574 changes the coherence unknowable candidate to only apply if all the super trait bounds may hold. This allows more code to compile and fixes a regression in `pyella`
https://github.com/rust-lang/rust/pull/130617 bails with ambiguity if the query response would contain too many non-region inference variables. This should only be triggered in case the result contains a lot of ambiguous aliases in which case further constraining the goal should resolve this.
https://github.com/rust-lang/rust/pull/130821 adds caching to a lot of type folders, which is necessary to handle exponentially large types and handles the hang in `nalgebra` together with #130617.
## This does not stabilize the whole solver
While this stabilizes the use of the new solver in coherence checking, there are many parts of the solver which will remain fully unstable. We may still adapt these areas while working towards stabilizing the new solver everywhere. We are confident that we are able to do so without negatively impacting coherence.
### goals with a non-empty `ParamEnv`
Coherence always uses an empty environment. We therefore do not depend on the behavior of `AliasBound` and `ParamEnv` candidates. We only stabilizes the behavior of user-defined and builtin implementations of traits. There are still many open questions there.
### opaque types in the defining scope
The handling of opaque types - `impl Trait` - in both the new and old solver is still not fully figured out. Luckily this can be ignored for now. While opaque types are reachable during coherence checking by using `impl_trait_in_associated_types`, the behavior during coherence is separate and self-contained. The old and new solver fully agree here.
### normalization is hard
This stabilizes that we equate associated types involving bound variables using deferred-alias-equality. We also stop eagerly normalizing in coherence, which should not have any user-facing impact.
We do not stabilize the normalization behavior outside of coherence, e.g. we currently deeply normalize all types during writeback with the new solver. This may change going forward
### how to replace `select` from the old solver
We sometimes depend on getting a single `impl` for a given trait bound, e.g. when resolving a concrete method for codegen/CTFE. We do not depend on this during coherence, so the exact approach here can still be freely changed going forward.
## Acknowledgements
This work would not have been possible without `@compiler-errors.` He implemented large chunks of the solver himself but also and did a lot of testing and experimentation, eagerly discovering multiple issues which had a significant impact on our approach. `@BoxyUwU` has also done some amazing work on the solver. Thank you for the endless hours of discussion resulting in the current approach. Especially the way aliases are handled has gone through multiple revisions to get to its current state.
There were also many contributions from - and discussions with - other members of the community and the rest of `@rust-lang/types.` This solver builds upon previous improvements to the compiler, as well as lessons learned from `chalk` and `a-mir-formality`. Getting to this point would not have been possible without that and I am incredibly thankful to everyone involved. See the [list of relevant PRs](https://github.com/rust-lang/rust/pulls?q=is%3Apr+is%3Amerged+label%3AWG-trait-system-refactor+-label%3Arollup+closed%3A%3C2024-03-22+).
