Empowering everyone to build reliable and efficient software.
Go to file
bors 43f4f2a3b1 Auto merge of - lcnr:leak-check-2, r=jackh726
instantiate higher ranked goals outside of candidate selection

This PR modifies `evaluate` to more eagerly instantiate higher-ranked goals, preventing the `leak_check` during candidate selection from detecting placeholder errors involving that binder.

For a general background regarding higher-ranked region solving and the leak check, see https://hackmd.io/qd9Wp03cQVy06yOLnro2Kg.

> The first is something called the **leak check**. You can think of it as a "quick and dirty" approximation for the region check, which will come later. The leak check detects some kinds of errors early, essentially deciding between "this set of outlives constraints are guaranteed to result in an error eventually" or "this set of outlives constraints may be solvable".

## The ideal future

We would like to end up with the following idealized design to handle universal binders:
```rust
fn enter_forall<'tcx, T, R>(
    forall: Binder<'tcx, T>,
    f: impl FnOnce(T) -> R,
) -> R {
    let new_universe = infcx.increment_universe_index();
    let value = instantiate_binder_with_placeholders_in(new_universe, forall);

    let result = f(value);

    eagerly_handle_higher_ranked_region_constraints_in(new_universe);
    infcx.decrement_universe_index();

    assert!(!result.has_placeholders_in_or_above(new_universe));
    result
}
```

That is, when universally instantiating a binder, anything using the placeholders has to happen inside of a limited scope (the closure `f`). After this closure has completed, all constraints involving placeholders are known.

We then handle any *external constraints* which name these placeholders. We destructure `TypeOutlives` constraints involving placeholders and eagerly handle any region constraints involving these placeholders. We do not return anything mentioning the placeholders created inside of this function to the caller.

Being able to eagerly handle *all* region constraints involving placeholders will be difficult due to complex `TypeOutlives` constraints, involving inference variables or alias types, and higher ranked implied bounds. The exact issues and possible solutions are out of scope of this FCP.

#### How does the leak check fit into this

The `leak_check` is an underapproximation of `eagerly_handle_higher_ranked_region_constraints_in`. It detects some kinds of errors involving placeholders from `new_universe`, but not all of them.

It only looks at region outlives constraints, ignoring `TypeOutlives`, and checks whether one of the following two conditions are met for **placeholders in or above `new_universe`**, in which case it results in an error:
- `'!p1: '!p2` a placeholder `'!p2` outlives a different placeholder `'!p1`
- `'!p1: '?2` an inference variable `'?2` outlives a placeholder `'!p1` *which it cannot name*

It does not handle all higher ranked region constraints, so we still return constraints involving placeholders from `new_universe` which are then (re)checked by `lexical_region_resolve` or MIR borrowck.

As we check higher ranked constraints in the full regionck anyways, the `leak_check` is not soundness critical. It's current only purpose is to move some higher ranked region errors earlier, enabling it to guide type inference and trait solving. Adding additional uses of the `leak_check` in the future would only strengthen inference and is therefore not breaking.

## Where do we use currently use the leak check

The `leak_check` is currently used in two places:

Coherence does not use a proper regionck, only relying on the `leak_check` called [at the end of the implicit negative overlap check](8b94152af6/compiler/rustc_trait_selection/src/traits/coherence.rs (L235-L238)). During coherence all parameters are instantiated with inference variables, so the only possible region errors are higher-ranked. We currently also sometimes make guesses when destructuring `TypeOutlives` constraints which can theoretically result in incorrect errors. This could result in overlapping impls.

We also use the `leak_check` [at the end of `fn evaluation_probe`](8b94152af6/compiler/rustc_trait_selection/src/traits/select/mod.rs (L607-L610)). This function is used during candidate assembly for `Trait` goals. Most notably we use [inside of `evaluate_candidate` during winnowing](0e4243538b/compiler/rustc_trait_selection/src/traits/select/mod.rs (L491-L502)). Conceptionally, it is as if we compute each candidate in a separate `enter_forall`.

## The current use in `fn evaluation_probe` is undesirable

Because we only instantiate a higher-ranked goal once inside of `fn evaluation_probe`, errors involving placeholders from that binder can impact selection. This results in inconsistent behavior ([playground](
*[playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=dac60ebdd517201788899ffa77364831)*)):

```rust
trait Leak<'a> {}
impl Leak<'_>      for Box<u32> {}
impl Leak<'static> for Box<u16> {}

fn impls_leak<T: for<'a> Leak<'a>>() {}

trait IndirectLeak<'a> {}
impl<'a, T: Leak<'a>> IndirectLeak<'a> for T {}
fn impls_indirect_leak<T: for<'a> IndirectLeak<'a>>() {}

fn main() {
    // ok
    //
    // The `Box<u16>` impls fails the leak check,
    // meaning that we apply the `Box<u32>` impl.
    impls_leak::<Box<_>>();

    // error: type annotations needed
    //
    // While the `Box<u16>` impl would fail the leak check
    // we have already instantiated the binder while applying
    // the generic `IndirectLeak` impl, so during candidate
    // selection of `Leak` we do not detect the placeholder error.
    // Evaluation of `Box<_>: Leak<'!a>` is therefore ambiguous,
    // resulting in `for<'a> Box<_>: Leak<'a>` also being ambiguous.
    impls_indirect_leak::<Box<_>>();
}
```

We generally prefer `where`-bounds over implementations during candidate selection, both for [trait goals](11f32b73e0/compiler/rustc_trait_selection/src/traits/select/mod.rs (L1863-L1887)) and during [normalization](11f32b73e0/compiler/rustc_trait_selection/src/traits/project.rs (L184-L198)). However, we currently **do not** use the `leak_check` during candidate assembly in normalizing. This can result in inconsistent behavior:
```rust
trait Trait<'a> {
    type Assoc;
}
impl<'a, T> Trait<'a> for T {
    type Assoc = usize;
}

fn trait_bound<T: for<'a> Trait<'a>>() {}
fn projection_bound<T: for<'a> Trait<'a, Assoc = usize>>() {}

// A function with a trivial where-bound which is more
// restrictive than the impl.
fn function<T: Trait<'static, Assoc = usize>>() {
    // ok
    //
    // Proving `for<'a> T: Trait<'a>` using the where-bound results
    // in a leak check failure, so we use the more general impl,
    // causing this to succeed.
    trait_bound::<T>();

    // error
    //
    // Proving the `Projection` goal `for<'a> T: Trait<'a, Assoc = usize>`
    // does not use the leak check when trying the where-bound, causing us
    // to prefer it over the impl, resulting in a placeholder error.
    projection_bound::<T>();

    // error
    //
    // Trying to normalize the type `for<'a> fn(<T as Trait<'a>>::Assoc)`
    // only gets to `<T as Trait<'a>>::Assoc` once `'a` has been already
    // instantiated, causing us to prefer the where-bound over the impl
    // resulting in a placeholder error. Even if were were to also use the
    // leak check during candidate selection for normalization, this
    // case would still not compile.
    let _higher_ranked_norm: for<'a> fn(<T as Trait<'a>>::Assoc) = |_| ();
}
```

This is also likely to be more performant. It enables more caching in the new trait solver by simply [recursively calling the canonical query][new solver] after instantiating the higher-ranked goal.

It is also unclear how to add the leak check to normalization in the new solver. To handle https://github.com/rust-lang/trait-system-refactor-initiative/issues/1 `Projection` goals are implemented via `AliasRelate`. This again means that we instantiate the binder before ever normalizing any alias. Even if we were to avoid this, we lose the ability to [cache normalization by itself, ignoring the expected `term`](5bd5d214ef/compiler/rustc_trait_selection/src/solve/normalizes_to/mod.rs (L34-L49)). We cannot replace the `term` with an inference variable before instantiating the binder, as otherwise `for<'a> T: Trait<Assoc<'a> = &'a ()>` breaks. If we only replace the term after instantiating the binder, we cannot easily evaluate the goal in a separate context, as [we'd then lose the information necessary for the leak check](11f32b73e0/compiler/rustc_next_trait_solver/src/canonicalizer.rs (L230-L232)). Adding this information to the canonical input also seems non-trivial.

## Proposed solution

I propose to instantiate the binder outside of candidate assembly, causing placeholders from higher-ranked goals to get ignored while selecting their candidate. This mostly[^1] matches the [current behavior of the new solver][new solver]. The impact of this change is therefore as follows:

```rust
trait Leak<'a> {}
impl Leak<'_>      for Box<u32> {}
impl Leak<'static> for Box<u16> {}

fn impls_leak<T: for<'a> Leak<'a>>() {}

trait IndirectLeak<'a> {}
impl<'a, T: Leak<'a>> IndirectLeak<'a> for T {}
fn impls_indirect_leak<T: for<'a> IndirectLeak<'a>>() {}

fn guide_selection() {
    // ok -> ambiguous
    impls_leak::<Box<_>>();

    // ambiguous
    impls_indirect_leak::<Box<_>>();
}

trait Trait<'a> {
    type Assoc;
}
impl<'a, T> Trait<'a> for T {
    type Assoc = usize;
}

fn trait_bound<T: for<'a> Trait<'a>>() {}
fn projection_bound<T: for<'a> Trait<'a, Assoc = usize>>() {}

// A function which a trivial where-bound which is more
// restrictive than the impl.
fn function<T: Trait<'static, Assoc = usize>>() {
    // ok -> error
    trait_bound::<T>();

    // error
    projection_bound::<T>();

    // error
    let _higher_ranked_norm: for<'a> fn(<T as Trait<'a>>::Assoc) = |_| ();
}
```

This does not change the behavior if candidates have higher ranked nested goals, as in this case the `leak_check` causes the nested goal to result in an error ([playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=a74c25300b23db9022226de99d8a2fa6)):
```rust
trait LeakCheckFailure<'a> {}
impl LeakCheckFailure<'static> for () {}

trait Trait<T> {}
impl Trait<u32> for () where for<'a> (): LeakCheckFailure<'a> {}
impl Trait<u16> for () {}
fn impls_trait<T: Trait<U>, U>() {}
fn main() {
    // ok
    //
    // It does not matter whether candidate assembly
    // considers the placeholders from higher-ranked goal.
    //
    // Either `for<'a> (): LeakCheckFailure<'a>` has no
    // applicable candidate or it has a single applicable candidate
    // when then later results in an error. This allows us to
    // infer `U` to `u16`.
    impls_trait::<(), _>()
}
```

## Impact on existing crates

This is a **breaking change**. [A crater run](https://github.com/rust-lang/rust/pull/119820#issuecomment-1926862174) found 17 regressed crates with 7 root causes.

For a full analysis of all affected crates, see https://gist.github.com/lcnr/7c1c652f30567048ea240554a36ed95c.

---

I believe this breakage to be acceptable and would merge this change. I am confident that the new position of the leak check matches our idealized future and cannot envision any other consistent alternative. Where possible, I intend to open PRs fixing/avoiding the regressions before landing this PR.

I originally intended to remove the `coherence_leak_check` lint in the same PR. However, while I am confident in the *position* of the leak check, deciding on its exact behavior is left as future work, cc . This PR therefore only moves the leak check while keeping the lint when relying on it in coherence.

[new solver]: https://github.com/rust-lang/rust/blob/master/compiler/rustc_trait_selection/src/solve/eval_ctxt/mod.rs#L479-L484

[^1]: the new solver has a separate cause of inconsistent behavior rn https://github.com/rust-lang/trait-system-refactor-initiative/issues/53#issuecomment-1914310171

r? `@nikomatsakis`
2024-04-04 04:36:12 +00:00
.github Revert cargo update changes 2024-03-26 15:36:12 +00:00
.reuse std: move Once implementations to sys 2024-03-12 15:41:06 +01:00
compiler Auto merge of - lcnr:leak-check-2, r=jackh726 2024-04-04 04:36:12 +00:00
library Rollup merge of - devnexen:dfbsd_build_fix, r=jhpratt 2024-04-03 20:17:04 -04:00
LICENSES Add missing CC-BY-SA-4.0. 2023-11-27 11:03:53 +00:00
src Auto merge of - lcnr:leak-check-2, r=jackh726 2024-04-04 04:36:12 +00:00
tests Auto merge of - lcnr:leak-check-2, r=jackh726 2024-04-04 04:36:12 +00:00
.editorconfig Only use max_line_length = 100 for *.rs 2023-07-10 15:18:36 -07:00
.git-blame-ignore-revs Ignore compiletest test directive migration commits 2024-02-22 18:55:02 +00:00
.gitattributes Rename config.toml.example to config.example.toml 2023-03-11 14:10:00 -08:00
.gitignore don't globally ignore rustc-ice files 2023-09-16 09:44:44 +02:00
.gitmodules Update to LLVM 18 2024-02-13 10:33:40 +01:00
.mailmap Add my former address to .mailmap 2024-03-25 18:48:54 -07:00
Cargo.lock Rollup merge of - ferrocene:hoverbear/bump-sysinfo-to-0.30.8, r=Nilstrieb 2024-04-02 21:22:04 +02:00
Cargo.toml Add a test 2024-03-20 23:36:05 -04:00
CODE_OF_CONDUCT.md Remove the code of conduct; instead link https://www.rust-lang.org/conduct.html 2019-10-05 22:55:19 +02:00
config.example.toml use more accurate terminology 2024-03-19 20:32:55 +02:00
configure Ensure ./configure works when configure.py path contains spaces 2024-02-16 18:57:22 +00:00
CONTRIBUTING.md fix: Update CONTRIBUTING.md recommend -> recommended 2023-11-16 23:57:09 +05:30
COPYRIGHT Update COPYRIGHT file 2022-10-30 10:23:14 -04:00
INSTALL.md chore: fix some comments 2024-03-27 22:32:53 +08:00
LICENSE-APACHE Remove appendix from LICENCE-APACHE 2019-12-30 14:25:53 +00:00
LICENSE-MIT LICENSE-MIT: Remove inaccurate (misattributed) copyright notice 2017-07-26 16:51:58 -07:00
README.md Use SVG logos in the README.md. 2024-04-03 19:48:20 +02:00
RELEASES.md Forward port 1.77.1 release notes 2024-03-29 10:44:19 +03:00
rust-bors.toml Increase timeout for new bors bot 2024-03-13 08:31:07 +01:00
rustfmt.toml Remove exception for deleted file from rustfmt.toml 2024-03-08 20:46:33 +00:00
triagebot.toml beep boop 2024-03-31 20:18:37 +01:00
x Make x capable of resolving symlinks 2023-10-14 17:53:33 +03:00
x.ps1 use & instead of start-process in x.ps1 2023-12-09 09:46:16 -05:00
x.py Fix recent python linting errors 2023-08-02 04:40:28 -04:00

This is the main source code repository for Rust. It contains the compiler, standard library, and documentation.

Why Rust?

  • Performance: Fast and memory-efficient, suitable for critical services, embedded devices, and easily integrate with other languages.

  • Reliability: Our rich type system and ownership model ensure memory and thread safety, reducing bugs at compile-time.

  • Productivity: Comprehensive documentation, a compiler committed to providing great diagnostics, and advanced tooling including package manager and build tool (Cargo), auto-formatter (rustfmt), linter (Clippy) and editor support (rust-analyzer).

Quick Start

Read "Installation" from The Book.

Installing from Source

If you really want to install from source (though this is not recommended), see INSTALL.md.

Getting Help

See https://www.rust-lang.org/community for a list of chat platforms and forums.

Contributing

See CONTRIBUTING.md.

License

Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.

Trademark

The Rust Foundation owns and protects the Rust and Cargo trademarks and logos (the "Rust Trademarks").

If you want to use these names or brands, please read the media guide.

Third-party logos may be subject to third-party copyrights and trademarks. See Licenses for details.