Parenthesize break values containing leading label
The AST pretty printer previously produced invalid syntax in the case of `break` expressions with a value that begins with a loop or block label.
```rust
macro_rules! expr {
($e:expr) => {
$e
};
}
fn main() {
loop {
break expr!('a: loop { break 'a 1; } + 1);
};
}
```
`rustc -Zunpretty=expanded main.rs `:
```console
#![feature(prelude_import)]
#![no_std]
#[prelude_import]
use ::std::prelude::rust_2015::*;
#[macro_use]
extern crate std;
macro_rules! expr { ($e:expr) => { $e }; }
fn main() { loop { break 'a: loop { break 'a 1; } + 1; }; }
```
The expanded code is not valid Rust syntax. Printing invalid syntax is bad because it blocks `cargo expand` from being able to format the output as Rust syntax using rustfmt.
```console
error: parentheses are required around this expression to avoid confusion with a labeled break expression
--> <anon>:9:26
|
9 | fn main() { loop { break 'a: loop { break 'a 1; } + 1; }; }
| ^^^^^^^^^^^^^^^^^^^^^^^^
|
help: wrap the expression in parentheses
|
9 | fn main() { loop { break ('a: loop { break 'a 1; }) + 1; }; }
| + +
```
This PR updates the AST pretty-printer to insert parentheses around the value of a `break` expression as required to avoid this edge case.
delegation: Implement glob delegation
Support delegating to all trait methods in one go.
Overriding globs with explicit definitions is also supported.
The implementation is generally based on the design from https://github.com/rust-lang/rfcs/pull/3530#issuecomment-2020869823, but unlike with list delegation in https://github.com/rust-lang/rust/pull/123413 we cannot expand glob delegation eagerly.
We have to enqueue it into the queue of unexpanded macros (most other macros are processed this way too), and then a glob delegation waits in that queue until its trait path is resolved, and enough code expands to generate the identifier list produced from the glob.
Glob delegation is only allowed in impls, and can only point to traits.
Supporting it in other places gives very little practical benefit, but significantly raises the implementation complexity.
Part of https://github.com/rust-lang/rust/issues/118212.
We already do this for a number of crates, e.g. `rustc_middle`,
`rustc_span`, `rustc_metadata`, `rustc_span`, `rustc_errors`.
For the ones we don't, in many cases the attributes are a mess.
- There is no consistency about order of attribute kinds (e.g.
`allow`/`deny`/`feature`).
- Within attribute kind groups (e.g. the `feature` attributes),
sometimes the order is alphabetical, and sometimes there is no
particular order.
- Sometimes the attributes of a particular kind aren't even grouped
all together, e.g. there might be a `feature`, then an `allow`, then
another `feature`.
This commit extends the existing sorting to all compiler crates,
increasing consistency. If any new attribute line is added there is now
only one place it can go -- no need for arbitrary decisions.
Exceptions:
- `rustc_log`, `rustc_next_trait_solver` and `rustc_type_ir_macros`,
because they have no crate attributes.
- `rustc_codegen_gcc`, because it's quasi-external to rustc (e.g. it's
ignored in `rustfmt.toml`).
Print `token::Interpolated` with token stream pretty printing.
This is a step towards removing `token::Interpolated` (#124141). It unavoidably changes the output of the `stringify!` macro, generally for the better.
r? `@petrochenkov`
Parse unsafe attributes
Initial parse implementation for #123757
This is the initial work to parse unsafe attributes, which is represented as an extra `unsafety` field in `MetaItem` and `AttrItem`. There's two areas in the code where it appears that parsing is done manually and not using the parser stuff, and I'm not sure how I'm supposed to thread the change there.
Instead of using AST pretty printing.
This is a step towards removing `token::Interpolated`, which will
eventually (in #124141) be replaced with a token stream within invisible
delimiters.
This changes (improves) the output of the `stringify!` macro in some
cases. This is allowed. As the `stringify!` docs say: "Note that the
expanded results of the input tokens may change in the future. You
should be careful if you rely on the output."
Test changes:
- tests/ui/macros/stringify.rs: this used to test both token stream
pretty printing and AST pretty printing via different ways of invoking
of `stringify!` (i.e. `$expr` vs `$tt`). But those two different
invocations now give the same result, which is a nice consistency
improvement. This removes the need for all the `c2*` macros. The AST
pretty printer now has more thorough testing thanks to #125236.
- tests/ui/proc-macro/*: minor improvements where small differences
between `INPUT (DISPLAY)` output and `DEEP-RE-COLLECTED (DISPLAY)`
output disappear.
Rename HIR `TypeBinding` to `AssocItemConstraint` and related cleanup
Rename `hir::TypeBinding` and `ast::AssocConstraint` to `AssocItemConstraint` and update all items and locals using the old terminology.
Motivation: The terminology *type binding* is extremely outdated. "Type bindings" not only include constraints on associated *types* but also on associated *constants* (feature `associated_const_equality`) and on RPITITs of associated *functions* (feature `return_type_notation`). Hence the word *item* in the new name. Furthermore, the word *binding* commonly refers to a mapping from a binder/identifier to a "value" for some definition of "value". Its use in "type binding" made sense when equality constraints (e.g., `AssocTy = Ty`) were the only kind of associated item constraint. Nowadays however, we also have *associated type bounds* (e.g., `AssocTy: Bound`) for which the term *binding* doesn't make sense.
---
Old terminology (HIR, rustdoc):
```
`TypeBinding`: (associated) type binding
├── `Constraint`: associated type bound
└── `Equality`: (associated) equality constraint (?)
├── `Ty`: (associated) type binding
└── `Const`: associated const equality (constraint)
```
Old terminology (AST, abbrev.):
```
`AssocConstraint`
├── `Bound`
└── `Equality`
├── `Ty`
└── `Const`
```
New terminology (AST, HIR, rustdoc):
```
`AssocItemConstraint`: associated item constraint
├── `Bound`: associated type bound
└── `Equality`: associated item equality constraint OR associated item binding (for short)
├── `Ty`: associated type equality constraint OR associated type binding (for short)
└── `Const`: associated const equality constraint OR associated const binding (for short)
```
r? compiler-errors
Remove `NtIdent` and `NtLifetime`
This is one part of the bigger "remove `Nonterminal` and `TokenKind::Interpolated`" change drafted in #114647. More details in the individual commit messages.
r? `@petrochenkov`
The extra span is now recorded in the new `TokenKind::NtIdent` and
`TokenKind::NtLifetime`. These both consist of a single token, and so
there's no operator precedence problems with inserting them directly
into the token stream.
The other way to do this would be to wrap the ident/lifetime in invisible
delimiters, but there's a lot of code that assumes an interpolated
ident/lifetime fits in a single token, and changing all that code to work with
invisible delimiters would have been a pain. (Maybe it could be done in a
follow-up.)
This change might not seem like much of a win, but it's a first step toward the
much bigger and long-desired removal of `Nonterminal` and
`TokenKind::Interpolated`. That change is big and complex enough that it's
worth doing this piece separately. (Indeed, this commit is based on part of a
late commit in #114647, a prior attempt at that big and complex change.)
Avoid clone in `Comments::next`
`Comments::next`, in `rustc_ast_pretty`, has this comment:
```
// FIXME: This shouldn't probably clone lmao
```
The obvious thing to try is to return `Option<&Comment>` instead of `Option<Comment>`. But that leads to multiple borrows all over the place, because `Comments` must be borrowed from `PrintState` and then processed by `&mut self` methods within `PrintState`.
This PR instead rearranges things so that comments are consumed as they are used, preserving the `Option<Comment>` return type without requiring any cloning.
r? `@compiler-errors`
This span records the declaration of the metavariable in the LHS of the macro.
It's used in a couple of error messages. Unfortunately, it gets in the way of
the long-term goal of removing `TokenKind::Interpolated`. So this commit
removes it, which degrades a couple of (obscure) error messages but makes
things simpler and enables the next commit.
The current way of stepping through each comment in `Comments` is a bit
weird. There is a `Vec<Comments>` and a `current` index, which is fine.
The `Comments::next` method clones the current comment but doesn't
advance `current`; the advancing instead happens in `print_comment`,
which is where each cloned comment is actually finally used (or not, in
some cases, if the comment fails to satisfy a predicate).
This commit makes things more iterator-like:
- `Comments::next` now advances `current` instead of `print_comment`.
- `Comments::peek` is added so you can inspect a comment and check a
predicate without consuming it.
- This requires splitting `PrintState::comments` into immutable and
mutable versions. The commit also moves the ref inside the `Option` of
the return type, to save callers from having to use `as_ref`/`as_mut`.
- It also requires adding `PrintState::peek_comment` alongside the
existing `PrintState::next_comment`. (The lifetimes in the signature
of `peek_comment` ended up more complex than I expected.)
We now have a neat separation between consuming (`next`) and
non-consuming (`peek`) uses of each comment. As well as being clearer,
this will facilitate the next commit that avoids unnecessary cloning.
I didn't figure out how to reach this condition with `expr` containing
`ExprKind::MacCall`. All the approaches I tried ended up with the macro
call ending up in the `StmtKind::MacCall` case below instead.
In any case, from visual inspection this is a bugfix. If we do end up
with a `StmtKind::Expr` containing `ExprKind::MacCall` with brace
delimiter, it would not need ";" printed after it.
This commit by itself is supposed to have no effect on behavior. All of
the call sites are updated to preserve their previous behavior.
The behavior changes are in the commits that follow.
For each of these, we need to decide whether they need to be using
`expr_requires_semi_to_be_stmt`, or `expr_requires_comma_to_be_match_arm`,
which are supposed to be 2 different behaviors. Previously they were
conflated into one, causing either too much or too little
parenthesization.
Remove braces when fixing a nested use tree into a single item
[Back in 2019](https://github.com/rust-lang/rust/pull/56645) I added rustfix support for the `unused_imports` lint, to automatically remove them when running `cargo fix`. For the most part this worked great, but when removing all but one childs of a nested use tree it turned `use foo::{Unused, Used}` into `use foo::{Used}`. This is slightly annoying, because it then requires you to run `rustfmt` to get `use foo::Used`.
This PR automatically removes braces and the surrouding whitespace when all but one child of a nested use tree are unused. To get it done I had to add the span of the nested use tree to the AST, and refactor a bit the code I wrote back then.
A thing I noticed is, there doesn't seem to be any `//@ run-rustfix` test for fixing the `unused_imports` lint. I created a test in `tests/suggestions` (is that the right directory?) that for now tests just what I added in the PR. I can followup in a separate PR to add more tests for fixing `unused_lints`.
This PR is best reviewed commit-by-commit.
Add StaticForeignItem and use it on ForeignItemKind
This is in preparation for unsafe extern blocks that adds a safe variant for functions inside extern blocks.
r? `@oli-obk`
cc `@compiler-errors`
Implement syntax for `impl Trait` to specify its captures explicitly (`feature(precise_capturing)`)
Implements `impl use<'a, 'b, T, U> Sized` syntax that allows users to explicitly list the captured parameters for an opaque, rather than inferring it from the opaque's bounds (or capturing *all* lifetimes under 2024-edition capture rules). This allows us to exclude some implicit captures, so this syntax may be used as a migration strategy for changes due to #117587.
We represent this list of captured params as `PreciseCapturingArg` in AST and HIR, resolving them between `rustc_resolve` and `resolve_bound_vars`. Later on, we validate that the opaques only capture the parameters in this list.
We artificially limit the feature to *require* mentioning all type and const parameters, since we don't currently have support for non-lifetime bivariant generics. This can be relaxed in the future.
We also may need to limit this to require naming *all* lifetime parameters for RPITIT, since GATs have no variance. I have to investigate this. This can also be relaxed in the future.
r? `@oli-obk`
Tracking issue:
- https://github.com/rust-lang/rust/issues/123432
Cleanup: Rename `ModSep` to `PathSep`
`::` is usually referred to as the *path separator* (citation needed).
The existing name `ModSep` for *module separator* is a bit misleading since it in fact separates the segments of arbitrary path segments, not only ones resolving to modules. Let me just give a shout-out to associated items (`T::Assoc`, `<Ty as Trait>::function`) and enum variants (`Option::None`).
Motivation: Reduce friction for new contributors, prevent potential confusion.
cc `@petrochenkov`
r? nnethercote or compiler
Match ergonomics 2024: implement mutable by-reference bindings
Implements the mutable by-reference bindings portion of match ergonomics 2024 (#123076), with the `mut ref`/`mut ref mut` syntax, under feature gate `mut_ref`.
r? `@Nadrieril`
`@rustbot` label A-patterns A-edition-2024
Experimental feature postfix match
This has a basic experimental implementation for the RFC postfix match (rust-lang/rfcs#3295, #121618). [Liaison is](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Postfix.20Match.20Liaison/near/423301844) ```@scottmcm``` with the lang team's [experimental feature gate process](https://github.com/rust-lang/lang-team/blob/master/src/how_to/experiment.md).
This feature has had an RFC for a while, and there has been discussion on it for a while. It would probably be valuable to see it out in the field rather than continue discussing it. This feature also allows to see how popular postfix expressions like this are for the postfix macros RFC, as those will take more time to implement.
It is entirely implemented in the parser, so it should be relatively easy to remove if needed.
This PR is split in to 5 commits to ease review.
1. The implementation of the feature & gating.
2. Add a MatchKind field, fix uses, fix pretty.
3. Basic rustfmt impl, as rustfmt crashes upon seeing this syntax without a fix.
4. Add new MatchSource to HIR for Clippy & other HIR consumers
Add asm goto support to `asm!`
Tracking issue: #119364
This PR implements asm-goto support, using the syntax described in "future possibilities" section of [RFC2873](https://rust-lang.github.io/rfcs/2873-inline-asm.html#asm-goto).
Currently I have only implemented the `label` part, not the `fallthrough` part (i.e. fallthrough is implicit). This doesn't reduce the expressive though, since you can use label-break to get arbitrary control flow or simply set a value and rely on jump threading optimisation to get the desired control flow. I can add that later if deemed necessary.
r? ``@Amanieu``
cc ``@ojeda``
Add an ErrorGuaranteed to ast::TyKind::Err (attempt 2)
This makes it more like `hir::TyKind::Err`, and avoids a `has_errors` assertion in `LoweringContext::lower_ty_direct`.
r? ```@oli-obk```
This mostly works well, and eliminates a couple of delayed bugs.
One annoying thing is that we should really also add an
`ErrorGuaranteed` to `proc_macro::bridge::LitKind::Err`. But that's
difficult because `proc_macro` doesn't have access to `ErrorGuaranteed`,
so we have to fake it.
This makes it more like `hir::TyKind::Err`, and avoids a
`span_delayed_bug` call in `LoweringContext::lower_ty_direct`.
It also requires adding `ast::TyKind::Dummy`, now that
`ast::TyKind::Err` can't be used for that purpose in the absence of an
error emission.
There are a couple of cases that aren't as neat as I would have liked,
marked with `FIXME` comments.
Invert diagnostic lints.
That is, change `diagnostic_outside_of_impl` and `untranslatable_diagnostic` from `allow` to `deny`, because more than half of the compiler has been converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow` attributes, which proves that this change is warranted.
r? ````@davidtwco````
That is, change `diagnostic_outside_of_impl` and
`untranslatable_diagnostic` from `allow` to `deny`, because more than
half of the compiler has be converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow`
attributes, which proves that this change is warranted.
Further improve `space_between`
`space_between` is used by `print_tts` to decide when spaces should be put between tokens. This PR improves it in two ways:
- avoid unnecessary spaces before semicolons, and
- don't omit some necessary spaces before/after some punctuation symbols.
r? `@petrochenkov`
There are a number of cases where we erroneously omit the space between
two tokens, all involving an exception to a more general case. The
affected tokens are `$`, `!`, `.`, `,`, and `let` followed by a
parenthesis.
This fixes a lot of FIXME comments.
Fix parenthesization of subexprs containing statement boundary
This PR fixes a multitude of false negatives and false positives in the AST pretty printer's parenthesis insertion related to statement boundaries — statements which terminate unexpectedly early if there aren't parentheses.
Without this fix, the AST pretty printer (including both `stringify!` and `rustc -Zunpretty=expanded`) is prone to producing output which is not syntactically valid Rust. Invalid output is problematic because it means Rustfmt is unable to parse the output of `cargo expand`, for example, causing friction by forcing someone trying to debug a macro into reading poorly formatted code.
I believe the set of bugs fixed in this PR account for the most prevalent reason that `cargo expand` produces invalid output in real-world usage.
Fixes#98790.
## False negatives
The following is a correct program — `cargo check` succeeds.
```rust
macro_rules! m {
($e:expr) => {
match () { _ => $e }
};
}
fn main() {
m!({ 1 } - 1);
}
```
But `rustc -Zunpretty=expanded main.rs` produces output that is invalid Rust syntax, because parenthesization is needed and not being done by the pretty printer.
```rust
fn main() { match () { _ => { 1 } - 1, }; }
```
Piping this expanded code to rustfmt, it fails to parse.
```console
error: unexpected `,` in pattern
--> <stdin>:1:38
|
1 | fn main() { match () { _ => { 1 } - 1, }; }
| ^
|
help: try adding parentheses to match on a tuple...
|
1 | fn main() { match () { _ => { 1 } (- 1,) }; }
| + +
help: ...or a vertical bar to match on multiple alternatives
|
1 | fn main() { match () { _ => { 1 } - 1 | }; }
| ~~~~~
```
Fixed output after this PR:
```rust
fn main() { match () { _ => ({ 1 }) - 1, }; }
```
## False positives
Less problematic, but worth fixing (just like #118726).
```rust
fn main() {
let _ = match () { _ => 1 } - 1;
}
```
Output of `rustc -Zunpretty=expanded lib.rs` before this PR. There is no reason parentheses would need to be inserted there.
```rust
fn main() { let _ = (match () { _ => 1, }) - 1; }
```
After this PR:
```rust
fn main() { let _ = match () { _ => 1, } - 1; }
```
## Alternatives considered
In this PR I opted to parenthesize only the leading subexpression causing the statement boundary, rather than the entire statement. Example:
```rust
macro_rules! m {
($e:expr) => {
$e
};
}
fn main() {
m!(loop { break [1]; }[0] - 1);
}
```
This PR produces the following pretty-printed contents for fn main:
```rust
(loop { break [1]; })[0] - 1;
```
A different equally correct output would be:
```rust
(loop { break [1]; }[0] - 1);
```
I chose the one I did because it is the *only* approach used by handwritten code in the standard library and compiler. There are 4 places where parenthesization is being used to prevent a statement boundary, and in all 4, the developer has chosen to parenthesize the smallest subexpression rather than the whole statement:
b37d43efd9/compiler/rustc_codegen_cranelift/example/alloc_system.rs (L102)b37d43efd9/compiler/rustc_parse/src/errors.rs (L1021-L1029)b37d43efd9/library/core/src/future/poll_fn.rs (L151)b37d43efd9/library/core/src/ops/range.rs (L824-L828)
Clairify `ast::PatKind::Struct` presese of `..` by using an enum instead of a bool
The bool is mainly used for when a `..` is present, but it is also set on recovery to avoid errors. The doc comment not describes both of these cases.
See cee794ee98/compiler/rustc_parse/src/parser/pat.rs (L890-L897) for the only place this is constructed.
r? ``@compiler-errors``
Rid the AST & HIR pretty printer of cruft
Found while working on #119163.
For `trait Trait: ?Sized {}` (semantically malformed), we currently output `trait Trait for ? Sized {}` (sic!) / `trait Trait for ? Sized { }` (sic!) if `-Zunpretty=expanded` / `-Zunpretty=hir` is passed.
`trait Tr for Sized? {}` (#15521) and later also `trait Tr for ?Sized {}` (I guess, #20194) is former Rust syntax. Hence I'm removing these outdated branches.
~~This will conflict with #119163, therefore marking this PR as blocked.~~ Rebased
Add support for `for await` loops
This adds support for `for await` loops. This includes parsing, desugaring in AST->HIR lowering, and adding some support functions to the library.
Given a loop like:
```rust
for await i in iter {
...
}
```
this is desugared to something like:
```rust
let mut iter = iter.into_async_iter();
while let Some(i) = loop {
match core::pin::Pin::new(&mut iter).poll_next(cx) {
Poll::Ready(i) => break i,
Poll::Pending => yield,
}
} {
...
}
```
This PR also adds a basic `IntoAsyncIterator` trait. This is partly for symmetry with the way `Iterator` and `IntoIterator` work. The other reason is that for async iterators it's helpful to have a place apart from the data structure being iterated over to store state. `IntoAsyncIterator` gives us a good place to do this.
I've gated this feature behind `async_for_loop` and opened #118898 as the feature tracking issue.
r? `@compiler-errors`
Refactor AST trait bound modifiers
Instead of having two types to represent trait bound modifiers in the parser / the AST (`parser::ty::BoundModifiers` & `ast::TraitBoundModifier`), only to map one to the other later, just use `parser::ty::BoundModifiers` (moved & renamed to `ast::TraitBoundModifiers`).
The struct type is more extensible and easier to deal with (see [here](https://github.com/rust-lang/rust/pull/119099/files#r1430749981) and [here](https://github.com/rust-lang/rust/pull/119099/files#r1430752116) for context) since it more closely models what it represents: A compound of two kinds of modifiers, constness and polarity. Modeling this as an enum (the now removed `ast::TraitBoundModifier`) meant one had to add a new variant per *combination* of modifier kind, which simply isn't scalable and which lead to a lot of explicit non-DRY matches.
NB: `hir::TraitBoundModifier` being an enum is fine since HIR doesn't need to worry representing invalid modifier kind combinations as those get rejected during AST validation thereby immensely cutting down the number of possibilities.
