When finding item gated behind a `cfg` flag, point at it
Previously we would only mention that the item was gated out, and opportunisitically mention the feature flag name when possible. We now point to the place where the item was gated, which can be behind layers of macro indirection, or in different modules.
```
error[E0433]: failed to resolve: could not find `doesnt_exist` in `inner`
--> $DIR/diagnostics-cross-crate.rs:18:23
|
LL | cfged_out::inner::doesnt_exist::hello();
| ^^^^^^^^^^^^ could not find `doesnt_exist` in `inner`
|
note: found an item that was configured out
--> $DIR/auxiliary/cfged_out.rs:6:13
|
LL | pub mod doesnt_exist {
| ^^^^^^^^^^^^
note: the item is gated here
--> $DIR/auxiliary/cfged_out.rs:5:5
|
LL | #[cfg(FALSE)]
| ^^^^^^^^^^^^^
```
Previously we would only mention that the item was gated out, and opportunisitically mention the feature flag name when possible. We now point to the place where the item was gated, which can be behind layers of macro indirection, or in different modules.
```
error[E0433]: failed to resolve: could not find `doesnt_exist` in `inner`
--> $DIR/diagnostics-cross-crate.rs:18:23
|
LL | cfged_out::inner::doesnt_exist::hello();
| ^^^^^^^^^^^^ could not find `doesnt_exist` in `inner`
|
note: found an item that was configured out
--> $DIR/auxiliary/cfged_out.rs:6:13
|
LL | pub mod doesnt_exist {
| ^^^^^^^^^^^^
note: the item is gated here
--> $DIR/auxiliary/cfged_out.rs:5:5
|
LL | #[cfg(FALSE)]
| ^^^^^^^^^^^^^
```
Go over all structured parser suggestions and make them verbose style.
When suggesting to add or remove delimiters, turn them into multiple suggestion parts.
[`macro_metavar_expr_concat`] Add support for literals
Adds support for things like `${concat($variable, 123)}` or `${concat("hello", "_world")}` .
cc #124225
Remove `__rust_force_expr`.
This was added (with a different name) to improve an error message. It is no longer needed -- removing it changes the error message, but overall I think the new message is no worse:
- the mention of `#` in the first line is a little worse,
- but the extra context makes it very clear what the problem is, perhaps even clearer than the old message,
- and the removal of the note about the `expr` fragment (an internal detail of `__rust_force_expr`) is an improvement.
Overall I think the error is quite clear and still far better than the old message that prompted #61933, which didn't even mention patterns.
The motivation for this is #124141, which will cause pasted metavariables to be tokenized and reparsed instead of the AST node being cached. This change in behaviour occasionally has a non-zero perf cost, and `__rust_force_expr` causes the tokenize/reparse step to occur twice. Removing `__rust_force_expr` greatly reduces the extra overhead for the `deep-vector` benchmark.
r? ```@oli-obk```
This was added (with a different name) to improve an error message. It
is no longer needed -- removing it changes the error message, but overall
I think the new message is no worse:
- the mention of `#` in the first line is a little worse,
- but the extra context makes it very clear what the problem is, perhaps
even clearer than the old message,
- and the removal of the note about the `expr` fragment (an internal
detail of `__rust_force_expr`) is an improvement.
Overall I think the error is quite clear and still far better than the
old message that prompted #61933, which didn't even mention patterns.
The motivation for this is #124141, which will cause pasted
metavariables to be tokenized and reparsed instead of the AST node being
cached. This change in behaviour occasionally has a non-zero perf cost,
and `__rust_force_expr` causes the tokenize/reparse step to occur twice.
Removing `__rust_force_expr` greatly reduces the extra overhead for the
`deep-vector` benchmark.
Print `token::Interpolated` with token stream pretty printing.
This is a step towards removing `token::Interpolated` (#124141). It unavoidably changes the output of the `stringify!` macro, generally for the better.
r? `@petrochenkov`
Remove ignore-cross-compile directive from ui/macros/proc_macro
All the other proc-macro tests don't have this, presumably this was forgotten when the restriction got lifted as it does test just fine
r? `@pietroalbini`
Spruce up the diagnostics of some early lints
Implement the various "*(note to myself) in a follow-up PR we should turn parts of this message into a subdiagnostic (help msg or even struct sugg)*" drive-by comments I left in #124417 during my review.
For context, before #124417, only a few early lints touched/decorated/customized their diagnostic because the former API made it a bit awkward. Likely because of that, things that should've been subdiagnostics were just crammed into the primary message. This PR rectifies this.
Instead of using AST pretty printing.
This is a step towards removing `token::Interpolated`, which will
eventually (in #124141) be replaced with a token stream within invisible
delimiters.
This changes (improves) the output of the `stringify!` macro in some
cases. This is allowed. As the `stringify!` docs say: "Note that the
expanded results of the input tokens may change in the future. You
should be careful if you rely on the output."
Test changes:
- tests/ui/macros/stringify.rs: this used to test both token stream
pretty printing and AST pretty printing via different ways of invoking
of `stringify!` (i.e. `$expr` vs `$tt`). But those two different
invocations now give the same result, which is a nice consistency
improvement. This removes the need for all the `c2*` macros. The AST
pretty printer now has more thorough testing thanks to #125236.
- tests/ui/proc-macro/*: minor improvements where small differences
between `INPUT (DISPLAY)` output and `DEEP-RE-COLLECTED (DISPLAY)`
output disappear.
Add tests for `-Zunpretty=expanded` ported from stringify's tests
This PR adds a new set of tests for the AST pretty-printer.
Previously, pretty-printer edge cases were tested by way of `stringify!` in [tests/ui/macros/stringify.rs](https://github.com/rust-lang/rust/blob/1.78.0/tests/ui/macros/stringify.rs), such as the tests added by 419b26931b and 527e2eac17.
Those tests will no longer provide effective coverage of the AST pretty-printer after #124141. `Nonterminal` and `TokenKind::Interpolated` are being removed, and a consequence is that `stringify!` will perform token stream pretty printing, instead of AST pretty printing, in all of the `stringify!` cases including $:expr and all other interpolations.
This PR adds 2 new ui tests with `compile-flags: -Zunpretty=expanded`:
- **tests/ui/unpretty/expanded-exhaustive.rs** — this test aims for exhaustive coverage of all the variants of `ExprKind`, `ItemKind`, `PatKind`, `StmtKind`, `TyKind`, and `VisibilityKind`. Some parts could use being fleshed out further, but the current state is roughly on par with what exists in the old stringify-based tests.
- **tests/ui/unpretty/expanded-interpolation.rs** — this test covers tricky macro metavariable edge cases that require the AST pretty printer to synthesize parentheses in order for the printed code to be valid Rust syntax.
r? `@nnethercote`
Fix println! ICE when parsing percent prefix number
This PR fixes#125002 ICE occurring, for example, with `println!("%100000", 1)` or `println!("% 100000", 1)`.
## Test Case/Change Explanation
The return type of `Num::from_str` has been changed to `Option<Self>` to handle errors when parsing large integers fails.
1. The first `println!` in the test case covers the change of the first `Num::from_str` usage in `format_foreign.rs:426`.
2. The second `println!` in the test case covers the change of the second `Num::from_str` usage in line 460.
3. The 3rd to 5th `Num::from_str` usages behave the same as before.
The 3rd usage would cause an ICE when `num > u16::MAX` in the previous version, but this commit does not include a fix for the ICE in `println!("{:100000$}")`. I think we need to emit an error in the compiler and have more discussion in another issue/PR.
Update `expr` matcher for Edition 2024 and add `expr_2021` nonterminal
This commit adds a new nonterminal `expr_2021` in macro patterns, and `expr_fragment_specifier_2024` feature flag.
This change also updates `expr` so that on Edition 2024 it will also match `const { ... }` blocks, while `expr_2021` preserves the current behavior of `expr`, matching expressions without `const` blocks.
Joint work with `@vincenzopalazzo.`
Issue #123742
Rename `${length()}` to `${len()}`
Implements the rename suggested in https://github.com/rust-lang/rust/pull/122808#issuecomment-2047722187
> I brought this up in the doc PR but it belongs here – `length` should probably be renamed `len` before stabilization. The latter is de facto standard in the standard library, whereas the former is only used in a single unstable API. These metafunctions aren’t library items of course, but should presumably still be consistent with established names.
r? `@c410-f3r`
Remove `NtIdent` and `NtLifetime`
This is one part of the bigger "remove `Nonterminal` and `TokenKind::Interpolated`" change drafted in #114647. More details in the individual commit messages.
r? `@petrochenkov`
This commit adds a new nonterminal `expr_2021` in macro patterns, and
`expr_fragment_specifier_2024` feature flag. For now, `expr` and
`expr_2021` are treated the same, but in future PRs we will update
`expr` to match to new grammar.
Co-authored-by: Vincezo Palazzo <vincenzopalazzodev@gmail.com>
This span records the declaration of the metavariable in the LHS of the macro.
It's used in a couple of error messages. Unfortunately, it gets in the way of
the long-term goal of removing `TokenKind::Interpolated`. So this commit
removes it, which degrades a couple of (obscure) error messages but makes
things simpler and enables the next commit.
* The WASI targets deal with the `main` symbol a bit differently than
native so some `codegen` and `assembly` tests have been ignored.
* All `ignore-emscripten` directives have been updated to
`ignore-wasm32` to be more clear that all wasm targets are ignored and
it's not just Emscripten.
* Most `ignore-wasm32-bare` directives are now gone.
* Some ignore directives for wasm were switched to `needs-unwind`
instead.
* Many `ignore-wasm32*` directives are removed as the tests work with
WASI as opposed to `wasm32-unknown-unknown`.
This commit is extracted from #122036 and adds a new directive to the
`compiletest` test runner, `//@ needs-threads`. This is intended to
capture the need that a target must implement threading to execute a
specific test, typically one that uses `std::thread`. This is primarily
done for WebAssembly targets which currently do not have threads by
default. This enables transitioning a lot of `//@ ignore-wasm*`-style
ignores into a more self-documenting `//@ needs-threads` directive.
Additionally the `wasm32-wasi-preview1-threads` target, for example,
does actually have threads, but isn't tested in CI at this time. This
change enables running these tests for that target, but not other wasm
targets.
macro_rules: Preserve all metavariable spans in a global side table
This PR preserves spans of `tt` metavariables used to pass tokens to declarative macros.
Such metavariable spans can then be used in span combination operations like `Span::to` to improve all kinds of diagnostics.
Spans of non-`tt` metavariables are currently kept in nonterminal tokens, but the long term plan is remove all nonterminal tokens from rustc parser and rely on the proc macro model with invisible delimiters (#114647, #67062).
In particular, `NtIdent` nonterminal (corresponding to `ident` metavariables) becomes easy to remove when this PR lands (#119412 does it).
The metavariable spans are kept in a global side table keyed by `Span`s of original tokens.
The alternative to the side table is keeping them in `SpanData` instead, but the performance regressions would be large because any spans from tokens passed to declarative macros would stop being inline and would work through span interner instead, and the penalty would be paid even if we never use the metavar span for the given original span.
(But also see the comment on `fn maybe_use_metavar_location` describing the map collision issues with the side table approach.)
There are also other alternatives - keeping the metavar span in `Token` or `TokenTree`, but associating it with `Span` itsel is the most natural choice because metavar spans are used in span combining operations, and those operations are not necessarily tied to tokens.
There are a number of cases where we erroneously omit the space between
two tokens, all involving an exception to a more general case. The
affected tokens are `$`, `!`, `.`, `,`, and `let` followed by a
parenthesis.
This fixes a lot of FIXME comments.
fix: Overlapping spans in delimited meta-vars
Closes#118786
Delimited meta-vars inside of MBE's spans were set to have the same opening and closing position resulting in an ICE when debug assertions were enabled and an error was present in the templated code.
This ensures that the spans do not overlap, whilst still having the spans point at the usage of the meta-var inside the macro definition.
It includes a regression test.
🖤
[`RFC 3086`] Attempt to try to resolve blocking concerns
Implements what is described at https://github.com/rust-lang/rust/issues/83527#issuecomment-1744822345 to hopefully make some progress.
It is unknown if such approach is or isn't desired due to the lack of further feedback, as such, it is probably best to nominate this PR to the official entities.
`@rustbot` labels +I-compiler-nominated
Do not parenthesize exterior struct lit inside match guards
Before this PR, the AST pretty-printer injects parentheses around expressions any time parens _could_ be needed depending on what else is in the code that surrounds that expression. But the pretty-printer did not pass around enough context to understand whether parentheses really _are_ needed on any particular expression. As a consequence, there are false positives where unneeded parentheses are being inserted.
Example:
```rust
#![feature(if_let_guard)]
macro_rules! pp {
($e:expr) => {
stringify!($e)
};
}
fn main() {
println!("{}", pp!(match () { () if let _ = Struct {} => {} }));
}
```
**Before:**
```console
match () { () if let _ = (Struct {}) => {} }
```
**After:**
```console
match () { () if let _ = Struct {} => {} }
```
This PR introduces a bit of state that is passed across various expression printing methods to help understand accurately whether particular situations require parentheses injected by the pretty printer, and it fixes one such false positive involving match guards as shown above.
There are other parenthesization false positive cases not fixed by this PR. I intend to address these in follow-up PRs. For example here is one: the expression `{ let _ = match x {} + 1; }` is pretty-printed as `{ let _ = (match x {}) + 1; }` despite there being no reason for parentheses to appear there.
This is an extension of the previous commit. It means the output of
something like this:
```
stringify!(let a: Vec<u32> = vec![];)
```
goes from this:
```
let a: Vec<u32> = vec![] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![];
```
`tokenstream::Spacing` appears on all `TokenTree::Token` instances,
both punct and non-punct. Its current usage:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `Alone` means "cannot join with the next token *or* can join with the
next token but that token is not a punct".
The fact that `Alone` is used for two different cases is awkward.
This commit augments `tokenstream::Spacing` with a new variant
`JointHidden`, resulting in:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `JointHidden` means "can join with the next token *and* that token is a
not a punct".
- `Alone` means "cannot join with the next token".
This *drastically* improves the output of `print_tts`. For example,
this:
```
stringify!(let a: Vec<u32> = vec![];)
```
currently produces this string:
```
let a : Vec < u32 > = vec! [] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![] ;
```
(The space after the `]` is because `TokenTree::Delimited` currently
doesn't have spacing information. The subsequent commit fixes this.)
The new `print_tts` doesn't replicate original code perfectly. E.g.
multiple space characters will be condensed into a single space
character. But it's much improved.
`print_tts` still produces the old, uglier output for code produced by
proc macros. Because we have to translate the generated code from
`proc_macro::Spacing` to the more expressive `token::Spacing`, which
results in too much `proc_macro::Along` usage and no
`proc_macro::JointHidden` usage. So `space_between` still exists and
is used by `print_tts` in conjunction with the `Spacing` field.
This change will also help with the removal of `Token::Interpolated`.
Currently interpolated tokens are pretty-printed nicely via AST pretty
printing. `Token::Interpolated` removal will mean they get printed with
`print_tts`. Without this change, that would result in much uglier
output for code produced by decl macro expansions. With this change, AST
pretty printing and `print_tts` produce similar results.
The commit also tweaks the comments on `proc_macro::Spacing`. In
particular, it refers to "compound tokens" rather than "multi-char
operators" because lifetimes aren't operators.
To avoid `!matches!(...)`, which is hard to think about. Instead every
case now uses direct pattern matching and returns true or false.
Also add a couple of cases to the `stringify.rs` test that currently
print badly.
