The parser pushes a `TokenType` to `Parser::expected_token_types` on
every call to the various `check`/`eat` methods, and clears it on every
call to `bump`. Some of those `TokenType` values are full tokens that
require cloning and dropping. This is a *lot* of work for something
that is only used in error messages and it accounts for a significant
fraction of parsing execution time.
This commit overhauls `TokenType` so that `Parser::expected_token_types`
can be implemented as a bitset. This requires changing `TokenType` to a
C-style parameterless enum, and adding `TokenTypeSet` which uses a
`u128` for the bits. (The new `TokenType` has 105 variants.)
The new types `ExpTokenPair` and `ExpKeywordPair` are now arguments to
the `check`/`eat` methods. This is for maximum speed. The elements in
the pairs are always statically known; e.g. a
`token::BinOp(token::Star)` is always paired with a `TokenType::Star`.
So we now compute `TokenType`s in advance and pass them in to
`check`/`eat` rather than the current approach of constructing them on
insertion into `expected_token_types`.
Values of these pair types can be produced by the new `exp!` macro,
which is used at every `check`/`eat` call site. The macro is for
convenience, allowing any pair to be generated from a single identifier.
The ident/keyword filtering in `expected_one_of_not_found` is no longer
necessary. It was there to account for some sloppiness in
`TokenKind`/`TokenType` comparisons.
The existing `TokenType` is moved to a new file `token_type.rs`, and all
its new infrastructure is added to that file. There is more boilerplate
code than I would like, but I can't see how to make it shorter.
`rustc_span::symbol` defines some things that are re-exported from
`rustc_span`, such as `Symbol` and `sym`. But it doesn't re-export some
closely related things such as `Ident` and `kw`. So you can do `use
rustc_span::{Symbol, sym}` but you have to do `use
rustc_span::symbol::{Ident, kw}`, which is inconsistent for no good
reason.
This commit re-exports `Ident`, `kw`, and `MacroRulesNormalizedIdent`,
and changes many `rustc_span::symbol::` qualifiers in `compiler/` to
`rustc_span::`. This is a 200+ net line of code reduction, mostly
because many files with two `use rustc_span` items can be reduced to
one.
Currently there are two ways to peek at a `TokenStreamIter`.
- Wrap it in a `Peekable` and use that traits `peek` method.
- Use `TokenStreamIter`'s inherent `peek` method.
Some code uses one, some use the other. This commit converts all places
to the inherent method. This eliminates mixing of `TokenStreamIter` and
`Peekable<TokenStreamIter>` and some use of `impl Iterator` and `dyn
Iterator`.
Because `TokenStreamIter` is a much better name for a `TokenStream`
iterator. Also rename the `TokenStream::trees` method as
`TokenStream::iter`, and some local variables.
When we expand a `mod foo;` and parse `foo.rs`, we now track whether that file had an unrecovered parse error that reached the end of the file. If so, we keep that information around. When resolving a path like `foo::bar`, we do not emit any errors for "`bar` not found in `foo`", as we know that the parse error might have caused `bar` to not be parsed and accounted for.
When this happens in an existing project, every path referencing `foo` would be an irrelevant compile error. Instead, we now skip emitting anything until `foo.rs` is fixed. Tellingly enough, we didn't have any test for errors caused by `mod` expansion.
Fix#97734.
Initial implementation of `#[feature(default_field_values]`, proposed in https://github.com/rust-lang/rfcs/pull/3681.
Support default fields in enum struct variant
Allow default values in an enum struct variant definition:
```rust
pub enum Bar {
Foo {
bar: S = S,
baz: i32 = 42 + 3,
}
}
```
Allow using `..` without a base on an enum struct variant
```rust
Bar::Foo { .. }
```
`#[derive(Default)]` doesn't account for these as it is still gating `#[default]` only being allowed on unit variants.
Support `#[derive(Default)]` on enum struct variants with all defaulted fields
```rust
pub enum Bar {
#[default]
Foo {
bar: S = S,
baz: i32 = 42 + 3,
}
}
```
Check for missing fields in typeck instead of mir_build.
Expand test with `const` param case (needs `generic_const_exprs` enabled).
Properly instantiate MIR const
The following works:
```rust
struct S<A> {
a: Vec<A> = Vec::new(),
}
S::<i32> { .. }
```
Add lint for default fields that will always fail const-eval
We *allow* this to happen for API writers that might want to rely on users'
getting a compile error when using the default field, different to the error
that they would get when the field isn't default. We could change this to
*always* error instead of being a lint, if we wanted.
This will *not* catch errors for partially evaluated consts, like when the
expression relies on a const parameter.
Suggestions when encountering `Foo { .. }` without `#[feature(default_field_values)]`:
- Suggest adding a base expression if there are missing fields.
- Suggest enabling the feature if all the missing fields have optional values.
- Suggest removing `..` if there are no missing fields.
Current places where `Interpolated` is used are going to change to
instead use invisible delimiters. This prepares for that.
- It adds invisible delimiter cases to the `can_begin_*`/`may_be_*`
methods and the `failed_to_match_macro` that are equivalent to the
existing `Interpolated` cases.
- It adds panics/asserts in some places where invisible delimiters
should never occur.
- In `Parser::parse_struct_fields` it excludes an ident + invisible
delimiter from special consideration in an error message, because
that's quite different to an ident + paren/brace/bracket.
Unify FnKind between AST visitors and make WalkItemKind more straight forward
Unifying `FnKind` requires a bunch of changes to `WalkItemKind::walk` signature so I'll change them in one go
related to #128974
r? `@petrochenkov`
Use `token_descr` more in error messages
This is the first two commits from #124141, put into their own PR to get things rolling. Commit messages have the details.
r? ``@estebank``
cc ``@petrochenkov``
Much like the previous commit.
I think the removal of "the token" in each message is fine here. There
are many more error messages that mention tokens without saying "the
token" than those that do say it.
By using `token_descr`, as is done for many other errors, we can get
slightly better descriptions in error messages, e.g.
"macro expansion ignores token `let` and any following" becomes
"macro expansion ignores keyword `let` and any tokens following".
This will be more important once invisible delimiters start being
mentioned in error messages -- without this commit, that leads to error
messages such as "error at ``" because invisible delimiters are
pretty printed as an empty string.
Use `Enabled{Lang,Lib}Feature` instead of n-tuples
Instead of passing around e.g. `(gate_name, attr_span, stable_since)` 3-tuples for enabled lang features or `(gate_name, attr_span)` 2-tuples for enabled lib features, use `Enabled{Lang,Lib}Feature` structs with named fields.
Also did some minor code-golfing of involved iterator chains to hopefully make them easier to follow.
Follow-up to https://github.com/rust-lang/rust/pull/132098#issuecomment-2434523431 cc `@RalfJung.`
Fundamentally, we have *three* disjoint categories of functions:
1. const-stable functions
2. private/unstable functions that are meant to be callable from const-stable functions
3. functions that can make use of unstable const features
This PR implements the following system:
- `#[rustc_const_stable]` puts functions in the first category. It may only be applied to `#[stable]` functions.
- `#[rustc_const_unstable]` by default puts functions in the third category. The new attribute `#[rustc_const_stable_indirect]` can be added to such a function to move it into the second category.
- `const fn` without a const stability marker are in the second category if they are still unstable. They automatically inherit the feature gate for regular calls, it can now also be used for const-calls.
Also, several holes in recursive const stability checking are being closed.
There's still one potential hole that is hard to avoid, which is when MIR
building automatically inserts calls to a particular function in stable
functions -- which happens in the panic machinery. Those need to *not* be
`rustc_const_unstable` (or manually get a `rustc_const_stable_indirect`) to be
sure they follow recursive const stability. But that's a fairly rare and special
case so IMO it's fine.
The net effect of this is that a `#[unstable]` or unmarked function can be
constified simply by marking it as `const fn`, and it will then be
const-callable from stable `const fn` and subject to recursive const stability
requirements. If it is publicly reachable (which implies it cannot be unmarked),
it will be const-unstable under the same feature gate. Only if the function ever
becomes `#[stable]` does it need a `#[rustc_const_unstable]` or
`#[rustc_const_stable]` marker to decide if this should also imply
const-stability.
Adding `#[rustc_const_unstable]` is only needed for (a) functions that need to
use unstable const lang features (including intrinsics), or (b) `#[stable]`
functions that are not yet intended to be const-stable. Adding
`#[rustc_const_stable]` is only needed for functions that are actually meant to
be directly callable from stable const code. `#[rustc_const_stable_indirect]` is
used to mark intrinsics as const-callable and for `#[rustc_const_unstable]`
functions that are actually called from other, exposed-on-stable `const fn`. No
other attributes are required.
"innermost", "outermost", "leftmost", and "rightmost" don't need hyphens
These are all standard dictionary words and don't require hyphenation.
-----
Encountered an instance of this in error messages and it bugged me, so I
figured I'd fix it across the entire codebase.
Rollup of 8 pull requests
Successful merges:
- #125205 (Fixup Windows verbatim paths when used with the `include!` macro)
- #131049 (Validate args are correct for `UnevaluatedConst`, `ExistentialTraitRef`/`ExistentialProjection`)
- #131549 (Add a note for `?` on a `impl Future<Output = Result<..>>` in sync function)
- #131731 (add `TestFloatParse` to `tools.rs` for bootstrap)
- #131732 (Add doc(plugins), doc(passes), etc. to INVALID_DOC_ATTRIBUTES)
- #132006 (don't stage-off to previous compiler when CI rustc is available)
- #132022 (Move `cmp_in_dominator_order` out of graph dominator computation)
- #132033 (compiletest: Make `line_directive` return a `DirectiveLine`)
r? `@ghost`
`@rustbot` modify labels: rollup
Fixup Windows verbatim paths when used with the `include!` macro
On Windows, the following code can fail if the `OUT_DIR` environment variable is a [verbatim path](https://doc.rust-lang.org/std/path/enum.Prefix.html) (i.e. begins with `\\?\`):
```rust
include!(concat!(env!("OUT_DIR"), "/src/repro.rs"));
```
This is because verbatim paths treat `/` literally, as if it were just another character in the file name.
The good news is that the standard library already has code to fix this. We can simply use `components` to normalize the path so it works as intended.
Rollup of 9 pull requests
Successful merges:
- #122670 (Fix bug where `option_env!` would return `None` when env var is present but not valid Unicode)
- #131095 (Use environment variables instead of command line arguments for merged doctests)
- #131339 (Expand set_ptr_value / with_metadata_of docs)
- #131652 (Move polarity into `PolyTraitRef` rather than storing it on the side)
- #131675 (Update lint message for ABI not supported)
- #131681 (Fix up-to-date checking for run-make tests)
- #131702 (Suppress import errors for traits that couldve applied for method lookup error)
- #131703 (Resolved python deprecation warning in publish_toolstate.py)
- #131710 (Remove `'apostrophes'` from `rustc_parse_format`)
r? `@ghost`
`@rustbot` modify labels: rollup
Make deprecated_cfg_attr_crate_type_name a hard error
Turns the forward compatibility lint added by #83744 into a hard error, so now, while the `#![crate_name]` and `#![crate_type]` attributes are still allowed in raw form, they are now forbidden to be nested inside a `#![cfg_attr()]` attribute.
The following will now be an error:
```Rust
#![cfg_attr(foo, crate_name = "foobar")]
#![cfg_attr(foo, crate_type = "bin")]
```
This code will continue working and is not deprecated:
```Rust
#![crate_name = "foobar"]
#![crate_type = "lib"]
```
The reasoning for this is explained in #83744: it allows us to not have to cfg-expand in order to determine the crate's type and name.
As of filing the PR, exactly two years have passed since #99784 has been merged, which has turned the lint's default warning level into an error, so there has been ample time to move off the now-forbidden syntax.
cc #91632 - tracking issue for the lint
