Remove edition umbrella features.
In the 2018 edition, there was an "umbrella" feature `#[feature(rust_2018_preview)]` which was used to enable several other features at once. This umbrella mechanism was not used in the 2021 edition and likely will not be used in 2024 either. During 2018 users reported that setting the feature was awkward, especially since they already needed to opt-in via the edition mechanism.
This PR removes this mechanism because I believe it will not be used (and will clean up and simplify the code). I believe that there are better ways to handle features and editions. In short:
- For highly experimental features, that may or may not be involved in an edition, they can implement regular feature gates like `tcx.features().my_feature`.
- For experimental features that *might* be involved in an edition, they should implement gates with `tcx.features().my_feature && span.at_least_rust_20xx()`. This requires the user to still specify `#![feature(my_feature)]`, to avoid disrupting testing of other edition features which are ready and have been accepted within the edition.
- For experimental features that have graduated to definitely be part of an edition, they should implement gates with `tcx.features().my_feature || span.at_least_rust_20xx()`, or just remove the feature check altogether and just check `span.at_least_rust_20xx()`.
- For relatively simple changes, they can skip the whole feature gating thing and just check `span.at_least_rust_20xx()`, and rely on the instability of the edition itself (which requires `-Zunstable-options`) to gate it.
I am working on documenting all of this in the rustc-dev-guide.
This is an extension of the previous commit. It means the output of
something like this:
```
stringify!(let a: Vec<u32> = vec![];)
```
goes from this:
```
let a: Vec<u32> = vec![] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![];
```
`tokenstream::Spacing` appears on all `TokenTree::Token` instances,
both punct and non-punct. Its current usage:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `Alone` means "cannot join with the next token *or* can join with the
next token but that token is not a punct".
The fact that `Alone` is used for two different cases is awkward.
This commit augments `tokenstream::Spacing` with a new variant
`JointHidden`, resulting in:
- `Joint` means "can join with the next token *and* that token is a
punct".
- `JointHidden` means "can join with the next token *and* that token is a
not a punct".
- `Alone` means "cannot join with the next token".
This *drastically* improves the output of `print_tts`. For example,
this:
```
stringify!(let a: Vec<u32> = vec![];)
```
currently produces this string:
```
let a : Vec < u32 > = vec! [] ;
```
With this PR, it now produces this string:
```
let a: Vec<u32> = vec![] ;
```
(The space after the `]` is because `TokenTree::Delimited` currently
doesn't have spacing information. The subsequent commit fixes this.)
The new `print_tts` doesn't replicate original code perfectly. E.g.
multiple space characters will be condensed into a single space
character. But it's much improved.
`print_tts` still produces the old, uglier output for code produced by
proc macros. Because we have to translate the generated code from
`proc_macro::Spacing` to the more expressive `token::Spacing`, which
results in too much `proc_macro::Along` usage and no
`proc_macro::JointHidden` usage. So `space_between` still exists and
is used by `print_tts` in conjunction with the `Spacing` field.
This change will also help with the removal of `Token::Interpolated`.
Currently interpolated tokens are pretty-printed nicely via AST pretty
printing. `Token::Interpolated` removal will mean they get printed with
`print_tts`. Without this change, that would result in much uglier
output for code produced by decl macro expansions. With this change, AST
pretty printing and `print_tts` produce similar results.
The commit also tweaks the comments on `proc_macro::Spacing`. In
particular, it refers to "compound tokens" rather than "multi-char
operators" because lifetimes aren't operators.
This keeps track of usage of internal features, and changes the message
to instead tell them that using internal features is not supported.
See MCP 620.
It's a better name, and lets "active features" refer to the features
that are active in a particular program, due to being declared or
enabled by the edition.
The commit also renames `Features::enabled` as `Features::active` to
match this; I changed my mind and have decided that "active" is a little
better thatn "enabled" for this, particularly because a number of
pre-existing comments use "active" in this way.
Finally, the commit renames `Status::Stable` as `Status::Accepted`, to
match `ACCEPTED_FEATURES`.
`State` is used to distinguish active vs accepted vs removed features.
However, these can also be distinguished by their location, in
`ACTIVE_FEATURES`, `ACCEPTED_FEATURES`, and `REMOVED_FEATURES`.
So this commit removes `State` and moves the internals of its variants
next to the `Feature` in each element of `*_FEATURES`, introducing new
types `ActiveFeature` and `RemovedFeature`. (There is no need for
`AcceptedFeature` because `State::Accepted` had no fields.)
This is a tighter type representation, avoids the need for some runtime
checks, and makes the code a bit shorter.
Currently `rust_20XX_preview` features aren't recorded as declared even
when they are explicit declared. Similarly, redundant edition-dependent
features (e.g. `test_2018_feature`) aren't recorded as declared.
This commit marks them as recorded. There is no detectable functional
change, but it makes things more consistent.
The word "active" is currently used in two different and confusing ways:
- `ACTIVE_FEATURES` actually means "available unstable features"
- `Features::active_features` actually means "features declared in the
crate's code", which can include feature within `ACTIVE_FEATURES` but
also others.
(This is also distinct from "enabled" features which includes declared
features but also some edition-specific features automatically enabled
depending on the edition in use.)
This commit changes the `Features::active_features` to
`Features::declared_features` which actually matches its meaning.
Likewise, `Features::active` becomes `Features::declared`.
The new way of doing things:
- Avoids some code duplication.
- Distinguishes the `crate_edition` (which comes from `--edition`) and
the `features_edition` (which combines `--edition` along with any
`rustc_20XX_preview` features), which is useful.
- Has a simpler initial loop, one that just looks for
`rustc_20XX_preview` features in order to compute `features_edition`.
- Creates a fallible alternative to `Features::enabled`, which is
useful.
It's not easy to see how exactly the old and new code are equivalent,
but it's reassuring to know that the test coverage is quite good for
this stuff.
There is a single features (`no_stack_check`) in
`STABLE_REMOVED_FEATURES`. But the treatment of
`STABLE_REMOVED_FEATURES` and `REMOVED_FEATURES` is actually identical.
So this commit just merges them, and uses a comment to record
`no_stack_check`'s unique "stable removed" status.
This also lets `State::Stabilized` (which was a terrible name) be
removed.
This is surprising, but the new comment explains why. It's a logical
conclusion in the drive to avoid `TokenTree` clones.
`TokenTreeCursor` is now only used within `Parser`. It's still needed
due to `replace_prev_and_rewind`.
Previously it removed all other attributes from the crate root.
Now it removes only attributes below itself.
So it becomes possible to configure some global crate properties even for fully unconfigured crates.
`#[cfg]`s are frequently used to gate crate content behind cargo
features. This can lead to very confusing errors when features are
missing. For example, `serde` doesn't have the `derive` feature by
default. Therefore, `serde::Serialize` fails to resolve with a generic
error, even though the macro is present in the docs.
This commit adds a list of all stripped item names to metadata. This is
filled during macro expansion and then, through a fed query, persisted
in metadata. The downstream resolver can then access the metadata to
look at possible candidates for mentioning in the errors.
This slightly increases metadata (800k->809k for the feature-heavy
windows crate), but not enough to really matter.
It partially expands crate attributes before the main expansion pass (without modifying the crate), and the produced preliminary crate attribute list is used for querying a few attributes that are required very early.
Crate-level cfg attributes are then expanded normally during the main expansion pass, like attributes on any other nodes.
After removing the `map_in_place` method, which isn't much use because
modifying every element in a collection such as a `Vec` can be done
trivially with iteration.
This migrates everything but the `mbe` and `proc_macro` modules. It also
contains a few cleanups and drive-by/accidental diagnostic improvements
which can be seen in the diff for the UI tests.
`To` is better than `Create` for indicating that this is a non-consuming
conversion, rather than creating something out of nothing.
And the addition of `Attr` is because the current names makes them sound
like they relate to `TokenStream`, but really they relate to
`AttrTokenStream`.
These two type names are long and have long matching prefixes. I find
them hard to read, especially in combinations like
`AttrAnnotatedTokenStream::new(vec![AttrAnnotatedTokenTree::Token(..)])`.
This commit renames them as `AttrToken{Stream,Tree}`.
In some places we use `Vec<Attribute>` and some places we use
`ThinVec<Attribute>` (a.k.a. `AttrVec`). This results in various points
where we have to convert between `Vec` and `ThinVec`.
This commit changes the places that use `Vec<Attribute>` to use
`AttrVec`. A lot of this is mechanical and boring, but there are
some interesting parts:
- It adds a few new methods to `ThinVec`.
- It implements `MapInPlace` for `ThinVec`, and introduces a macro to
avoid the repetition of this trait for `Vec`, `SmallVec`, and
`ThinVec`.
Overall, it makes the code a little nicer, and has little effect on
performance. But it is a precursor to removing
`rustc_data_structures::thin_vec::ThinVec` and replacing it with
`thin_vec::ThinVec`, which is implemented more efficiently.
Remove `TreeAndSpacing`.
A `TokenStream` contains a `Lrc<Vec<(TokenTree, Spacing)>>`. But this is
not quite right. `Spacing` makes sense for `TokenTree::Token`, but does
not make sense for `TokenTree::Delimited`, because a
`TokenTree::Delimited` cannot be joined with another `TokenTree`.
This commit fixes this problem, by adding `Spacing` to `TokenTree::Token`,
changing `TokenStream` to contain a `Lrc<Vec<TokenTree>>`, and removing the
`TreeAndSpacing` typedef.
The commit removes these two impls:
- `impl From<TokenTree> for TokenStream`
- `impl From<TokenTree> for TreeAndSpacing`
These were useful, but also resulted in code with many `.into()` calls
that was hard to read, particularly for anyone not highly familiar with
the relevant types. This commit makes some other changes to compensate:
- `TokenTree::token()` becomes `TokenTree::token_{alone,joint}()`.
- `TokenStream::token_{alone,joint}()` are added.
- `TokenStream::delimited` is added.
This results in things like this:
```rust
TokenTree::token(token::Semi, stmt.span).into()
```
changing to this:
```rust
TokenStream::token_alone(token::Semi, stmt.span)
```
This makes the type of the result, and its spacing, clearer.
These changes also simplifies `Cursor` and `CursorRef`, because they no longer
need to distinguish between `next` and `next_with_spacing`.
r? `@petrochenkov`
A `TokenStream` contains a `Lrc<Vec<(TokenTree, Spacing)>>`. But this is
not quite right. `Spacing` makes sense for `TokenTree::Token`, but does
not make sense for `TokenTree::Delimited`, because a
`TokenTree::Delimited` cannot be joined with another `TokenTree`.
This commit fixes this problem, by adding `Spacing` to `TokenTree::Token`,
changing `TokenStream` to contain a `Lrc<Vec<TokenTree>>`, and removing the
`TreeAndSpacing` typedef.
The commit removes these two impls:
- `impl From<TokenTree> for TokenStream`
- `impl From<TokenTree> for TreeAndSpacing`
These were useful, but also resulted in code with many `.into()` calls
that was hard to read, particularly for anyone not highly familiar with
the relevant types. This commit makes some other changes to compensate:
- `TokenTree::token()` becomes `TokenTree::token_{alone,joint}()`.
- `TokenStream::token_{alone,joint}()` are added.
- `TokenStream::delimited` is added.
This results in things like this:
```rust
TokenTree::token(token::Semi, stmt.span).into()
```
changing to this:
```rust
TokenStream::token_alone(token::Semi, stmt.span)
```
This makes the type of the result, and its spacing, clearer.
These changes also simplifies `Cursor` and `CursorRef`, because they no longer
need to distinguish between `next` and `next_with_spacing`.
Change `span_suggestion` (and variants) to take `impl ToString` rather
than `String` for the suggested code, as this simplifies the
requirements on the diagnostic derive.
Signed-off-by: David Wood <david.wood@huawei.com>
