mirror of
https://github.com/rust-lang/rust.git
synced 2024-11-01 15:01:51 +00:00
925f7fad57
`tokenstream::Spacing` appears on all `TokenTree::Token` instances, both punct and non-punct. Its current usage: - `Joint` means "can join with the next token *and* that token is a punct". - `Alone` means "cannot join with the next token *or* can join with the next token but that token is not a punct". The fact that `Alone` is used for two different cases is awkward. This commit augments `tokenstream::Spacing` with a new variant `JointHidden`, resulting in: - `Joint` means "can join with the next token *and* that token is a punct". - `JointHidden` means "can join with the next token *and* that token is a not a punct". - `Alone` means "cannot join with the next token". This *drastically* improves the output of `print_tts`. For example, this: ``` stringify!(let a: Vec<u32> = vec![];) ``` currently produces this string: ``` let a : Vec < u32 > = vec! [] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![] ; ``` (The space after the `]` is because `TokenTree::Delimited` currently doesn't have spacing information. The subsequent commit fixes this.) The new `print_tts` doesn't replicate original code perfectly. E.g. multiple space characters will be condensed into a single space character. But it's much improved. `print_tts` still produces the old, uglier output for code produced by proc macros. Because we have to translate the generated code from `proc_macro::Spacing` to the more expressive `token::Spacing`, which results in too much `proc_macro::Along` usage and no `proc_macro::JointHidden` usage. So `space_between` still exists and is used by `print_tts` in conjunction with the `Spacing` field. This change will also help with the removal of `Token::Interpolated`. Currently interpolated tokens are pretty-printed nicely via AST pretty printing. `Token::Interpolated` removal will mean they get printed with `print_tts`. Without this change, that would result in much uglier output for code produced by decl macro expansions. With this change, AST pretty printing and `print_tts` produce similar results. The commit also tweaks the comments on `proc_macro::Spacing`. In particular, it refers to "compound tokens" rather than "multi-char operators" because lifetimes aren't operators. |
||
|---|---|---|
| .. | ||
| auxiliary | ||
| arguments.rs | ||
| arguments.stderr | ||
| assoc_item_ctxt.rs | ||
| assoc_item_ctxt.stderr | ||
| assoc_ty_bindings.rs | ||
| cross-crate-codegen-attrs.rs | ||
| cross-crate-define-and-use.rs | ||
| cross-crate-fields.rs | ||
| cross-crate-glob-hygiene.rs | ||
| cross-crate-glob-hygiene.stderr | ||
| cross-crate-methods.rs | ||
| cross-crate-name-collision.rs | ||
| cross-crate-name-hiding-2.rs | ||
| cross-crate-name-hiding-2.stderr | ||
| cross-crate-name-hiding.rs | ||
| cross-crate-name-hiding.stderr | ||
| cross-crate-redefine.rs | ||
| cross-crate-redefine.stderr | ||
| cross-crate-variants.rs | ||
| dollar-crate-modern.rs | ||
| duplicate_lifetimes.rs | ||
| duplicate_lifetimes.stderr | ||
| eager-from-opaque-2.rs | ||
| eager-from-opaque.rs | ||
| expansion-info-reset.rs | ||
| expansion-info-reset.stderr | ||
| extern-prelude-from-opaque-fail-2018.rs | ||
| extern-prelude-from-opaque-fail-2018.stderr | ||
| extern-prelude-from-opaque-fail.rs | ||
| extern-prelude-from-opaque-fail.stderr | ||
| fields-definition.rs | ||
| fields-definition.stderr | ||
| fields-move.rs | ||
| fields-move.stderr | ||
| fields-numeric-borrowck.rs | ||
| fields-numeric-borrowck.stderr | ||
| fields.rs | ||
| fields.stderr | ||
| for-loop.rs | ||
| for-loop.stderr | ||
| format-args.rs | ||
| generate-mod.rs | ||
| generate-mod.stderr | ||
| generic_params.rs | ||
| globs.rs | ||
| globs.stderr | ||
| hir-res-hygiene.rs | ||
| hygiene-dodging-1.rs | ||
| hygiene.rs | ||
| hygienic-label-1.rs | ||
| hygienic-label-1.stderr | ||
| hygienic-label-2.rs | ||
| hygienic-label-2.stderr | ||
| hygienic-label-3.rs | ||
| hygienic-label-3.stderr | ||
| hygienic-label-4.rs | ||
| hygienic-label-4.stderr | ||
| hygienic-labels-in-let.rs | ||
| hygienic-labels.rs | ||
| impl_items-2.rs | ||
| impl_items-2.stderr | ||
| impl_items.rs | ||
| impl_items.stderr | ||
| intercrate.rs | ||
| intercrate.stderr | ||
| issue-15221.rs | ||
| issue-32922.rs | ||
| issue-40847.rs | ||
| issue-44128.rs | ||
| issue-47311.rs | ||
| issue-47312.rs | ||
| issue-61574-const-parameters.rs | ||
| issue-77523-def-site-async-await.rs | ||
| items.rs | ||
| lambda-var-hygiene.rs | ||
| legacy_interaction.rs | ||
| lexical.rs | ||
| local_inner_macros.rs | ||
| macro-metavars-legacy.rs | ||
| macro-metavars-transparent.rs | ||
| missing-self-diag.rs | ||
| missing-self-diag.stderr | ||
| nested_macro_privacy.rs | ||
| nested_macro_privacy.stderr | ||
| nested-dollar-crate.rs | ||
| no_implicit_prelude-2018.rs | ||
| no_implicit_prelude-2018.stderr | ||
| no_implicit_prelude-2021.rs | ||
| no_implicit_prelude.rs | ||
| no_implicit_prelude.stderr | ||
| panic-location.rs | ||
| panic-location.run.stderr | ||
| pattern-macro.rs | ||
| pattern-macro.stderr | ||
| prelude-import-hygiene.rs | ||
| privacy-early.rs | ||
| privacy-early.stderr | ||
| privacy.rs | ||
| privacy.stderr | ||
| rustc-macro-transparency.rs | ||
| rustc-macro-transparency.stderr | ||
| specialization.rs | ||
| stdlib-prelude-from-opaque-early.rs | ||
| stdlib-prelude-from-opaque-late.rs | ||
| thread-local-not-in-prelude.rs | ||
| trait_items-2.rs | ||
| trait_items.rs | ||
| trait_items.stderr | ||
| traits-in-scope.rs | ||
| transparent-basic.rs | ||
| unpretty-debug.rs | ||
| unpretty-debug.stdout | ||
| wrap_unhygienic_example.rs | ||
| xcrate.rs | ||