mirror of
https://github.com/rust-lang/rust.git
synced 2025-02-14 16:03:17 +00:00
925f7fad57
`tokenstream::Spacing` appears on all `TokenTree::Token` instances, both punct and non-punct. Its current usage: - `Joint` means "can join with the next token *and* that token is a punct". - `Alone` means "cannot join with the next token *or* can join with the next token but that token is not a punct". The fact that `Alone` is used for two different cases is awkward. This commit augments `tokenstream::Spacing` with a new variant `JointHidden`, resulting in: - `Joint` means "can join with the next token *and* that token is a punct". - `JointHidden` means "can join with the next token *and* that token is a not a punct". - `Alone` means "cannot join with the next token". This *drastically* improves the output of `print_tts`. For example, this: ``` stringify!(let a: Vec<u32> = vec![];) ``` currently produces this string: ``` let a : Vec < u32 > = vec! [] ; ``` With this PR, it now produces this string: ``` let a: Vec<u32> = vec![] ; ``` (The space after the `]` is because `TokenTree::Delimited` currently doesn't have spacing information. The subsequent commit fixes this.) The new `print_tts` doesn't replicate original code perfectly. E.g. multiple space characters will be condensed into a single space character. But it's much improved. `print_tts` still produces the old, uglier output for code produced by proc macros. Because we have to translate the generated code from `proc_macro::Spacing` to the more expressive `token::Spacing`, which results in too much `proc_macro::Along` usage and no `proc_macro::JointHidden` usage. So `space_between` still exists and is used by `print_tts` in conjunction with the `Spacing` field. This change will also help with the removal of `Token::Interpolated`. Currently interpolated tokens are pretty-printed nicely via AST pretty printing. `Token::Interpolated` removal will mean they get printed with `print_tts`. Without this change, that would result in much uglier output for code produced by decl macro expansions. With this change, AST pretty printing and `print_tts` produce similar results. The commit also tweaks the comments on `proc_macro::Spacing`. In particular, it refers to "compound tokens" rather than "multi-char operators" because lifetimes aren't operators. |
||
|---|---|---|
| .. | ||
| api | ||
| add-impl.rs | ||
| amputate-span.rs | ||
| append-impl.rs | ||
| assert-span-pos.rs | ||
| attr-args.rs | ||
| attr-cfg.rs | ||
| attr-on-trait.rs | ||
| attr-stmt-expr-rpass.rs | ||
| attr-stmt-expr.rs | ||
| attribute-spans-preserved.rs | ||
| attributes-included.rs | ||
| attributes-on-definitions.rs | ||
| bang_proc_macro2.rs | ||
| bang-macro.rs | ||
| builtin-attrs.rs | ||
| call-deprecated.rs | ||
| call-site.rs | ||
| cond_plugin.rs | ||
| count_compound_ops.rs | ||
| custom-attr-only-one-derive.rs | ||
| custom-quote.rs | ||
| derive-a.rs | ||
| derive-atob.rs | ||
| derive-attr-cfg.rs | ||
| derive-b-rpass.rs | ||
| derive-b.rs | ||
| derive-bad.rs | ||
| derive-clona.rs | ||
| derive-ctod.rs | ||
| derive-foo.rs | ||
| derive-helper-shadowed-2.rs | ||
| derive-helper-shadowing-2.rs | ||
| derive-helper-shadowing.rs | ||
| derive-nothing.rs | ||
| derive-same-struct.rs | ||
| derive-two-attrs.rs | ||
| derive-union.rs | ||
| derive-unstable-2.rs | ||
| derive-unstable.rs | ||
| dollar-crate-external.rs | ||
| double.rs | ||
| duplicate.rs | ||
| edition-gated-async-move-syntax.rs | ||
| edition-imports-2015.rs | ||
| empty-crate.rs | ||
| expand-expr.rs | ||
| expand-with-a-macro.rs | ||
| exports_no_mangle.rs | ||
| external-crate-var.rs | ||
| first-second.rs | ||
| gen-lifetime-token.rs | ||
| gen-macro-rules-hygiene.rs | ||
| gen-macro-rules.rs | ||
| generate-dollar-ident.rs | ||
| generate-mod.rs | ||
| hygiene_example_codegen.rs | ||
| hygiene_example.rs | ||
| included-file.txt | ||
| invalid-punct-ident.rs | ||
| is-available.rs | ||
| issue-38586.rs | ||
| issue-39889.rs | ||
| issue-42708.rs | ||
| issue-50061.rs | ||
| issue-50493.rs | ||
| issue-59191.rs | ||
| issue-66286.rs | ||
| issue-75801.rs | ||
| issue-79242.rs | ||
| issue-79825.rs | ||
| issue-83510.rs | ||
| issue-91800-macro.rs | ||
| issue-104884.rs | ||
| issue-107113.rs | ||
| lifetimes-rpass.rs | ||
| lifetimes.rs | ||
| macro-only-syntax.rs | ||
| make-macro.rs | ||
| meta-delim.rs | ||
| meta-macro.rs | ||
| mixed-site-span.rs | ||
| modify-ast.rs | ||
| multiple-derives.rs | ||
| multispan.rs | ||
| negative-token.rs | ||
| nested-macro-rules.rs | ||
| nonterminal-recollect-attr.rs | ||
| not-joint.rs | ||
| parent-source-spans.rs | ||
| print-tokens.rs | ||
| proc-macro-panic.rs | ||
| raw-ident.rs | ||
| re-export.rs | ||
| recollect.rs | ||
| resolved-located-at.rs | ||
| span-api-tests.rs | ||
| span-from-proc-macro.rs | ||
| span-test-macros.rs | ||
| subspan.rs | ||
| test-macros.rs | ||
| three-equals.rs | ||
| weird-hygiene.rs | ||