rust/compiler/rustc_parse
Nicholas Nethercote 9d31f86f0d Overhaul token collection.
This commit does the following.

- Renames `collect_tokens_trailing_token` as `collect_tokens`, because
  (a) it's annoying long, and (b) the `_trailing_token` bit is less
  accurate now that its types have changed.

- In `collect_tokens`, adds a `Option<CollectPos>` argument and a
  `UsePreAttrPos` in the return type of `f`. These are used in
  `parse_expr_force_collect` (for vanilla expressions) and in
  `parse_stmt_without_recovery` (for two different cases of expression
  statements). Together these ensure are enough to fix all the problems
  with token collection and assoc expressions. The changes to the
  `stringify.rs` test demonstrate some of these.

- Adds a new test. The code in this test was causing an assertion
  failure prior to this commit, due to an invalid `NodeRange`.

The extra complexity is annoying, but necessary to fix the existing
problems.
2024-08-16 09:07:55 +10:00
..
src Overhaul token collection. 2024-08-16 09:07:55 +10:00
Cargo.toml Move some tests from rustc_expand to rustc_parse. 2024-05-06 09:06:02 +10:00
messages.ftl Rollup merge of #128496 - clubby789:box-syntax-multipart, r=compiler-errors 2024-08-01 18:43:41 +02:00