Don't try to label `ObligationCauseCode::CompareImplItem` for an RPITIT, since it has no name
The old (current) trait solver has a limitation that when a where clause in param-env must be normalized using the same where clause, then we get spurious errors in `normalize_param_env_or_error`. I don't think there's an issue tracking it, but it's the root cause for many of the "fixed-by-next-solver" labeled issues.
Specifically, these errors may occur when checking predicate entailment of the GAT that comes out of desugaring RPITITs. Since we use `ObligationCauseCode::CompareImplItem` for these predicates, we try calling `item_name` on an RPITIT which fails, since the RPITIT has no name.
We simply suppress this logic when we're reporting a predicate entailment error for an RPITIT. RPITITs should never have predicate entailment errors, *by construction*, but they may due to this bug in the old solver.
Addresses the ICE in #127331, though doesn't fix the underlying issue (which is fundamental to the old solver).
r? types
Uplift outlives components to `rustc_type_ir`
We need this to uplift `push_outlives_components`, since the elaborator uses `push_outlives_components` to elaborate type outlives obligations and I want to uplift elaboration.
This ends up reworking and inlining a fair portion of the `GenericArg::walk_shallow` function, whose only callsite was this one. I believe I got the logic correct, but may be worthwhile to look at it closely just in case. Unfortunately github was too dumb to understand that this is a rename + change -- I could also rework the git history to split the "copy the file over" part from the actual logical changes if that makes this easier to review.
r? lcnr
Elaboration tweaks
Removes `Filter::OnlySelfThatDefines` and reimplements `transitive_bounds_that_define_assoc_item` as a separate function, since I don't want to have to uplift that mode since it's both an implementation detail (only exists to avoid cycles in astconv) and requires exposing `Ident` as an associated type on `Interner`.
r? lcnr
offset_from, offset: clearly separate safety requirements the user needs to prove from corollaries that automatically follow
By landing https://github.com/rust-lang/rust/pull/116675 we decided that objects larger than `isize::MAX` cannot exist in the address space of a Rust program, which lets us simplify these rules.
For `offset_from`, we can even state that the *absolute* distance fits into an `isize`, and therefore exclude `isize::MIN`. This PR also changes Miri to treat an `isize::MIN` difference like the other isize-overflowing cases.
Match ergonomics 2024: align with RFC again
- `&` matches `&mut` on old editions
- Add some more tests
r? ``@Nadrieril``
cc https://github.com/rust-lang/rust/issues/123076
``@rustbot`` label A-edition-2024 A-patterns
Added dots at the sentence ends of rustc AST doc
Just a tiny improvement for the AST documentation by bringing consistency to sentence ends. I intentionally didn't terminate every sentence, there are still some members not having them, but at least there's no mixing style on the type level.
Use `ControlFlow` results for visitors that are only looking for a single value
These visitors all had a `Option<Value>` or `bool` field, that, once set, was never unset or modified again. They have been refactored by removing the field and returning `ControlFlow` directly from the visitor
Split `SolverDelegate` back out from `InferCtxtLike`
This is because in order to uplift things like the `Generalizer` and other `TypeRelation`s, we want to be able to interface with `InferCtxtLike` (and `InferCtxt` as its implementation), rather that `SolverDelegate`, which only really exists as a hack to be able to define some downstream methods in `rustc_type_ir`.
r? lcnr
Improve well known value check-cfg diagnostic for the standard library
This PR adjust the current logic for hidding the rustc/Cargo suggestion to add a value to a well-known name to exclude the standard library and rustc crates.
This is done in order to improve the contributor experience, in particular when adding a new target, which often requires adding some cfgs like `target_os` which may not be available yet in stage0.
<details>
The diagnostic code would look like this.
```text
error: unexpected `cfg` condition value: `blable`
--> library/core/src/lib.rs:369:7
|
369 | #[cfg(target_os = "blable")]
| ^^^^^^^^^^^^^^^^^^^^
|
= note: expected values for `target_os` are: `aix`, `android`, `cuda`, `dragonfly`, `emscripten`, `espidf`, `freebsd`, `fuchsia`, `haiku`, `hermit`, `horizon`, `hurd`, `illumos`, `ios`, `l4re`, `linux`, `macos`, `netbsd`, `none`, `nto`, `openbsd`, `psp`, `redox`, `solaris`, `solid_asp3`, `teeos`, `tvos`, `uefi`, `unknown`, `visionos`, `vita`, `vxworks`, `wasi`, `watchos`, and `windows` and 2 more
= help: consider using a Cargo feature instead
= help: or consider adding in `Cargo.toml` the `check-cfg` lint config for the lint:
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(target_os, values("blable"))'] }
= help: or consider adding `println!("cargo::rustc-check-cfg=cfg(target_os, values(\"blable\"))");` to the top of the `build.rs`
= note: see <https://doc.rust-lang.org/nightly/rustc/check-cfg/cargo-specifics.html> for more information about checking conditional configuration
= note: `-D unexpected-cfgs` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unexpected_cfgs)]`
```
</details>
Improve dead code analysis
Fixes#120770
1. check impl items later if self ty is private although the trait method is public, cause we must use the ty firstly if it's private
2. mark the adt live if it appears in pattern, like generic argument, this implies the use of the adt
3. based on the above, we can handle the case that private adts impl Default, so that we don't need adding rustc_trivial_field_reads on Default, and the logic in should_ignore_item
r? ``@pnkfelix``
coverage: Rename `mir::coverage::BranchInfo` to `CoverageInfoHi`
This opens the door to collecting and storing coverage information that is unrelated to branch coverage or MC/DC, during MIR building.
There is no change to the output of coverage instrumentation, but one deliberate change is that functions now *always* have an attached `CoverageInfoHi` (if coverage is enabled and they are eligible), even if they didn't collect any interesting branch information.
---
`@rustbot` label +A-code-coverage
DependencyList: removed outdated comment
Comment was outdated. Didn't updated description, as `Linkage` enum have descriptive names.
Also added fixme about moving this file to rustc_metadata.
Match ergonomics 2024: Implement TC's match ergonomics proposal
Under gate `ref_pat_eat_one_layer_2024_structural`. Enabling `ref_pat_eat_one_layer_2024` at the same time allows the union of what the individual gates allow. `@traviscross`
r? `@Nadrieril`
cc https://github.com/rust-lang/rust/issues/123076
`@rustbot` label A-edition-2024 A-patterns
Miri function identity hack: account for possible inlining
Having a non-lifetime generic is not the only reason a function can be duplicated. Another possibility is that the function may be eligible for cross-crate inlining. So also take into account the inlining attribute in this Miri hack for function pointer identity.
That said, `cross_crate_inlinable` will still sometimes return true even for `inline(never)` functions:
- when they are `DefKind::Ctor(..) | DefKind::Closure` -- I assume those cannot be `InlineAttr::Never` anyway?
- when `cross_crate_inline_threshold == InliningThreshold::Always`
so maybe this is still not quite the right criterion to use for function pointer identity.
cache type sizes in type-size limit visitor
This is basically https://github.com/rust-lang/rust/pull/125507#issuecomment-2206813779 as lcnr can't open the PR now.
Locally it reduces the `itertools` regression by quite a bit, to "only +50%" compared to nightly (that includes overhead from the local lack of artifact post-processing, and is just a data point to compare to the 10-20x timings without the cache).
```console
Benchmark 1: cargo +stage1 build --release
Time (mean ± σ): 2.721 s ± 0.009 s [User: 2.446 s, System: 0.325 s]
Range (min … max): 2.710 s … 2.738 s 10 runs
Benchmark 2: cargo +nightly build --release
Time (mean ± σ): 1.784 s ± 0.005 s [User: 1.540 s, System: 0.279 s]
Range (min … max): 1.778 s … 1.792 s 10 runs
Summary
cargo +nightly build --release ran
1.52 ± 0.01 times faster than cargo +stage1 build --release
```
On master, it's from 34s to the 2.7s above.
r? compiler-errors
Rollup of 9 pull requests
Successful merges:
- #123043 (Disable dead variant removal for `#[repr(C)]` enums.)
- #126405 (Migrate some rustc_builtin_macros to SessionDiagnostic)
- #127037 (Remove some duplicated tests)
- #127283 (Reject SmartPointer constructions not serving the purpose)
- #127301 (Tweak some structured suggestions to be more verbose and accurate)
- #127307 (Allow to have different types for arguments of `Rustc::remap_path_prefix`)
- #127309 (jsondocck: add `$FILE` built-in variable)
- #127314 (Trivial update on tidy bless note)
- #127319 (Remove a use of `StructuredDiag`, which is incompatible with automatic error tainting and error translations)
r? `@ghost`
`@rustbot` modify labels: rollup
Remove a use of `StructuredDiag`, which is incompatible with automatic error tainting and error translations
fixes#127219
I want to remove all of `StructuredDiag`, but it's a bit more involved as it is also used from the `ItemCtxt`, which doesn't support tainting yet.
Tweak some structured suggestions to be more verbose and accurate
Addressing some issues I found while working on #127282.
```
error: this URL is not a hyperlink
--> $DIR/auxiliary/include-str-bare-urls.md:1:11
|
LL | HEADS UP! https://example.com MUST SHOW UP IN THE STDERR FILE!
| ^^^^^^^^^^^^^^^^^^^
|
= note: bare URLs are not automatically turned into clickable links
note: the lint level is defined here
--> $DIR/include-str-bare-urls.rs:14:9
|
LL | #![deny(rustdoc::bare_urls)]
| ^^^^^^^^^^^^^^^^^^
help: use an automatic link instead
|
LL | HEADS UP! <https://example.com> MUST SHOW UP IN THE STDERR FILE!
| + +
```
```
error[E0384]: cannot assign twice to immutable variable `v`
--> $DIR/assign-imm-local-twice.rs:7:5
|
LL | v = 1;
| ----- first assignment to `v`
LL | println!("v={}", v);
LL | v = 2;
| ^^^^^ cannot assign twice to immutable variable
|
help: consider making this binding mutable
|
LL | let mut v: isize;
| +++
```
```
error[E0393]: the type parameter `Rhs` must be explicitly specified
--> $DIR/issue-22560.rs:9:23
|
LL | trait Sub<Rhs=Self> {
| ------------------- type parameter `Rhs` must be specified for this
...
LL | type Test = dyn Add + Sub;
| ^^^
|
= note: because of the default `Self` reference, type parameters must be specified on object types
help: set the type parameter to the desired type
|
LL | type Test = dyn Add + Sub<Rhs>;
| +++++
```
```
error[E0596]: cannot borrow `v` as mutable, as it is not declared as mutable
--> $DIR/issue-33819.rs:4:34
|
LL | Some(ref v) => { let a = &mut v; },
| ^^^^^^ cannot borrow as mutable
|
help: try removing `&mut` here
|
LL - Some(ref v) => { let a = &mut v; },
LL + Some(ref v) => { let a = v; },
|
```
```
help: remove the invocation before committing it to a version control system
|
LL - dbg!();
|
```
```
error[E0308]: mismatched types
--> $DIR/issue-39974.rs:1:21
|
LL | const LENGTH: f64 = 2;
| ^ expected `f64`, found integer
|
help: use a float literal
|
LL | const LENGTH: f64 = 2.0;
| ++
```
```
error[E0529]: expected an array or slice, found `Vec<i32>`
--> $DIR/match-ergonomics.rs:8:9
|
LL | [&v] => {},
| ^^^^ pattern cannot match with input type `Vec<i32>`
|
help: consider slicing here
|
LL | match x[..] {
| ++++
```
```
error[E0609]: no field `0` on type `[u32; 1]`
--> $DIR/parenthesized-deref-suggestion.rs:10:21
|
LL | (x as [u32; 1]).0;
| ^ unknown field
|
help: instead of using tuple indexing, use array indexing
|
LL | (x as [u32; 1])[0];
| ~ +
```
Reject SmartPointer constructions not serving the purpose
Tracking issue: #123430
With this PR we will reject a row of malformed `SmartPointer` implementor candidates.
cc `@Darksonn` `@davidtwco` for context.
Migrate some rustc_builtin_macros to SessionDiagnostic
<!--
If this PR is related to an unstable feature or an otherwise tracked effort,
please link to the relevant tracking issue here. If you don't know of a related
tracking issue or there are none, feel free to ignore this.
This PR will get automatically assigned to a reviewer. In case you would like
a specific user to review your work, you can assign it to them by using
r? <reviewer name>
-->
Part of https://github.com/rust-lang/rust/issues/100717.
pick up abandoned pr: #101935
`@rustbot` label +A-translation
Disable dead variant removal for `#[repr(C)]` enums.
This prevents removing dead branches from a `#[repr(C)]` enum (they now get discriminants allocated as if they were inhabited).
Implementation notes: ABI of something like
```rust
#[repr(C)]
enum Foo {
Foo(!),
}
```
is still `Uninhabited`, but its layout is now computed as if all the branches were inhabited.
This seemed to me like a proper way to do it, especially given that ABI sanity check explicitly asserts that type-level uninhabitedness implies ABI uninhabitedness.
This probably needs some sort of FCP (given that it changes `#[repr(C)]` layout, which is a stable guarantee), but I’m not sure how to call for one or which team is the most relevant.
See https://github.com/rust-lang/unsafe-code-guidelines/issues/500.
The rules for casting `*mut X<dyn A>` -> `*mut Y<dyn B>` are as follows:
- If `B` has a principal
- `A` must have exactly the same principal (including generics)
- Auto traits of `B` must be a subset of autotraits in `A`
Note that `X<_>` and `Y<_>` can be identity, or arbitrary structs with last field being the dyn type.
The lifetime of the trait object itself (`dyn ... + 'a`) is not checked.
This prevents a few soundness issues with `#![feature(arbitrary_self_types)]` and trait upcasting.
Namely, these checks make sure that vtable is always valid for the pointee.
Stop using specialization in rustc_index and rustc_borrowck
For rustc_borrowck the version with specialization isn't much more readable anyway IMO. For rustc_index it probably doesn't affect perf in any noticeable way anyway.
Optimize SipHash by reordering compress instructions
This PR optimizes hashing by changing the order of instructions in the sip.rs `compress` macro so the CPU can parallelize it better. The new order is taken directly from Fig 2.1 in [the SipHash paper](https://eprint.iacr.org/2012/351.pdf) (but with the xors moved which makes it a little faster). I attempted to optimize it some more after this, but I think this might be the optimal instruction order. Note that this shouldn't change the behavior of hashing at all, only statements that don't depend on each other were reordered.
It appears like the current order hasn't changed since its [original implementation from 2012](fada46c421 (diff-b751133c229259d7099bbbc7835324e5504b91ab1aded9464f0c48cd22e5e420R35)) which doesn't look like it was written with data dependencies in mind.
Running `./x bench library/core --stage 0 --test-args hash` before and after this change shows the following results:
Before:
```
benchmarks:
hash::sip::bench_bytes_4 7.20/iter +/- 0.70
hash::sip::bench_bytes_7 9.01/iter +/- 0.35
hash::sip::bench_bytes_8 8.12/iter +/- 0.10
hash::sip::bench_bytes_a_16 10.07/iter +/- 0.44
hash::sip::bench_bytes_b_32 13.46/iter +/- 0.71
hash::sip::bench_bytes_c_128 37.75/iter +/- 0.48
hash::sip::bench_long_str 121.18/iter +/- 3.01
hash::sip::bench_str_of_8_bytes 11.20/iter +/- 0.25
hash::sip::bench_str_over_8_bytes 11.20/iter +/- 0.26
hash::sip::bench_str_under_8_bytes 9.89/iter +/- 0.59
hash::sip::bench_u32 9.57/iter +/- 0.44
hash::sip::bench_u32_keyed 6.97/iter +/- 0.10
hash::sip::bench_u64 8.63/iter +/- 0.07
```
After:
```
benchmarks:
hash::sip::bench_bytes_4 6.64/iter +/- 0.14
hash::sip::bench_bytes_7 8.19/iter +/- 0.07
hash::sip::bench_bytes_8 8.59/iter +/- 0.68
hash::sip::bench_bytes_a_16 9.73/iter +/- 0.49
hash::sip::bench_bytes_b_32 12.70/iter +/- 0.06
hash::sip::bench_bytes_c_128 32.38/iter +/- 0.20
hash::sip::bench_long_str 102.99/iter +/- 0.82
hash::sip::bench_str_of_8_bytes 10.71/iter +/- 0.21
hash::sip::bench_str_over_8_bytes 11.73/iter +/- 0.17
hash::sip::bench_str_under_8_bytes 10.33/iter +/- 0.41
hash::sip::bench_u32 10.41/iter +/- 0.29
hash::sip::bench_u32_keyed 9.50/iter +/- 0.30
hash::sip::bench_u64 8.44/iter +/- 1.09
```
I ran this on my computer so there's some noise, but you can tell at least `bench_long_str` is significantly faster (~18%).
Also, I noticed the same compress function from the library is used in the compiler as well, so I took the liberty of copy-pasting this change to there as well.
Thanks `@semisol` for porting SipHash for another project which led me to notice this issue in Rust, and for helping investigate. <3
rustdoc: update to pulldown-cmark 0.11
r? rustdoc
This pull request updates rustdoc to the latest version of pulldown-cmark. Along with adding new markdown extensions (which this PR doesn't enable), the new pulldown-cmark version also fixes a large number of bugs. Because all text files successfully parse as markdown, these bugfixes change the output, which can break people's existing docs.
A crater run, https://github.com/rust-lang/rust/pull/121659, has already been run for this change.
The first commit upgrades and fixes rustdoc. The second commit adds a lint for the footnote and block quote parser changes, which break the largest numbers of docs in the Crater run. The strikethrough change was mitigated in pulldown-cmark itself.
Unblocks https://github.com/rust-lang/rust-clippy/pull/12876
Fix incorrect suggestion for extra argument with a type error
Fixes#126246
I tried to fix it in the `find_errors` of ArgMatrix, but seems it's hard to avoid breaking some other test cases.
The root cause is we eliminate the first argument even with a type error at here:
6292b2af62/compiler/rustc_hir_typeck/src/fn_ctxt/checks.rs (L664)
So the left argument is always treated as extra one.
But if there is already a type error, an error message will be generated firstly, which make this issue a trivial one.
Make jump threading state sparse
Continuation of https://github.com/rust-lang/rust/pull/127024
Both dataflow const-prop and jump threading involve cloning the state vector a lot. This PR replaces the data structure by a sparse vector, considering:
- that jump threading state is typically very sparse (at most 1 or 2 set entries);
- that dataflow const-prop is disabled by default;
- that place/value map is very eager, and prone to creating an overly large state.
The first commit is shared with the previous PR to avoid needless conflicts.
r? `@oli-obk`
Remove global error count checks from typeck
Some of these are not reachable anymore, others can now rely on information local to the current typeck run. One check was actually invalid, because it was relying on wfcheck running before typeck, which is not guaranteed in the query system and usually easy to create ICEing examples for via const eval (which runs typeck before wfcheck)
Add `as_lang_item` to `LanguageItems`, new trait solver
Add `as_lang_item` which turns `DefId` into a `TraitSolverLangItem` in the new trait solver, so we can turn the large chain of if statements in `assemble_builtin_impl_candidates` into a match instead.
r? lcnr
linker: Link dylib crates by path
Linkers seem to support linking dynamic libraries by path.
Not sure why the previous scheme with splitting the path into a directory (passed with `-L`) and a name (passed with `-l`) was used (upd: likely due to https://github.com/rust-lang/rust/pull/126094#issuecomment-2155063414).
When we split a library path `some/dir/libfoo.so` into `-L some/dir` and `-l foo` we add `some/dir` to search directories for *all* libraries looked up by the linker, not just `foo`, and `foo` is also looked up in *all* search directories not just `some/dir`.
Technically we may find some unintended libraries this way.
Therefore linking dylibs via a full path is both simpler and more reliable.
It also makes the set of search directories more easily reproducible when we need to lookup some native library manually (like in https://github.com/rust-lang/rust/pull/123436).
Re-implement a type-size based limit
r? lcnr
This PR reintroduces the type length limit added in #37789, which was accidentally made practically useless by the caching changes to `Ty::walk` in #72412, which caused the `walk` function to no longer walk over identical elements.
Hitting this length limit is not fatal unless we are in codegen -- so it shouldn't affect passes like the mir inliner which creates potentially very large types (which we observed, for example, when the new trait solver compiles `itertools` in `--release` mode).
This also increases the type length limit from `1048576 == 2 ** 20` to `2 ** 24`, which covers all of the code that can be reached with craterbot-check. Individual crates can increase the length limit further if desired.
Perf regression is mild and I think we should accept it -- reinstating this limit is important for the new trait solver and to make sure we don't accidentally hit more type-size related regressions in the future.
Fixes#125460
Actually report normalization-based type errors correctly for alias-relate obligations in new solver
We have some special casing to report type mismatch errors that come from projection predicates, but we don't do that for alias-relate obligations. This PR implements that. There's a bit of code duplication, but 🤷
Best reviewed without whitespace.
r? lcnr
Check alias args for WF even if they have escaping bound vars
#### What
This PR stops skipping arguments of aliases if they have escaping bound vars, instead recursing into them and only discarding the resulting obligations referencing bounds vars.
#### An example:
From the test:
```
trait Trait {
type Gat<U: ?Sized>;
}
fn test<T>(f: for<'a> fn(<&'a T as Trait>::Gat<&'a [str]>)) where for<'a> &'a T: Trait {}
//~^ ERROR the size for values of type `[()]` cannot be known at compilation time
fn main() {}
```
We now prove that `str: Sized` in order for `&'a [str]` to be well-formed. We were previously unconditionally skipping over `&'a [str]` as it referenced a buond variable. We now recurse into it and instead only discard the `[str]: 'a` obligation because of the escaping bound vars.
#### Why?
This is a change that improves consistency about proving well-formedness earlier in the pipeline, which is necessary for future work on where-bounds in binders and correctly handling higher-ranked implied bounds. I don't expect this to fix any unsoundness.
#### What doesn't it fix?
Specifically, this doesn't check projection predicates' components are well-formed, because there are too many regressions: https://github.com/rust-lang/rust/pull/123737#issuecomment-2052198478
Since this codegen flag now only controls LLVM-generated comments rather than
all assembly comments, make the name more accurate (and also match Clang).
Rewrite handling of universe-leaking placeholder regions into outlives constraints
This commit prepares for Polonius by moving handling of leak check/universe errors out of the inference step by rewriting any universe error into an outlives-static constraint.
This variant is a work in progress but seems to pass most tests.
Note that a few debug assertions no longer hold; a few extra eyes on those changes are appreciated!
Fix `FnMut::call_mut`/`Fn::call` shim for async closures that capture references
I adjusted async closures to be able to implement `Fn` and `FnMut` *even if* they capture references, as long as those references did not need to borrow data from the closure captures themselves. See #125259.
However, when I did this, I didn't actually relax an assertion in the `build_construct_coroutine_by_move_shim` shim code, which builds the `Fn`/`FnMut`/`FnOnce` implementations for async closures. Therefore, if we actually tried to *call* `FnMut`/`Fn` on async closures, it would ICE.
This PR adjusts this assertion to ensure that we only capture immutable references in closures if they implement `Fn`/`FnMut`. It also adds a bunch of tests and makes more of the async-closure tests into `build-pass` since we often care about these tests actually generating the right closure shims and stuff. I think it might be excessive to *always* use build-pass here, but 🤷 it's not that big of a deal.
Fixes#127019Fixes#127012
r? oli-obk
Parenthesize break values containing leading label
The AST pretty printer previously produced invalid syntax in the case of `break` expressions with a value that begins with a loop or block label.
```rust
macro_rules! expr {
($e:expr) => {
$e
};
}
fn main() {
loop {
break expr!('a: loop { break 'a 1; } + 1);
};
}
```
`rustc -Zunpretty=expanded main.rs `:
```console
#![feature(prelude_import)]
#![no_std]
#[prelude_import]
use ::std::prelude::rust_2015::*;
#[macro_use]
extern crate std;
macro_rules! expr { ($e:expr) => { $e }; }
fn main() { loop { break 'a: loop { break 'a 1; } + 1; }; }
```
The expanded code is not valid Rust syntax. Printing invalid syntax is bad because it blocks `cargo expand` from being able to format the output as Rust syntax using rustfmt.
```console
error: parentheses are required around this expression to avoid confusion with a labeled break expression
--> <anon>:9:26
|
9 | fn main() { loop { break 'a: loop { break 'a 1; } + 1; }; }
| ^^^^^^^^^^^^^^^^^^^^^^^^
|
help: wrap the expression in parentheses
|
9 | fn main() { loop { break ('a: loop { break 'a 1; }) + 1; }; }
| + +
```
This PR updates the AST pretty-printer to insert parentheses around the value of a `break` expression as required to avoid this edge case.
And update the comment. Clearly the return type of this function was
changed at some point in the past, but its name and comment weren't
updated to match.
The number of source code bytes can't exceed a `u32`'s range, so a token
position also can't. This reduces the size of `Parser` and
`LazyAttrTokenStreamImpl` by eight bytes each.
Currently it uses a mixture of functional style (`flat_map`) and
imperative style (`push`), which is a bit hard to read. This commit
converts it to fully imperative, which is more concise and avoids the
need for `smallvec`.
I.e. change the return type from `TokenStream` to `Vec<TokenTree>`.
Most of the callsites require a `TokenStream`, but the recursive call
used to create `target_tokens` requires a `Vec<TokenTree>`. It's easy
to convert a `Vec<TokenTree>` to a `TokenStream` (just call
`TokenStream::new`) but it's harder to convert a `TokenStream` to a
`Vec<TokenTree>` (either iterate/clone/collect, or use `Lrc::into_inner`
if appropriate).
So this commit changes the return value to simplify that `target_tokens`
call site.
