Add note about reverting a workaround in the future
The root cause was fixed upstream in LLVM main. This adds a reminder to revert the workaround once the LLVM rustc depends on is new enough. Since I'm not sure how such optimizations get routed to LLVM releases, I used the conservative assumption that it will only show up with LLVM 13.
Implement token-based handling of attributes during expansion
This PR modifies the macro expansion infrastructure to handle attributes
in a fully token-based manner. As a result:
* Derives macros no longer lose spans when their input is modified
by eager cfg-expansion. This is accomplished by performing eager
cfg-expansion on the token stream that we pass to the derive
proc-macro
* Inner attributes now preserve spans in all cases, including when we
have multiple inner attributes in a row.
This is accomplished through the following changes:
* New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced.
These are very similar to a normal `TokenTree`, but they also track
the position of attributes and attribute targets within the stream.
They are built when we collect tokens during parsing.
An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when
we invoke a macro.
* Token capturing and `LazyTokenStream` are modified to work with
`AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which
is created during the parsing of a nested AST node to make the 'outer'
AST node aware of the attributes and attribute target stored deeper in the token stream.
* When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
This PR modifies the macro expansion infrastructure to handle attributes
in a fully token-based manner. As a result:
* Derives macros no longer lose spans when their input is modified
by eager cfg-expansion. This is accomplished by performing eager
cfg-expansion on the token stream that we pass to the derive
proc-macro
* Inner attributes now preserve spans in all cases, including when we
have multiple inner attributes in a row.
This is accomplished through the following changes:
* New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced.
These are very similar to a normal `TokenTree`, but they also track
the position of attributes and attribute targets within the stream.
They are built when we collect tokens during parsing.
An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when
we invoke a macro.
* Token capturing and `LazyTokenStream` are modified to work with
`AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which
is created during the parsing of a nested AST node to make the 'outer'
AST node aware of the attributes and attribute target stored deeper in the token stream.
* When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`),
we tokenize and reparse our target, capturing additional information about the locations of
`#[cfg]` and `#[cfg_attr]` attributes at any depth within the target.
This is a performance optimization, allowing us to perform less work
in the typical case where captured tokens never have eager cfg-expansion run.
Fix NixOS patching
Moving the `.nix-deps` has resulted in rpath links being broken and
therefore bootstrap on NixOS broken entirely.
This PR still produces a `.nix-deps` but only for the purposes of
producing a gc root. We rpath a symlink-resolved result instead.
For purposes of simplicity we also use joinSymlink to produce a single
merged output directory so that we don't need to update multiple
locations every time we add a library or something.
Fixes a regression from https://github.com/rust-lang/rust/pull/82739.
fix incorrect Box::from_raw_in doctest
Now that Miri can run doctests, I ran it on liballoc, and found exactly one problem: this test creates a `Box<u8>` to deallocate a 4-byte allocation!
Introduced by https://github.com/rust-lang/rust/pull/80310 so r? `@Manishearth` `@kennytm`
Expand derive invocations in left-to-right order
While derives were being collected in left-to-order order, the
corresponding `Invocation`s were being pushed in the wrong order.
Moving the `.nix-deps` has resulted in rpath links being broken and
therefore bootstrap on NixOS broken entirely.
This PR still produces a `.nix-deps` but only for the purposes of
producing a gc root. We rpath a symlink-resolved result instead.
For purposes of simplicity we also use joinSymlink to produce a single
merged output directory so that we don't need to update multiple
locations every time we add a library or something.
The root cause was fixed upstream in LLVM main. This adds a reminder to revert the workaround once the LLVM rustc depends on is new enough. Since I'm not sure how such optimizations get routed to LLVM releases, I used the conservative assumption that it will only show up with LLVM 13.
fix Miri errors in libcore doctests
Now that Miri can run doctests, it found some issues in the libcore doctests:
* The `AtomicPtr` tests accessed dangling memory! `AtomicPtr::new(&mut 10);` makes the `10` a temporary that is deallocated after the end of this expression.
* The tests for `set_ptr_value` used `&array[0] as *const _` to get a pointer to the array; this needs to be `array.as_ptr()` instead (Cc https://github.com/rust-lang/unsafe-code-guidelines/issues/134).
* I reduced a buffer size in a `MaybeUninit` test to make it less slow in Miri, and added a spin loop hint to fix a diverging loop in Miri.
Don't tell users to use a nightly flag on the stable channel
When a crate requires a newer edition, currently rustc tells users to use `-Z unstable-options`. This is not ideal, because:
* This flag doesn't work on the stable channel, so solution to one error only causes another error, which is frustrating.
* Directs users towards the nightly channel, which is not necessarily the correct solution. Once the next edition is released, this message will be mostly seen by users of out-of-date stable Rust versions who merely need to update their Rust to the latest stable.
Avoid `;` -> `,` recovery and unclosed `}` recovery from being too verbose
Those two recovery attempts have a very bad interaction that causes too
unnecessary output. Add a simple gate to avoid interpreting a `;` as a
`,` when there are unclosed braces.
Fix#83498.
Categorize and explain target features support
There are 3 different uses of the `-C target-feature` args passed to rustc:
1. All of the features are passed to LLVM, which uses them to configure code-generation. This is sort-of stabilized since 1.0 though LLVM does change/add/remove target features regularly.
2. Target features which are in [the compiler's allowlist](69e1d22ddb/compiler/rustc_codegen_ssa/src/target_features.rs (L12-L34)) can be used in `cfg!(target_feature)` etc. These may have different names than in LLVM and are renamed before passing them to LLVM.
3. Target features which are in the allowlist and which are stabilized or feature-gate-enabled can be used in `#[target_feature]`.
It can be confusing that `rustc --print target-features` just prints out the LLVM features without separating out the rustc features or even mentioning that the dichotomy exists.
This improves the situation by separating out the rustc and LLVM target features and adding a brief explanation about the difference.
Abbreviated Example Output:
```
$ rustc --print target-features
Features supported by rustc for this target:
adx - Support ADX instructions.
aes - Enable AES instructions.
...
xsaves - Support xsaves instructions.
crt-static - Enables libraries with C Run-time Libraries(CRT) to be statically linked.
Code-generation features supported by LLVM for this target:
16bit-mode - 16-bit mode (i8086).
32bit-mode - 32-bit mode (80386).
...
x87 - Enable X87 float instructions.
xop - Enable XOP instructions.
Use +feature to enable a feature, or -feature to disable it.
For example, rustc -C target-cpu=mycpu -C target-feature=+feature1,-feature2
Code-generation features cannot be used in cfg or #[target_feature],
and may be renamed or removed in a future version of LLVM or rustc.
```
Motivated by #83975.
CC https://github.com/rust-lang/rust/issues/49653
Those two recovery attempts have a very bad interaction that causes too
unnecessary output. Add a simple gate to avoid interpreting a `;` as a
`,` when there are unclosed braces.
reduce threads spawned by ui-tests
The test harness already spawns enough tests to keep all cores busy.
Individual tests should keep their own threading to a minimum to avoid context switch overhead.
When running ui tests with lld enabled this shaves about 10% off that testsuite on my machine.
Resolves#81946
rustdoc: Don't generate blanket impls when running --show-coverage
`get_blanket_impls` is the slowest part of rustdoc, and the coverage pass
completely ignores blanket impls. This stops running it at all, and also
removes some unnecessary checks in `calculate_doc_coverage` that ignored
the impl anyway.
We don't currently measure --show-coverage in perf.rlo, but I tested
this locally on cargo and it brought the time down from 2.9 to 1.6
seconds.
This also adds back a commented-out test; Rustdoc has been able to deal with `impl trait` for almost a year now.
r? `@GuillaumeGomez`
get_blanket_impls is the slowest part of rustdoc, and the coverage pass
completely ignores blanket impls. This stops running it at all, and also
removes some unnecessary checks in `calculate_doc_coverage` that ignored
the impl anyway.
We don't currently measure --show-coverage in perf.rlo, but I tested
this locally on cargo and it brought the time down from 2.9 to 1.6
seconds.
Mention missing 1.38.0 change in RELEASES.md
Mention that doc comments on `pub use` statements are prepended to the documentation of the reexported item
Fixes#84007
the test harness already spawns enough tests for all cores, individual
tests should keep their own threading to a minimum to avoid context switch
overhead
some tests fail with 1 CGU, so explicit compile flags have been added
to keep their old behavior
Don't concatenate binders across types
Partially addresses #83737
There's actually two issues that I uncovered in #83737. The first is that we are concatenating bound vars across types, i.e. in
```
F: Fn(&()) -> &mut (dyn Future<Output = ()> + Unpin)
```
the bound vars on `Future` get set as `for<anon>` since those are the binders on `Fn(&()`. This is obviously wrong, since we should only concatenate directly nested trait refs. This is solved here by introducing a new `TraitRefBoundary` scope, that we put around the "syntactical" trait refs and basically don't allow concatenation across.
Now, this alone *shouldn't* be a super terrible problem. At least not until you consider the other issue, which is a much more elusive and harder to design a "perfect" fix. A repro can be seen in:
```
use core::future::Future;
async fn handle<F>(slf: &F)
where
F: Fn(&()) -> &mut (dyn for<'a> Future<Output = ()> + Unpin),
{
(slf)(&()).await;
}
```
Notice the `for<'a>` around `Future`. Here, `'a` is unused, so the `for<'a>` Binder gets changed to a `for<>` Binder in the generator witness, but the "local decl" still has it. This has heavy intersections with region anonymization and erasing. Luckily, it's not *super* common to find this unique set of circumstances. It only became apparently because of the first issue mentioned here. However, this *is* still a problem, so I'm leaving #83737 open.
r? `@nikomatsakis`
