Move expansion of query macros in rustc_middle to rustc_middle::query
This moves the expansion of `define_callbacks!` and `define_feedable!` from `rustc_middle::ty::query` to `rustc_middle::query`.
This means that types used in queries are both imported and used in `rustc_middle::query` instead of being split between these modules. It also decouples `rustc_middle::ty::query` further from `rustc_middle` which is helpful since we want to move `rustc_middle::ty::query` to the query system crates.
Simplify find_width_of_character_at_span.
This makes `find_width_of_character_at_span` simpler and more robust against bad spans.
Fixes (but does not close, per beta policy) https://github.com/rust-lang/rust/issues/111485
add util function to TokenStream to eliminate some clones
Another proposed change in the same vein as #111492 trying to get rid of some clones.
This adds a TokenStream helper function so that rustdoc can directly get a chunks iterator over the underlying token trees so that it no longer needs the clones and vec.
Start node has no immediate dominator
Change the immediate_dominator return type to Option, and use None to
indicate that node has no immediate dominator.
Also fix the issue where the start node would be returned as its own
immediate dominator.
Fix the `FileEncoder` buffer size.
It allows a variable size, but in practice we always use the default of 8192 bytes. This commit fixes it to that size, which makes things slightly faster because the size can be hard-wired in generated code.
The commit also:
- Rearranges some buffer capacity checks so they're all in the same form (`x > BUFSIZE`).
- Removes some buffer capacity assertions and comments about them. With an 8192 byte buffer, we're not in any danger of overflowing a `usize`.
r? `@WaffleLapkin`
After seeing a `0`, if it's followed by any of `[0-9]`, `_`, `.`, `e`,
or `E`, we consume all the digits. But in the `.`, `e` and `E` cases
this is pointless because we know there aren't any digits.
Rename const error methods for consistency
renames `ty::Const`'s methods for creating a `ConstKind::Error` to be in the same naming style as `ty::Ty`'s equivalent methods.
r? `@BoxyUwU`
When constant evaluation fails because its MIR is tainted by errors,
suppress note indicating that erroneous constant was used, since those
errors have to be fixed regardless of the constant being used or not.
It allows a variable size, but in practice we always use the default of
8192 bytes. This commit fixes it to that size, which makes things
slightly faster because the size can be hard-wired in generated code.
The commit also:
- Rearranges some buffer capacity checks so they're all in the same form
(`x > BUFSIZE`).
- Removes some buffer capacity assertions and comments about them. With
an 8192 byte buffer, we're not in any danger of overflowing a `usize`.
Partially reverts 109612, as after 109993 these aren't actually equivalent any more, and I'm no longer confident this was ever an improvement in the first place.
Change the immediate_dominator return type to Option, and use None to
indicate that node has no immediate dominator.
Also fix the issue where the start node would be returned as its own
immediate dominator.
Use dynamic dispatch for queries
This replaces most concrete query values `V` with `MaybeUninit<[u8; { size_of::<V>() }]>` reducing the code instantiated by queries. The compile time of `rustc_query_impl` is reduced by 27%. It is an alternative to https://github.com/rust-lang/rust/pull/107937 which uses unstable const generics while this uses a `EraseType` trait which maps query values to their erased variant.
This is achieved by introducing an `Erased` type which does sanity check with `cfg(debug_assertions)`. The query caches gets instantiated with these erased types leaving the code in `rustc_query_system` unaware of them. `rustc_query_system` is changed to use instances of `QueryConfig` so that `rustc_query_impl` can pass in `DynamicConfig` which holds a pointer to a virtual table.
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check</td><td align="right">1.7055s</td><td align="right">1.6949s</td><td align="right"> -0.62%</td></tr><tr><td>🟣 <b>hyper</b>:check</td><td align="right">0.2547s</td><td align="right">0.2528s</td><td align="right"> -0.73%</td></tr><tr><td>🟣 <b>regex</b>:check</td><td align="right">0.9590s</td><td align="right">0.9553s</td><td align="right"> -0.39%</td></tr><tr><td>🟣 <b>syn</b>:check</td><td align="right">1.5457s</td><td align="right">1.5440s</td><td align="right"> -0.11%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check</td><td align="right">5.9092s</td><td align="right">5.9009s</td><td align="right"> -0.14%</td></tr><tr><td>Total</td><td align="right">10.3741s</td><td align="right">10.3479s</td><td align="right"> -0.25%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9960s</td><td align="right"> -0.40%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:initial</td><td align="right">2.0605s</td><td align="right">2.0575s</td><td align="right"> -0.15%</td></tr><tr><td>🟣 <b>hyper</b>:check:initial</td><td align="right">0.3218s</td><td align="right">0.3216s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>regex</b>:check:initial</td><td align="right">1.1848s</td><td align="right">1.1839s</td><td align="right"> -0.07%</td></tr><tr><td>🟣 <b>syn</b>:check:initial</td><td align="right">1.9409s</td><td align="right">1.9376s</td><td align="right"> -0.17%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:initial</td><td align="right">7.3105s</td><td align="right">7.2928s</td><td align="right"> -0.24%</td></tr><tr><td>Total</td><td align="right">12.8185s</td><td align="right">12.7935s</td><td align="right"> -0.20%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">0.9986s</td><td align="right"> -0.14%</td></tr></table>
<table><tr><td rowspan="2">Benchmark</td><td colspan="1"><b>Before</b></th><td colspan="2"><b>After</b></th></tr><tr><td align="right">Time</td><td align="right">Time</td><td align="right">%</th></tr><tr><td>🟣 <b>clap</b>:check:unchanged</td><td align="right">0.4606s</td><td align="right">0.4617s</td><td align="right"> 0.24%</td></tr><tr><td>🟣 <b>hyper</b>:check:unchanged</td><td align="right">0.1335s</td><td align="right">0.1336s</td><td align="right"> 0.08%</td></tr><tr><td>🟣 <b>regex</b>:check:unchanged</td><td align="right">0.3324s</td><td align="right">0.3346s</td><td align="right"> 0.65%</td></tr><tr><td>🟣 <b>syn</b>:check:unchanged</td><td align="right">0.6268s</td><td align="right">0.6307s</td><td align="right"> 0.64%</td></tr><tr><td>🟣 <b>syntex_syntax</b>:check:unchanged</td><td align="right">1.8248s</td><td align="right">1.8508s</td><td align="right">💔 1.43%</td></tr><tr><td>Total</td><td align="right">3.3779s</td><td align="right">3.4113s</td><td align="right"> 0.99%</td></tr><tr><td>Summary</td><td align="right">1.0000s</td><td align="right">1.0061s</td><td align="right"> 0.61%</td></tr></table>
It's based on https://github.com/rust-lang/rust/pull/108167.
r? `@cjgillot`
allow mutating function args through `&raw const`
Fixes https://github.com/rust-lang/rust/issues/111502 by "turning off the sketchy optimization while we figure out if this is ok", like `@JakobDegen` said.
The first commit in this PR removes some suspicious looking logic from the same method, but should have no functional changes, since it doesn't modify the `context` outside of the method. Best reviewed commit by commit.
r? opsem
Allow MIR debuginfo to point to a variable's address
MIR optimizations currently do not to operate on borrowed locals.
When enabling #106285, many borrows will be left as-is because they are used in debuginfo. This pass allows to replace this pattern directly in MIR debuginfo:
```rust
a => _1
_1 = &raw? mut? _2
```
becomes
```rust
a => &_2
// No statement to borrow _2.
```
This pass is implemented as a drive-by in ReferencePropagation MIR pass.
This transformation allows following following MIR opts to treat _2 as an unborrowed local, and optimize it as such, even in builds with debuginfo.
In codegen, when encountering `a => &..&_2`, we create a list of allocas:
```llvm
store ptr %_2.dbg.spill, ptr %a.ref0.dbg.spill
store ptr %a.ref0.dbg.spill, ptr %a.ref1.dbg.spill
...
call void `@llvm.dbg.declare(metadata` ptr %a.ref{n}.dbg.spill, /* ... */)
```
Caveat: this transformation looses the exact type, we do not differentiate `a` as a immutable, mutable reference or a raw pointer. Everything is declared to `*mut` to codegen. I'm not convinced this is a blocker.
Align unsized locals
Allocate an extra space for unsized locals and manually align the storage, since alloca doesn't support dynamic alignment.
Fixes#71416.
Fixes#71695.
Introduce `DynSend` and `DynSync` auto trait for parallel compiler
part of parallel-rustc #101566
This PR introduces `DynSend / DynSync` trait and `FromDyn / IntoDyn` structure in rustc_data_structure::marker. `FromDyn` can dynamically check data structures for thread safety when switching to parallel environments (such as calling `par_for_each_in`). This happens only when `-Z threads > 1` so it doesn't affect single-threaded mode's compile efficiency.
r? `@cjgillot`
Apply simulate-remapped-rust-src-base even if remap-debuginfo is set in config.toml
This is really a mess. Here is the situation before this change:
- UI tests depend on not having `rust-src` available. In particular, <3f374128ee/tests/ui/tuple/wrong_argument_ice.stderr (L7-L8)> is depending on the `note` being a single line and not showing the source code.
- When `download-rustc` is disabled, we pass `-Zsimulate-remapped-rust-src-base=/rustc/FAKE_PREFIX` `-Ztranslate-remapped-path-to-local-path=no`, which changes the diagnostic to something like ` --> /rustc/FAKE_PREFIX/library/alloc/src/collections/vec_deque/mod.rs:1657:12`
- When `download-rustc` is enabled, we still pass those flags, but they no longer have an effect. Instead rustc emits diagnostic paths like this: ` --> /rustc/39c6804b92aa202369e402525cee329556bc1db0/library/alloc/src/collections/vec_deque/mod.rs:1657:12`. Notice how there's a real commit and not `FAKE_PREFIX`. This happens because we set `CFG_VIRTUAL_RUST_SOURCE_BASE_DIR` during bootstrapping for CI artifacts, and rustc previously didn't allow for `simulate-remapped` to affect paths that had already been remapped.
- Pietro noticed this and decided the right thing was to normalize `/rustc/<commit>` to `$SRC_DIR` in compiletest: 470423c3d2
- After my change to `x test core`, which rebuilds stage 2 std from source so `build/stage2-std` and `build/stage2` use the same `.rlib` metadata, the compiler suddenly notices it has sources for `std` available and prints those in the diagnostic, causing the test to fail.
This changes `simulate-remapped-rust-src-base` to support remapping paths that have already been remapped, unblocking download-rustc.
Unfortunately, although this fixes the specific problem for
download-rustc, it doesn't seem to affect all the compiler's
diagnostics. In particular, various `mir-opt` tests are failing to
respect `simulate-remapped-path-prefix` (I looked into fixing this but
it seems non-trivial). As a result, we can't remove the normalization in
compiletest that maps `/rustc/<commit>` to `$SRC_DIR`, so this change is
currently untested anywhere except locally.
You can test this locally yourself by setting `rust.remap-debuginfo = true`, running any UI test with `ERROR` annotations, then rerunning the test manually with a dev toolchain to verify it prints `/rustc/FAKE_PREFIX`, not `/rustc/1.71.0`.
Helps with https://github.com/rust-lang/rust/issues/110352.
Encode `VariantIdx` so we can decode ADT variants in the right order
As far as I can tell, we don't guarantee anything about the ordering of `DefId`s and module children...
The code that motivated this PR (#111483) looks something like:
```rust
#[derive(Protocol)]
pub enum Data {
#[protocol(discriminator(0x00))]
Disconnect(Disconnect),
EncryptionRequest,
/* more variants... */
}
```
The specific macro ([`protocol`](https://github.com/dylanmckay/protocol)) doesn't really matter, but as far as I can tell (from calls to `build_reduced_graph`), the presence of that `#[protocol(..)]` helper attribute causes the def-id of the `Disconnect` enum variant to be collected *after* its siblings, and it shows up after the other variants in `module_children`.
When we decode the variants for `Data` in a child crate (an example test, in this case), this means that the `Disconnect` variant is moved to the end of the variants list, and all of the other variants now have incorrect relative discriminant data, causing the ICE.
This PR fixes this by sorting manually by variant index after they are decoded. I guess there are alternative ways of fixing this, such as not reusing `module_children_non_reexports` to encode the order-sensitive ADT variants, or to do some sorting in `rustc_resolve`... but none of those seemed particularly satisfying either.
~I really struggled to create a reproduction here -- it required at least 3 crates, one of which is a proc macro, and then some code to actually compute discriminants in the child crate... Needless to say, I failed to repro this in a test, but I can confirm that it fixes the regression in #111483.~ Test exists now.
r? `@petrochenkov` but feel free to reassign. ~Again, sorry for no test, but I hope the explanation at least suggests why a fix like this is likely necessary.~ Feedback is welcome.
Fix data race in llvm source code coverage
Fixes#91092 .
Before this patch, increment of counters for code coverage looks like this:
```
movq .L__profc__RNvCsd6wgJFC5r19_3lib6bugaga+8(%rip), %rax
addq $1, %rax
movq %rax, .L__profc__RNvCsd6wgJFC5r19_3lib6bugaga+8(%rip)
```
after this patch:
```
lock incq .L__profc__RNvCs3JgIB2SjHh2_3lib6bugaga+8(%rip)
```
Note user-facing types of coercion failure
When coercing, for example, `Box<A>` into `Box<dyn B>`, make sure that any failure notes mention *those* specific types, rather than mentioning inner types, like "the cast from `A` to `dyn B`".
I expect end-users are often confused when we skip layers of types and only mention the "innermost" part of a coercion, especially when other notes point at HIR, e.g. #111406.
Add support for `cfg(overflow_checks)`
This PR adds support for detecting if overflow checks are enabled in similar fashion as `debug_assertions` are detected. Possible use-case of this, for example, if we want to use checked integer casts in builds with overflow checks, e.g.
```rust
pub fn cast(val: usize)->u16 {
if cfg!(overflow_checks) {
val.try_into().unwrap()
}
else{
vas as _
}
}
```
Resolves#91130.
Require impl Trait in associated types to appear in method signatures
This implements the limited version of TAIT that was proposed in https://github.com/rust-lang/rust/issues/107645#issuecomment-1477899536
Similar to `impl Trait` in return types, `impl Trait` in associated types may only be used within the impl block which it is a part of. To make everything simpler and forward compatible to getting desugared to a plain type alias impl trait in the future, we're requiring that any associated functions or constants that want to register hidden types must be using the associated type in their signature (type of the constant or argument/return type of the associated method. Where bounds mentioning the associated type are ignored).
We have preexisting tests checking that this works transitively across multiple associated types in situations like
```rust
impl Foo for Bar {
type A = impl Trait;
type B = impl Iterator<Item = Self::A>;
fn foo() -> Self::B { ...... }
}
```
Reduce BitSet size used in `Borrows` dataflow analysis
It looks like it is not needed to multiply the number of borrows by 2. Bits greater than `self.borrow_set.len()` are never set in this bitset. This should decrease the memory usage by an epsilon.
Uplift `clippy::{drop,forget}_{ref,copy}` lints
This PR aims at uplifting the `clippy::drop_ref`, `clippy::drop_copy`, `clippy::forget_ref` and `clippy::forget_copy` lints.
Those lints are/were declared in the correctness category of clippy because they lint on useless and most probably is not what the developer wanted.
## `drop_ref` and `forget_ref`
The `drop_ref` and `forget_ref` lint checks for calls to `std::mem::drop` or `std::mem::forget` with a reference instead of an owned value.
### Example
```rust
let mut lock_guard = mutex.lock();
std::mem::drop(&lock_guard) // Should have been drop(lock_guard), mutex
// still locked
operation_that_requires_mutex_to_be_unlocked();
```
### Explanation
Calling `drop` or `forget` on a reference will only drop the reference itself, which is a no-op. It will not call the `drop` or `forget` method on the underlying referenced value, which is likely what was intended.
## `drop_copy` and `forget_copy`
The `drop_copy` and `forget_copy` lint checks for calls to `std::mem::forget` or `std::mem::drop` with a value that derives the Copy trait.
### Example
```rust
let x: i32 = 42; // i32 implements Copy
std::mem::forget(x) // A copy of x is passed to the function, leaving the
// original unaffected
```
### Explanation
Calling `std::mem::forget` [does nothing for types that implement Copy](https://doc.rust-lang.org/std/mem/fn.drop.html) since the value will be copied and moved into the function on invocation.
-----
Followed the instructions for uplift a clippy describe here: https://github.com/rust-lang/rust/pull/99696#pullrequestreview-1134072751
cc `@m-ou-se` (as T-libs-api leader because the uplifting was discussed in a recent meeting)
use by ref TokenTree iterator to avoid a few clones
Just a handful of swaps from the by-value cursor to by-ref cursor so as to avoid some unnecessary clones.
I've been doing some analysis on internal cleanup opportunities within rustfmt and as part of that yak-shave I found myself perusing broader token stream and tree usage (which we use within rustfmt). As reflected in some inline comments on the cursor structs (not part of this diff), there's probably many other such cases throughout the code, but figured I'd start small with these while I had the time. May take a look at the other sites in the future
Don't ICE in layout computation for placeholder types
We use `layout_of` for the built-in `PointerLike` trait to check if a type can be coerced to a `dyn*`.
Since the new solver canonicalizes parameter types to placeholders, that code needs to be able to treat placeholders like params, and for the most part it does, **except** for a call to `is_trivially_sized`. This PR fixes that.
Verify copies of mutable pointers in 2 stages in ReferencePropagation
Fixes#111422
In the first stage, we mark the copies as reborrows, to be checked later.
In the second stage, we walk the reborrow chains to verify that all stages are fully replacable.
The replacement itself mirrors the check, and iterates through the reborrow chain.
r? ``````@RalfJung``````
cc ``````@JakobDegen``````
bump windows crate 0.46 -> 0.48
This drops duped version of crate(0.46), reduces `rustc_driver.dll` ~800kb and reduces exported functions number from 26k to 22k.
Also while here, added `tidy-alphabetical` sorting to lists in tidy allowed lists.
CFI: Fix SIGILL reached via trait objects
Fix#106547 by transforming the concrete self into a reference to a trait object before emitting type metadata identifiers for trait methods.
use implied bounds when checking opaque types
During opaque type inference, we check for the well-formedness of the hidden type in the opaque type's own environment, not the one of the defining site, which are different in the case of TAIT.
However in the case of associated-type-impl-trait, we don't use implied bounds from the impl header. This caused us to reject the following:
```rust
trait Service<Req> {
type Output;
fn call(req: Req) -> Self::Output;
}
impl<'a, Req> Service<&'a Req> for u8 {
type Output= impl Sized; // we can't prove WF of hidden type `WF(&'a Req)` although it's implied by the impl
//~^ ERROR type parameter Req doesn't live long enough
fn call(req: &'a Req) -> Self::Output {
req
}
}
```
although adding an explicit bound would make it pass:
```diff
- impl<'a, Req> Service<&'a Req> for u8 {
+ impl<'a, Req> Service<&'a Req> for u8 where Req: 'a, {
```
I believe it should pass as we already allow the concrete type to be used:
```diff
impl<'a, Req> Service<&'a Req> for u8 {
- type Output= impl Sized;
+ type Output= &'a Req;
```
Fixes#95922
Builds on #105982
cc ``@lcnr`` (because implied bounds)
r? ``@oli-obk``
This PR adds support for detecting if overflow checks are enabled in similar fashion as debug_assertions are detected.
Possible use-case of this, for example, if we want to use checked integer casts in builds with overflow checks, e.g.
```rust
pub fn cast(val: usize)->u16 {
if cfg!(overflow_checks) {
val.try_into().unwrap()
}
else{
vas as _
}
}
```
Resolves#91130.
Tracking issue: #111466.
Shrink `SelectionError` a lot
`SelectionError` used to be 80 bytes (on 64 bit). That's quite big. Especially because the selection cache contained `Result<_, SelectionError>. The Ok type is only 32 bytes, so the 80 bytes significantly inflate the size of the cache.
Most variants of the `SelectionError` seem to be hard errors, only `Unimplemented` shows up in practice (for cranelift-codegen, it occupies 23.4% of all cache entries). We can just box away the biggest variant, `OutputTypeParameterMismatch`, to get the size down to 16 bytes, well within the size of the Ok type inside the cache.
Isolate coverage FFI type layouts from their underlying LLVM C++ types
I noticed that several of the types used to send coverage information through FFI are not properly isolated from the layout of their corresponding C++ types in the LLVM API.
This PR adds more explicitly-defined FFI struct/enum types in `CoverageMappingWrapper.cpp`, so that Rust source files in `rustc_codegen_ssa` and `rustc_codegen_llvm` aren't directly exposed to LLVM C++ types.
Fix mishandled `--check-cfg` arguments order
This PR fixes a bug in `--check-cfg` where the order of `--check-cfg=names(a)` and `--check-cfg=values(a,…)` would trip the compiler.
Fixes https://github.com/rust-lang/rust/issues/111291
cc `@taiki-e` `@petrochenkov`
Prevent ICE with broken borrow in closure
r? `@Nilstrieb`
Fixes#108683
This solution isn't ideal, I'm hoping to find a way to continue compilation without ICEing.
Optimize dataflow-const-prop place-tracking infra
Optimization opportunities found while investigating https://github.com/rust-lang/rust/pull/110719
Computing places breadth-first ensures that we create short projections before deep projections, since the former are more likely to be propagated.
The most relevant is the pre-computation of flooded places. Callgrind showed `flood_*` methods and especially `preorder_preinvoke` were especially hot. This PR attempts to pre-compute the set of `ValueIndex` that `preorder_invoke` would visit.
Using this information, we make some `PlaceIndex` inaccessible when they contain no `ValueIndex`, allowing to skip computations for those places.
cc `@jachris` as original author
