Stabilize Arm64EC inline assembly
This stabilizes inline assembly for Arm64EC ("Emulation Compatible").
Corresponding reference PR: https://github.com/rust-lang/reference/pull/1653
---
From the requirements of stabilization mentioned in https://github.com/rust-lang/rust/issues/93335
> Each architecture needs to be reviewed before stabilization:
> - It must have clobber_abi.
Done in https://github.com/rust-lang/rust/pull/131332.
> - It must be possible to clobber every register that is normally clobbered by a function call.
This is possible from the time of the initial implementation.
> - Generally review that the exposed register classes make sense.
The registers available in this target are a subset of those available in the AArch64 inline assembly which is already stable.
The following registers cannot be used in Arm64EC compared to AArch64:
- `x13`, `x14`, `x23`, `x24`, `x28` (register class: `reg`)
- `v[16-31]` (register class: `vreg`)
- `p[0-15]`, `ffr` (clobber-only register class `preg`)
These are disallowed by the ABI (see also [abi docs](https://learn.microsoft.com/en-us/cpp/build/arm64ec-windows-abi-conventions?view=msvc-170#register-mapping) for `reg`/`vreg` and https://github.com/rust-lang/rust/pull/131332#issuecomment-2401189142 for `preg`).
Although not listed in the above requirements, preserves_flags is also implemented and the same as AArch64.
---
cc `@dpaoliello`
r? `@Amanieu`
`@rustbot` label O-windows O-AArch64 +A-inline-assembly +T-lang -T-compiler +needs-fcp
coverage: Restrict empty-span expansion to only cover `{` and `}`
Coverage instrumentation has some tricky code for converting a coverage-relevant `Span` into a set of start/end line/byte-column coordinates that will be embedded in the CGU's coverage metadata.
A big part of this complexity is special code for handling empty spans, which are expanded into non-empty spans (if possible) because LLVM's coverage reporter does not handle empty spans well.
This PR simplifies that code by restricting it to only apply in two specific situations: when the character after the empty span is `{`, or the character before the empty span is `}`.
(As an added benefit, this means that the expanded spans no longer extend awkwardly beyond the end of a physical line, which was common under the previous implementation.)
Along the way, this PR also removes some unhelpful code for dealing with function source code spread across multiple files. Functions currently can't have coverage spans in multiple files, and if that ever changes (e.g. to properly support expansion regions) then this code will need to be completely overhauled anyway.
Dont suggest `use<impl Trait>` when we have an edition-2024-related borrowck issue
#131186 implements some machinery to detect in borrowck when we may have RPIT overcaptures due to edition 2024, and suggests adding `+ use<'a, T>` to go back to the edition 2021 capture rules. However, we weren't filtering out cases when there are APITs in scope.
This PR implements a more sophisticated diagnostic where we will suggest turning any APITs in scope into type parameters, and applies this to both the borrowck error note, and to the `impl_trait_overcaptures` migration lint.
cc #132809
require const_impl_trait gate for all conditional and trait const calls
Alternative to https://github.com/rust-lang/rust/pull/132786.
`@compiler-errors` this is basically what I meant with my proposals. I found it's easier to express this in code than English. ;)
r? `@compiler-errors`
interpret: get_alloc_info: also return mutability
This will be needed for https://github.com/rust-lang/miri/pull/3971
This then tuned into a larger refactor where we introduce a new type for the `get_alloc_info` return data, and we move some code to methods on `GlobalAlloc` to avoid duplicating it between the validity check and `get_alloc_info`.
Stabilize s390x inline assembly
This stabilizes inline assembly for s390x (SystemZ).
Corresponding reference PR: https://github.com/rust-lang/reference/pull/1643
---
From the requirements of stabilization mentioned in https://github.com/rust-lang/rust/issues/93335
> Each architecture needs to be reviewed before stabilization:
> - It must have clobber_abi.
Done in https://github.com/rust-lang/rust/pull/130630.
> - It must be possible to clobber every register that is normally clobbered by a function call.
Done in the PR that added support for clobber_abi.
> - Generally review that the exposed register classes make sense.
The followings can be used as input/output:
- `reg` (`r[0-10]`, `r[12-14]`): General-purpose register
- `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context
This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See https://github.com/rust-lang/rust/pull/119431 for details.
- `freg` (`f[0-15]`): Floating-point register
The followings are clobber-only:
- `vreg` (`v[0-31]`): Vector register
Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (https://github.com/rust-lang/rust/issues/130869 tracks unstable stuff here)
- `areg` (`a[2-15]`): Access register
All of the above register classes except `reg_addr` are needed for `clobber_abi`.
The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM):
- `r11`: frame pointer
- `r15`: stack pointer
- `a0`, `a1`: Reserved for system use
- `c[0-15]` (control register) Reserved by the kernel
Although not listed in the above requirements, `preserves_flags` is implemented in https://github.com/rust-lang/rust/pull/111331.
---
cc ``@uweigand``
r? ``@Amanieu``
``@rustbot`` label +O-SystemZ +A-inline-assembly
Emit warning when calling/declaring functions with unavailable vectors.
On some architectures, vector types may have a different ABI depending on whether the relevant target features are enabled. (The ABI when the feature is disabled is often not specified, but LLVM implements some de-facto ABI.)
As discussed in rust-lang/lang-team#235, this turns out to very easily lead to unsound code.
This commit makes it a post-monomorphization future-incompat warning to declare or call functions using those vector types in a context in which the corresponding target features are disabled, if using an ABI for which the difference is relevant. This ensures that these functions are always called with a consistent ABI.
See the [nomination comment](https://github.com/rust-lang/rust/pull/127731#issuecomment-2288558187) for more discussion.
Part of #116558
r? RalfJung
Rollup of 7 pull requests
Successful merges:
- #132341 (Reject raw lifetime followed by `'`, like regular lifetimes do)
- #132363 (Enforce that raw lifetimes must be valid raw identifiers)
- #132744 (add regression test for #90781)
- #132754 (Simplify the internal API for declaring command-line options)
- #132772 (use `download-rustc="if-unchanged"` as a global default)
- #132774 (Use lld with non-LLVM backends)
- #132799 (Make `Ty::primitive_symbol` recognize `str`)
r? `@ghost`
`@rustbot` modify labels: rollup
Make `Ty::primitive_symbol` recognize `str`
Make `Ty::primitive_symbol` recognize `str`, which makes `str` eligible for the "expected primitive, found local type" (and vice versa) [diagnostic](https://github.com/rust-lang/rust/blob/master/compiler/rustc_trait_selection/src/error_reporting/infer/mod.rs#L1430-L1437) that already exists for other primitives.
<details><summary> diagnostic difference</summary>
```rs
#[allow(non_camel_case_types)]
struct str;
fn foo() {
let _: &str = "hello";
let _: &core::primitive::str = &str;
}
```
`rustc --crate-type lib --edition 2021 a.rs`
Current nightly:
```rs
error[E0308]: mismatched types
--> a.rs:5:19
|
5 | let _: &str = "hello";
| ---- ^^^^^^^ expected `str`, found a different `str`
| |
| expected due to this
|
= note: expected reference `&str`
found reference `&'static str`
error[E0308]: mismatched types
--> a.rs:6:36
|
6 | let _: &core::primitive::str = &str;
| --------------------- ^^^^ expected `str`, found a different `str`
| |
| expected due to this
|
= note: expected reference `&str` (`str`)
found reference `&str` (`str`)
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0308`.
```
With this patch:
```rs
error[E0308]: mismatched types
--> a.rs:5:19
|
5 | let _: &str = "hello";
| ---- ^^^^^^^ expected `str`, found a different `str`
| |
| expected due to this
|
= note: str and `str` have similar names, but are actually distinct types
= note: str is a primitive defined by the language
note: `str` is defined in the current crate
--> a.rs:2:1
|
2 | struct str;
| ^^^^^^^^^^
error[E0308]: mismatched types
--> a.rs:6:36
|
6 | let _: &core::primitive::str = &str;
| --------------------- ^^^^ expected `str`, found a different `str`
| |
| expected due to this
|
= note: str and `str` have similar names, but are actually distinct types
= note: str is a primitive defined by the language
note: `str` is defined in the current crate
--> a.rs:2:1
|
2 | struct str;
| ^^^^^^^^^^
error: aborting due to 2 previous errors
For more information about this error, try `rustc --explain E0308`.
```
</details>
Use lld with non-LLVM backends
On arm64, Cranelift used to produce object files that don't work with lld. This has since been fixed. The GCC backend should always produce object files that work with lld unless lld for whatever reason drops GCC support. Most of the other more niche backends don't use cg_ssa's linker code at all. If they do and don't work with lld, they can always disable lld usage using a cli argument.
Without this commit using cg_clif is by default in a non-trivial amount of cases a perf regression on Linux due to ld.bfd being a fair bit slower than lld. It is possible to explicitly enable it without this commit, but most users are unlikely to do this.
Simplify the internal API for declaring command-line options
The internal APIs for declaring command-line options are old, and intimidatingly complex. This PR replaces them with a single function that takes explicit `stability` and `kind` arguments, making it easier to see how each option is handled, and whether it is treated as stable or unstable.
We also don't appear to have any tests for the output of `rustc --help` and similar, so I've added a run-make test to verify that this PR doesn't change any output. (There is already a similar run-make test for rustdoc's help output.)
---
The librustdoc changes are simply adjusting to updated compiler APIs; no functional change intended.
---
A side-effect of these changes is that rustfmt can once again format the entirety of these option declaration lists, which it was not doing before.
Enforce that raw lifetimes must be valid raw identifiers
Make sure that the identifier part of a raw lifetime is a valid raw identifier. This precludes `'r#_` and all module segment paths for now.
I don't believe this is compelling to support. This was raised by `@ehuss` in https://github.com/rust-lang/reference/pull/1603#discussion_r1822726753 (well, specifically the `'r#_` case), but I don't see why we shouldn't just make it consistent with raw identifiers.
Reject raw lifetime followed by `'`, like regular lifetimes do
See comment. We want to reject cases like `'r#long'id`, which currently gets interpreted as a raw lifetime (`'r#long`) followed by a lifetime (`'id`). This could have alternative lexes, such as an overlong char literal (`'r#long'`) followed by an identifier (`id`). To avoid committing to this in any case, let's reject the whole thing.
`@mattheww,` is this what you were looking for in https://github.com/rust-lang/reference/pull/1603#issuecomment-2339237325? I'd say ignore the details about the specific error message (the fact that this gets reinterpreted as a char literal is 🤷), just that because this causes a lexer error we're effectively saving syntactical space like you wanted.
Add discriminators to DILocations when multiple functions are inlined into a single point.
LLVM does not expect to ever see multiple dbg_declares for the same variable at the same location with different values. proc-macros make it possible for arbitrary code, including multiple calls that get inlined, to happen at any given location in the source code. Add discriminators when that happens so these locations are different to LLVM.
This may interfere with the AddDiscriminators pass in LLVM, which is added by the unstable flag -Zdebug-info-for-profiling.
LLVM does not expect to ever see multiple dbg_declares for the same variable at the same
location with different values. proc-macros make it possible for arbitrary code,
including multiple calls that get inlined, to happen at any given location in the source
code. Add discriminators when that happens so these locations are different to LLVM.
This may interfere with the AddDiscriminators pass in LLVM, which is added by the
unstable flag -Zdebug-info-for-profiling.
Fixes#131944
Subtree sync for rustc_codegen_cranelift
Apart from a perf optimization for some crates (https://github.com/rust-lang/rustc_codegen_cranelift/pull/1541) not much changed this time as the last sync was less than a week ago.
r? `@ghost`
`@rustbot` label +A-codegen +A-cranelift +T-compiler
Rollup of 5 pull requests
Successful merges:
- #132552 (Add v9, v8plus, and leoncasa target feature to sparc and use v8plus in create_object_file)
- #132745 (pointee_info_at: fix logic for recursing into enums)
- #132777 (try_question_mark_nop: update test for LLVM 20)
- #132785 (rustc_target: more target string fixes for LLVM 20)
- #132794 (Use a separate dir for r-a builds consistently in helix config)
r? `@ghost`
`@rustbot` modify labels: rollup
rustc_target: more target string fixes for LLVM 20
LLVM continues to clean these up, and we continue to make this consistent. This is similar to 9caced7bad and e985396145.
`@rustbot` label: +llvm-main
pointee_info_at: fix logic for recursing into enums
Fixes https://github.com/rust-lang/rust/issues/131834
The logic in `pointee_info_at` was likely written at a time when the null pointer optimization was the *only* enum layout optimization -- and as `Variant::Multiple` kept getting expanded, nobody noticed that the logic is now unsound.
The job of this function is to figure out whether there is a dereferenceable-or-null and aligned pointer at a given offset inside a type. So when we recurse into a multi-variant enum, we better make sure that all the other enum variants must be null! This is the part that was forgotten, and this PR adds it.
The reason this didn't explode in many ways so far is that our references only have 1 niche value (null), so it's not possible on stable to have a multi-variant enum with a dereferenceable pointer and other enum variants that are not null. But with `rustc_layout_scalar_valid_range` attributes one can force such a layout, and if `@the8472's` work on alignment niches ever lands, that will make this possible on stable.
Trim and tidy includes in `rustc_llvm`
These includes tend to accumulate over time, and are usually only removed when something breaks in a new LLVM version, so it's nice to clean them up manually once in a while.
General strategy used for this PR:
- Remove all includes from `LLVMWrapper.h` that aren't needed by the header itself, transplanting them to individual source files as necessary.
- For each source file, temporarily remove each include if doing so doesn't cause a compile error.
- If a “required” include looks like it shouldn't be needed, try replacing it with its sub-includes, then trim that list.
- After doing all of the above, go back and re-add any removed include if the file does actually use things defined in that header, even if the header happens to also be included by something else.
use verbose for path separator suggestion
A single `-` of suggestion underlining that is adjacent to a much more significant `^^^` underlying of the LHS path component is hard to distinguish. IMO this presents much more cleanly when it's verbose, especially because it's a *replacment* suggestion.
r? estebank
Don't suggest `.into_iter()` on iterators
This makes the the suggestion to call `.into_iter()` only consider unsatisfied `Iterator` bounds for the receiver type itself. That way, it ignores predicates generated by trying to auto-ref the receiver (the result of which usually won't implement `Iterator`).
Fixes#127511
Unfortunately, the error in that case is still confusing: it labels `Iterator` as an unsatisfied bound because `&impl Iterator: Iterator` can't be satisfied, despite that not being required or helpful. I'd like to handle that in a separate PR. ~~I'm hoping fixing #124802 will fix it too.~~ It doesn't look connected to that issue. Still, I think it'd be clearest to visually distinguish unsatisfied predicates from different attempts at `pick_method`; I'll make a PR for that soon.
Get rid of `check_opaque_type_well_formed`
Instead, replicate it by improving the span of the opaque in `check_opaque_meets_bounds`.
This has two consequences:
1. We now prefer "concrete type differs" errors, since we'll hit those first before we check the opaque is WF.
2. Spans have gotten slightly worse.
Specifically, (2.) could be improved by adding a new obligation cause that explains that the definition's environment has stronger assumptions than the declaration.
r? lcnr
Make `RustString` an extern type to avoid `improper_ctypes` warnings
Currently, any FFI function that uses `&RustString` needs to also add `#[ignore(improper_ctypes)]` to silence a warning.
The warning is not _completely_ bogus, because `RustString` contains `Vec<u8>` and therefore does not have a guaranteed layout. But we have no way of telling the lint that this doesn't matter, because the C++ code only uses that pointer opaquely and never relies on its underlying layout.
Ideally there would be some way to silence `improper_ctypes` at the type-definition site. But because there isn't, casting to and from a separate extern type is better than having to annotate every single use site.
On arm64, Cranelift used to produce object files that don't work with
lld. This has since been fixed. The GCC backend should always produce
object files that work with lld unless lld for whatever reason drops GCC
support. Most of the other more niche backends don't use cg_ssa's linker
code at all. If they do and don't work with lld, they can always disable
lld usage using a cli argument.
Without this commit using cg_clif is by default in a non-trivial amount
of cases a perf regression on Linux due to ld.bfd being a fair bit
slower than lld. It is possible to explicitly enable it without this
commit, but most users are unlikely to do this.
remove support for rustc_safe_intrinsic attribute; use rustc_intrinsic functions instead
This brings us one step closer towards removing support for `extern "rust-intrinsic"` blocks, in favor of `#[rustc_intrinsic]` functions.
Also move `#[rustc_intrinsic]` under the `intrinsics` feature gate, to match the `extern "rust-intrinsic"` style.
Functions currently can't have mappings in multiple files, and if that ever
changes (e.g. to properly support expansion regions), this code will need to be
completely overhauled anyway.
