rustc_target: Add various aarch64 features
Add various aarch64 features already supported by LLVM and Linux.
Additionally include some comment fixes to ensure consistency of feature names with the Arm ARM.
Compiler support for features added to stdarch by https://github.com/rust-lang/stdarch/pull/1614.
Tracking issue for unstable aarch64 features is https://github.com/rust-lang/rust/issues/127764.
List of added features:
- FEAT_CSSC
- FEAT_ECV
- FEAT_FAMINMAX
- FEAT_FLAGM2
- FEAT_FP8
- FEAT_FP8DOT2
- FEAT_FP8DOT4
- FEAT_FP8FMA
- FEAT_HBC
- FEAT_LSE128
- FEAT_LSE2
- FEAT_LUT
- FEAT_MOPS
- FEAT_LRCPC3
- FEAT_SVE_B16B16
- FEAT_SVE2p1
- FEAT_WFxT
- FEAT_SME
- FEAT_SME_F16F16
- FEAT_SME_F64F64
- FEAT_SME_F8F16
- FEAT_SME_F8F32
- FEAT_SME_FA64
- FEAT_SME_I16I64
- FEAT_SME_LUTv2
- FEAT_SME2
- FEAT_SME2p1
- FEAT_SSVE_FP8DOT2
- FEAT_SSVE_FP8DOT4
- FEAT_SSVE_FP8FMA
FEAT_FPMR is added in the first commit and then removed in a separate one to highlight it being removed from upstream LLVM 19. The intention is for it to be detectable at runtime through stdarch but not have a corresponding Rust compile-time feature.
add repr to the allowlist for naked functions
Fixes#129412 (combining unstable features #90957 (`#![feature(naked_functions)]`) and #82232 (`#![feature(fn_align)]`)
Set the cfi-normalize-integers and kcfi-offset module flags when
Control-Flow Integrity sanitizers are used, so functions generated by
the LLVM backend use the same CFI/KCFI options as rustc.
cfi-normalize-integers tells LLVM to also use integer normalization
for generated functions when -Zsanitizer-cfi-normalize-integers is
used.
kcfi-offset specifies the number of prefix nops between the KCFI
type hash and the function entry when -Z patchable-function-entry is
used. Note that LLVM assumes all indirectly callable functions use the
same number of prefix NOPs with -Zsanitizer=kcfi.
Unconditionally allow shadow call-stack sanitizer for AArch64
It is possible to do so whenever `-Z fixed-x18` is applied.
cc ``@Darksonn`` for context
The reasoning is that, as soon as reservation on `x18` is forced through the flag `fixed-x18`, on AArch64 the option to instrument with [Shadow Call Stack sanitizer](https://clang.llvm.org/docs/ShadowCallStack.html) is then applicable regardless of the target configuration.
At the every least, we would like to relax the restriction on specifically `aarch64-unknonw-none`. For this option, we can include a documentation change saying that users of compiled objects need to ensure that they are linked to runtime with Shadow Call Stack instrumentation support.
Related: #121972
Rework MIR inlining debuginfo so function parameters show up in debuggers.
Line numbers of multiply-inlined functions were fixed in #114643 by using a single DISubprogram. That, however, triggered assertions because parameters weren't deduplicated. The "solution" to that in #115417 was to insert a DILexicalScope below the DISubprogram and parent all of the parameters to that scope. That fixed the assertion, but debuggers (including gdb and lldb) don't recognize variables that are not parented to the subprogram itself as parameters, even if they are emitted with DW_TAG_formal_parameter.
Consider the program:
```rust
use std::env;
#[inline(always)]
fn square(n: i32) -> i32 {
n * n
}
#[inline(never)]
fn square_no_inline(n: i32) -> i32 {
n * n
}
fn main() {
let x = square(env::vars().count() as i32);
let y = square_no_inline(env::vars().count() as i32);
println!("{x} == {y}");
}
```
When making a release build with debug=2 and rustc 1.82.0-nightly (8b3870784 2024-08-07)
```
(gdb) r
Starting program: /ephemeral/tmp/target/release/tmp [Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, tmp::square () at src/main.rs:5
5 n * n
(gdb) info args
No arguments.
(gdb) info locals
n = 31
(gdb) c
Continuing.
Breakpoint 2, tmp::square_no_inline (n=31) at src/main.rs:10
10 n * n
(gdb) info args
n = 31
(gdb) info locals
No locals.
```
This issue is particularly annoying because it removes arguments from stack traces.
The DWARF for the inlined function looks like this:
```
< 2><0x00002132 GOFF=0x00002132> DW_TAG_subprogram
DW_AT_linkage_name _ZN3tmp6square17hc507052ff3d2a488E
DW_AT_name square
DW_AT_decl_file 0x0000000f /ephemeral/tmp/src/main.rs
DW_AT_decl_line 0x00000004
DW_AT_type 0x00001a56<.debug_info+0x00001a56>
DW_AT_inline DW_INL_inlined
< 3><0x00002142 GOFF=0x00002142> DW_TAG_lexical_block
< 4><0x00002143 GOFF=0x00002143> DW_TAG_formal_parameter
DW_AT_name n
DW_AT_decl_file 0x0000000f /ephemeral/tmp/src/main.rs
DW_AT_decl_line 0x00000004
DW_AT_type 0x00001a56<.debug_info+0x00001a56>
< 4><0x0000214e GOFF=0x0000214e> DW_TAG_null
< 3><0x0000214f GOFF=0x0000214f> DW_TAG_null
```
That DW_TAG_lexical_block inhibits every debugger I've tested from recognizing 'n' as a parameter.
This patch removes the additional lexical scope. Parameters can be easily deduplicated by a tuple of their scope and the argument index, at the trivial cost of taking a Hash + Eq bound on DIScope.
Line numbers of multiply-inlined functions were fixed in #114643 by using a
single DISubprogram. That, however, triggered assertions because parameters
weren't deduplicated. The "solution" to that in #115417 was to insert a
DILexicalScope below the DISubprogram and parent all of the parameters to that
scope. That fixed the assertion, but debuggers (including gdb and lldb) don't
recognize variables that are not parented to the subprogram itself as parameters,
even if they are emitted with DW_TAG_formal_parameter.
Consider the program:
use std::env;
fn square(n: i32) -> i32 {
n * n
}
fn square_no_inline(n: i32) -> i32 {
n * n
}
fn main() {
let x = square(env::vars().count() as i32);
let y = square_no_inline(env::vars().count() as i32);
println!("{x} == {y}");
}
When making a release build with debug=2 and rustc 1.82.0-nightly (8b3870784 2024-08-07)
(gdb) r
Starting program: /ephemeral/tmp/target/release/tmp
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, tmp::square () at src/main.rs:5
5 n * n
(gdb) info args
No arguments.
(gdb) info locals
n = 31
(gdb) c
Continuing.
Breakpoint 2, tmp::square_no_inline (n=31) at src/main.rs:10
10 n * n
(gdb) info args
n = 31
(gdb) info locals
No locals.
This issue is particularly annoying because it removes arguments from stack traces.
The DWARF for the inlined function looks like this:
< 2><0x00002132 GOFF=0x00002132> DW_TAG_subprogram
DW_AT_linkage_name _ZN3tmp6square17hc507052ff3d2a488E
DW_AT_name square
DW_AT_decl_file 0x0000000f /ephemeral/tmp/src/main.rs
DW_AT_decl_line 0x00000004
DW_AT_type 0x00001a56<.debug_info+0x00001a56>
DW_AT_inline DW_INL_inlined
< 3><0x00002142 GOFF=0x00002142> DW_TAG_lexical_block
< 4><0x00002143 GOFF=0x00002143> DW_TAG_formal_parameter
DW_AT_name n
DW_AT_decl_file 0x0000000f /ephemeral/tmp/src/main.rs
DW_AT_decl_line 0x00000004
DW_AT_type 0x00001a56<.debug_info+0x00001a56>
< 4><0x0000214e GOFF=0x0000214e> DW_TAG_null
< 3><0x0000214f GOFF=0x0000214f> DW_TAG_null
That DW_TAG_lexical_block inhibits every debugger I've tested from recognizing
'n' as a parameter.
This patch removes the additional lexical scope. Parameters can be easily
deduplicated by a tuple of their scope and the argument index, at the trivial
cost of taking a Hash + Eq bound on DIScope.
const vector passed through to codegen
This allows constant vectors using a repr(simd) type to be propagated
through to the backend by reusing the functionality used to do a similar
thing for the simd_shuffle intrinsic
#118209
r? RalfJung
nontemporal_store: make sure that the intrinsic is truly just a hint
The `!nontemporal` flag for stores in LLVM *sounds* like it is just a hint, but actually, it is not -- at least on x86, non-temporal stores need very special treatment by the programmer or else the Rust memory model breaks down. LLVM still treats these stores as-if they were normal stores for optimizations, which is [highly dubious](https://github.com/llvm/llvm-project/issues/64521). Let's avoid all that dubiousness by making our own non-temporal stores be truly just a hint, which is possible on some targets (e.g. ARM). On all other targets, non-temporal stores become regular stores.
~~Blocked on https://github.com/rust-lang/stdarch/pull/1541 propagating to the rustc repo, to make sure the `_mm_stream` intrinsics are unaffected by this change.~~
Fixes https://github.com/rust-lang/rust/issues/114582
Cc `@Amanieu` `@workingjubilee`
fix: #128855 Ensure `Guard`'s `drop` method is removed at `opt-level=s` for `…
fix: #128855
…Copy` types
Added `#[inline]` to the `drop` method in the `Guard` implementation to ensure that the method is removed by the compiler at optimization level `opt-level=s` for `Copy` types. This change aims to align the method's behavior with optimization expectations and ensure it does not affect performance.
r? `@scottmcm`
This commit adds a new test file 'array-from_fn.rs' to the codegen test suite.
The test checks the behavior of std::array::from_fn under different optimization levels:
1. At opt-level=0 (debug build), it verifies that the core::array::Guard
is present in the generated code.
2. At opt-level=s (size optimization), it ensures that the Guard is
optimized out.
This test helps ensure that the compiler correctly optimizes array::from_fn
calls in release builds while maintaining safety checks in debug builds.
Rollup of 8 pull requests
Successful merges:
- #128221 (Add implied target features to target_feature attribute)
- #128261 (impl `Default` for collection iterators that don't already have it)
- #128353 (Change generate-copyright to generate HTML, with cargo dependencies included)
- #128679 (codegen: better centralize function declaration attribute computation)
- #128732 (make `import.vis` is immutable)
- #128755 (Integrate crlf directly into related test file instead via of .gitattributes)
- #128772 (rustc_codegen_ssa: Set architecture for object crate for 32-bit SPARC)
- #128782 (unused_parens: do not lint against parens around &raw)
r? `@ghost`
`@rustbot` modify labels: rollup
Rollup of 4 pull requests
Successful merges:
- #127574 (elaborate unknowable goals)
- #128141 (Set branch protection function attributes)
- #128315 (Fix vita build of std and forbid unsafe in unsafe in the os/vita module)
- #128339 ([rustdoc] Make the buttons remain when code example is clicked)
r? `@ghost`
`@rustbot` modify labels: rollup
Add `select_unpredictable` to force LLVM to use CMOV
Since https://reviews.llvm.org/D118118, LLVM will no longer turn CMOVs into branches if it comes from a `select` marked with an `unpredictable` metadata attribute.
This PR introduces `core::intrinsics::select_unpredictable` which emits such a `select` and uses it in the implementation of `binary_search_by`.
Set branch protection function attributes
Since LLVM 19, it is necessary to set not only module flags, but also function attributes for branch protection on aarch64. See e15d67cfc2 for the relevant LLVM change.
Fixes https://github.com/rust-lang/rust/issues/127829.
Since https://reviews.llvm.org/D118118, LLVM will no longer turn CMOVs
into branches if it comes from a `select` marked with an `unpredictable`
metadata attribute.
This PR introduces `core::intrinsics::select_unpredictable` which emits
such a `select` and uses it in the implementation of `binary_search_by`.
Delete `SimplifyArmIdentity` and `SimplifyBranchSame` tests
These two passes have already been deleted in #107256. I'm not sure why tidy didn't catch it.
As regression tests, I didn't delete `tests/ui/mir/issue-66851.rs` and `tests/ui/mir/simplify-branch-same.rs`.
r? compiler
Since LLVM 19, it is necessary to set not only module flags, but
also function attributes for branch protection on aarch64. See
e15d67cfc2
for the relevant LLVM change.
reenable some windows tests
Locally passing on `x86_64-pc-windows-msvc`, fingers crossed for `*-pc-windows-gnu`.
try-job: x86_64-msvc
try-job: x86_64-mingw
Ensure floats are returned losslessly by the Rust ABI on 32-bit x86
Solves #115567 for the (default) `"Rust"` ABI. When compiling for 32-bit x86, this PR changes the `"Rust"` ABI to return floats indirectly instead of in x87 registers (with the exception of single `f32`s, which this PR returns in general purpose registers as they are small enough to fit in one). No change is made to the `"C"` ABI as that ABI requires x87 register usage and therefore will need a different solution.
`-Z patchable-function-entry` works like `-fpatchable-function-entry`
on clang/gcc. The arguments are total nop count and function offset.
See MCP rust-lang/compiler-team#704
Account for things that optimize out in inlining costs
This updates the MIR inlining `CostChecker` to have both bonuses and penalties, rather than just penalties.
That lets us add bonuses for some things where we want to encourage inlining without risking wrapping into a gigantic cost. For example, `switchInt(const …)` we give an inlining bonus because codegen will actually eliminate the branch (and associated dead blocks) once it's monomorphized, so measuring both sides of the branch gives an unrealistically-high cost to it. Similarly, an `unreachable` terminator gets a small bonus, because whatever branch leads there doesn't actually exist post-codegen.
Stabilise `c_unwind`
Fix#74990Fix#115285 (that's also where FCP is happening)
Marking as draft PR for now due to `compiler_builtins` issues
r? `@Amanieu`
Don't build a broken/untested profiler runtime on mingw targets
Context: https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Why.20build.20a.20broken.2Funtested.20profiler.20runtime.20on.20mingw.3F#75872 added `--enable-profiler` to the `x86_64-mingw` job (to cause some additional tests to run), but had to also add `//@ ignore-windows-gnu` to all of the tests that rely on the profiler runtime actually *working*, because it's broken on that target.
We can achieve a similar outcome by going through all the `//@ needs-profiler-support` tests that don't actually need to produce/run a binary, and making them use `-Zno-profiler-runtime` instead, so that they can run even in configurations that don't have the profiler runtime available. Then we can remove `--enable-profiler` from `x86_64-mingw`, and still get the same amount of testing.
This PR also removes `--enable-profiler` from the mingw dist builds, since it is broken/untested on that target. Those builds have had that flag for a very long time.
For PGO/coverage tests that don't need to build or run an actual artifact, we
can use `-Zno-profiler-runtime` to run the test even when the profiler runtime
is not available.
Rollup of 16 pull requests
Successful merges:
- #123374 (DOC: Add FFI example for slice::from_raw_parts())
- #124514 (Recommend to never display zero disambiguators when demangling v0 symbols)
- #125978 (Cleanup: HIR ty lowering: Consolidate the places that do assoc item probing & access checking)
- #125980 (Nvptx remove direct passmode)
- #126187 (For E0277 suggest adding `Result` return type for function when using QuestionMark `?` in the body.)
- #126210 (docs(core): make more const_ptr doctests assert instead of printing)
- #126249 (Simplify `[T; N]::try_map` signature)
- #126256 (Add {{target}} substitution to compiletest)
- #126263 (Make issue-122805.rs big endian compatible)
- #126281 (set_env: State the conclusion upfront)
- #126286 (Make `storage-live.rs` robust against rustc internal changes.)
- #126287 (Update a cranelift patch file for formatting changes.)
- #126301 (Use `tidy` to sort crate attributes for all compiler crates.)
- #126305 (Make PathBuf less Ok with adding UTF-16 then `into_string`)
- #126310 (Migrate run make prefer rlib)
- #126314 (fix RELEASES: we do not support upcasting to auto traits)
r? `@ghost`
`@rustbot` modify labels: rollup
Instead of not generating the function at all on big endian (which
makes the CHECK lines fail), instead use to_le() on big endian,
so that we essentially perform a bswap for both endiannesses.
Remove hard-coded hashes from codegen tests
This removes hard-coded hashes from the codegen and assembly tests. These use FileCheck, which supports eliding part of the pattern being matched, including by capturing it as a pattern parameter for later matching-on. This is much more appropriate than asking contributors to engage with deliberately-opaque identifier schemes.
In order to reduce the likelihood of error, every hash-coded segment I've touched now expects a certain length. This correctly represents these cases, as our hash outputs have a predetermined amount of entropy attached to them.
This is not done for the UI test suite as those are comparatively easy to simply `--bless`, whereas that would be inappropriate for codegen tests. It is also not done for debuginfo tests as those tests do not support such elision in a correct and useful way.
