Fix clobber_abi in RV32E and RV64E inline assembly
Currently clobber_abi in RV32E and RV64E inline assembly is implemented using InlineAsmClobberAbi::RiscV, but broken since x16-x31 cannot be used in RV32E and RV64E.
```
error: cannot use register `x16`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x17`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x28`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x29`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x30`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
error: cannot use register `x31`: register can't be used with the `e` target feature
--> <source>:42:14
|
42 | asm!("", clobber_abi("C"), options(nostack, nomem, preserves_flags));
| ^^^^^^^^^^^^^^^^
```
r? `@Amanieu`
`@rustbot` label O-riscv +A-inline-assembly
The test was checking for two `ptr` arguments by matching commas (or
non-commas), however after
https://github.com/llvm/llvm-project/pull/117104 LLVM adds an
`initializes((0, 16))` attribute, which includes a comma.
So instead, we make the test check for two LLVM values, i.e. something
prefixed by %.
(See also https://crbug.com/380707238)
When labels are present, the `noreturn` option really means that asm block
won't fallthrough -- if labels are present, then outputs can still be
meaningfully used.
Allow disabling ASan instrumentation for globals
AddressSanitizer adds instrumentation to global variables unless the [`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes) attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to set this; previously it only had the desired effect on functions.
(cc https://github.com/rust-lang/rust/issues/39699)
aarch64 softfloat target: always pass floats in int registers
This is a part of https://github.com/rust-lang/rust/issues/131058: on softfloat aarch64 targets, the float registers may be unavailable. And yet, LLVM will happily use them to pass float types if the corresponding target features are enabled. That's a problem as it means enabling/disabling `neon` instructions can change the ABI.
Other targets have a `soft-float` target feature that forces the use of the soft-float ABI no matter whether float registers are enabled or not; aarch64 has nothing like that.
So we follow the aarch64 [softfloat ABI](https://github.com/rust-lang/rust/issues/131058#issuecomment-2385027423) and treat floats like integers for `extern "C"` functions. For the "Rust" ABI, we do the same for scalars, and then just do something reasonable for ScalarPair that avoids the pointer indirection.
Cc ```@workingjubilee```
There are a handful of tier 2 and tier 3 targets that cause a LLVM crash
or linker error when generating code that contains `f16` or `f128`. The
cranelift backend also does not support these types. To work around
this, every function in `std` or `core` that contains these types must
be marked `#[inline]` in order to avoid sending any code to the backend
unless specifically requested.
However, this is inconvenient and easy to forget. Introduce a check for
these types in the frontend that automatically inlines any function
signatures that take or return `f16` or `f128`.
Note that this is not a perfect fix because it does not account for the
types being passed by reference or as members of aggregate types, but
this is sufficient for what is currently needed in the standard library.
Fixes: https://github.com/rust-lang/rust/issues/133035
Closes: https://github.com/rust-lang/rust/pull/133037
CFI: Append debug location to CFI blocks
Currently we're not appending debug locations to the inserted CFI blocks. This shows up in #132615 and #100783. This change fixes that by passing down the debug location to the CFI type-test generation and appending it to the blocks.
Credits also belong to `@jakos-sec` who worked with me on this.
Stabilize s390x inline assembly
This stabilizes inline assembly for s390x (SystemZ).
Corresponding reference PR: https://github.com/rust-lang/reference/pull/1643
---
From the requirements of stabilization mentioned in https://github.com/rust-lang/rust/issues/93335
> Each architecture needs to be reviewed before stabilization:
> - It must have clobber_abi.
Done in https://github.com/rust-lang/rust/pull/130630.
> - It must be possible to clobber every register that is normally clobbered by a function call.
Done in the PR that added support for clobber_abi.
> - Generally review that the exposed register classes make sense.
The followings can be used as input/output:
- `reg` (`r[0-10]`, `r[12-14]`): General-purpose register
- `reg_addr` (`r[1-10]`, `r[12-14]`): General-purpose register except `r0` which is evaluated as zero in an address context
This class is needed because `r0`, which may be allocated when using the `reg` class, cannot be used as a register in certain contexts. This is identical to the `a` constraint in LLVM and GCC. See https://github.com/rust-lang/rust/pull/119431 for details.
- `freg` (`f[0-15]`): Floating-point register
The followings are clobber-only:
- `vreg` (`v[0-31]`): Vector register
Technically `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `vector` target feature added is enabled, but `core::arch` has no s390x vector type and both `#[repr(simd)]` and `core::simd` are unstable. Everything related is unstable, so the fact that this is currently a clobber-only should not be considered a stabilization blocker. (https://github.com/rust-lang/rust/issues/130869 tracks unstable stuff here)
- `areg` (`a[2-15]`): Access register
All of the above register classes except `reg_addr` are needed for `clobber_abi`.
The followings cannot be used as operands for inline asm (see also [getReservedRegs](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.cpp#L258-L282) and [SystemZELFRegisters](https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/SystemZ/SystemZRegisterInfo.h#L107-L128) in LLVM):
- `r11`: frame pointer
- `r15`: stack pointer
- `a0`, `a1`: Reserved for system use
- `c[0-15]` (control register) Reserved by the kernel
Although not listed in the above requirements, `preserves_flags` is implemented in https://github.com/rust-lang/rust/pull/111331.
---
cc ``@uweigand``
r? ``@Amanieu``
``@rustbot`` label +O-SystemZ +A-inline-assembly
LLVM does not expect to ever see multiple dbg_declares for the same variable at the same
location with different values. proc-macros make it possible for arbitrary code,
including multiple calls that get inlined, to happen at any given location in the source
code. Add discriminators when that happens so these locations are different to LLVM.
This may interfere with the AddDiscriminators pass in LLVM, which is added by the
unstable flag -Zdebug-info-for-profiling.
Fixes#131944
try_question_mark_nop: update test for LLVM 20
llvm/llvm-project@dd116369f6 changes the IR of this test in a way that I don't think is bad, but needs adjusting.
r? `@nikic`
`@rustbot` label: +llvm-main
pointee_info_at: fix logic for recursing into enums
Fixes https://github.com/rust-lang/rust/issues/131834
The logic in `pointee_info_at` was likely written at a time when the null pointer optimization was the *only* enum layout optimization -- and as `Variant::Multiple` kept getting expanded, nobody noticed that the logic is now unsound.
The job of this function is to figure out whether there is a dereferenceable-or-null and aligned pointer at a given offset inside a type. So when we recurse into a multi-variant enum, we better make sure that all the other enum variants must be null! This is the part that was forgotten, and this PR adds it.
The reason this didn't explode in many ways so far is that our references only have 1 niche value (null), so it's not possible on stable to have a multi-variant enum with a dereferenceable pointer and other enum variants that are not null. But with `rustc_layout_scalar_valid_range` attributes one can force such a layout, and if `@the8472's` work on alignment niches ever lands, that will make this possible on stable.
Add `{ignore,needs}-{rustc,std}-debug-assertions` directive support
Add `{ignore,needs}-{rustc,std}-debug-assertions` compiletest directives and retire the old `{ignore,only}-debug` directives. The old `{ignore,only}-debug` directives were ambiguous because you could have std built with debug assertions but rustc not built with debug assertions or vice versa. If we want to support the use case of controlling test run based on if rustc was built with debug assertions, then having `{ignore,only}-debug` will be very confusing.
cc ````@matthiaskrgr````
Closes#123987.
r? bootstrap (or compiler tbh)
This helps with the fact that the order in which debuginfo is emitted
differs between platforms, and is probably not guaranteed to be stable
in general.
The new `is_ascii` function is optimized to use the
`pmovmskb` vector instruction which tests the high bit in a lane.
This corresponds to the same check of whether a byte is ASCII so
ASCII validity checking can be vectorized. This instruction
does not exist on other platforms so it is likely to regress performance
and is gated to all(target_arch = "x86_64", target_feature = "sse2").
Add codegen test
Remove crate::mem import for functions included in the prelude
Remove the `wasm32-wasi` target from rustc
This commit is the final step in the journey of renaming the historical `wasm32-wasi` target in the Rust compiler to `wasm32-wasip1`. Various steps in this journey so far have been:
* 2023-04-03: rust-lang/compiler-team#607 - initial proposal for this rename
* 2024-11-27: rust-lang/compiler-team#695 - amended schedule/procedure for rename
* 2024-01-29: rust-lang/rust#120468 - initial introduction of `wasm32-wasip1`
* 2024-06-18: rust-lang/rust#126662 - warn on usage of `wasm32-wasi`
* 2024-11-08: this PR - remove the `wasm32-wasi` target
The full transition schedule is in [this comment][comment] and is summarized with:
* 2024-05-02: Rust 1.78 released with `wasm32-wasip1` target
* 2024-09-05: Rust 1.81 released warning on usage of `wasm32-wasi`
* 2025-01-09: Rust 1.84 to be released without the `wasm32-wasi` target
This means that support on stable for the replacement target of `wasm32-wasip1` has currently been available for 6 months. Users have already seen warnings on stable for 2 months about usage of `wasm32-wasi` and stable users have another 2 months of warnings before the target is removed from stable.
This commit is intended to be the final step in this transition so the source tree should no longer mention `wasm32-wasi` except in historical reference to the older name of the `wasm32-wasip1` target.
[comment]: https://github.com/rust-lang/rust/pull/120468#issuecomment-1977878747
rustc_codegen_llvm: Add a new 'pc' option to branch-protection
Add a new 'pc' option to -Z branch-protection for aarch64 that enables the use of PC as a diversifier in PAC branch protection code.
When the pauth-lr target feature is enabled in combination with -Z branch-protection=pac-ret,pc, the new 9.5-a instructions (pacibsppc, retaasppc, etc) will be generated.
Support clobber_abi and vector registers (clobber-only) in PowerPC inline assembly
This supports `clobber_abi` which is one of the requirements of stabilization mentioned in #93335.
This basically does a similar thing I did in https://github.com/rust-lang/rust/pull/130630 to implement `clobber_abi` for s390x, but for powerpc/powerpc64/powerpc64le.
- This also supports vector registers (as `vreg`) as clobber-only, which need to support clobbering of them to implement `clobber_abi`.
- `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `altivec` target feature is enabled, but `core::arch::{powerpc,powerpc64}` vector types, `#[repr(simd)]`, and `core::simd` are all unstable, so the fact that this is currently a clobber-only should not be considered a blocker of clobber_abi implementation or stabilization. So I have not implemented it in this PR.
- See https://github.com/rust-lang/rust/pull/131551 (which is based on this PR) for a PR to implement this.
- (I'm not sticking to whether that PR should be a separate PR or part of this PR, so I can merge that PR into this PR if needed.)
Refs:
- PPC32 SysV: Section "Function Calling Sequence" in [System V Application Binary Interface PowerPC Processor Supplement](https://refspecs.linuxfoundation.org/elf/elfspec_ppc.pdf)
- PPC64 ELFv1: Section 3.2 "Function Calling Sequence" in [64-bit PowerPC ELF Application Binary Interface Supplement](https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi.html#FUNC-CALL)
- PPC64 ELFv2: Section 2.2 "Function Calling Sequence" in [64-Bit ELF V2 ABI Specification](https://openpowerfoundation.org/specifications/64bitelfabi/)
- AIX: [Register usage and conventions](https://www.ibm.com/docs/en/aix/7.3?topic=overview-register-usage-conventions), [Special registers in the PowerPC®](https://www.ibm.com/docs/en/aix/7.3?topic=overview-special-registers-in-powerpc), [AIX vector programming](https://www.ibm.com/docs/en/aix/7.3?topic=concepts-aix-vector-programming)
- Register definition in LLVM: https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/PowerPC/PPCRegisterInfo.td#L189
If I understand the above four ABI documentations correctly, except for the PPC32 SysV's VR (Vector Registers) and 32-bit AIX (currently not supported by rustc)'s r13, there does not appear to be important differences in terms of implementing `clobber_abi`:
- The above four ABIs are consistent about FPR (0-13: volatile, 14-31: nonvolatile), CR (0-1,5-7: volatile, 2-4: nonvolatile), XER (volatile), and CTR (volatile).
- As for GPR, only the registers we are treating as reserved are slightly different
- r0, r3-r12 are volatile
- r1(sp, reserved), r14-31 are nonvolatile
- r2(reserved) is TOC pointer in PPC64 ELF/AIX, system-reserved register in PPC32 SysV (AFAIK used as thread pointer in Linux/BSDs)
- r13(reserved for non-32-bit-AIX) is thread pointer in PPC64 ELF, small data area pointer register in PPC32 SysV, "reserved under 64-bit environment; not restored across system calls[^r13]" in AIX)
- As for FPSCR, volatile in PPC64 ELFv1/AIX, some fields are volatile only in certain situations (rest are volatile) in PPC32 SysV/PPC64 ELFv2.
- As for VR (Vector Registers), it is not mentioned in PPC32 SysV, v0-v19 are volatile in both in PPC64 ELF/AIX, v20-v31 are nonvolatile in PPC64 ELF, reserved or nonvolatile depending on the ABI ([vec-extabi vs vec-default in LLVM](https://reviews.llvm.org/D89684), we are [using vec-extabi](https://github.com/rust-lang/rust/pull/131341#discussion_r1797693299)) in AIX:
> When the default Vector enabled mode is used, these registers are reserved and must not be used.
> In the extended ABI vector enabled mode, these registers are nonvolatile and their values are preserved across function calls
I left [FIXME comment about PPC32 SysV](https://github.com/rust-lang/rust/pull/131341#discussion_r1790496095) and added ABI check for AIX.
- As for VRSAVE, it is not mentioned in PPC32 SysV, nonvolatile in PPC64 ELFv1, reserved in PPC64 ELFv2/AIX
- As for VSCR, it is not mentioned in PPC32 SysV/PPC64 ELFv1, some fields are volatile only in certain situations (rest are volatile) in PPC64 ELFv2, volatile in AIX
We are currently treating r1-r2, r13 (non-32-bit-AIX), r29-r31, LR, CTR, and VRSAVE as reserved.
We are currently not processing anything about FPSCR and VSCR, but I feel those are things that should be processed by `preserves_flags` rather than `clobber_abi` if we need to do something about them. (However, PPCRegisterInfo.td in LLVM does not seem to define anything about them.)
Replaces #111335 and #124279
cc `@ecnelises` `@bzEq` `@lu-zero`
r? `@Amanieu`
`@rustbot` label +O-PowerPC +A-inline-assembly
[^r13]: callee-saved, according to [LLVM](6a6af0246b/llvm/lib/Target/PowerPC/PPCCallingConv.td (L322)) and [GCC](a9173a50e7/gcc/config/rs6000/rs6000.h (L859)).
This commit is the final step in the journey of renaming the historical
`wasm32-wasi` target in the Rust compiler to `wasm32-wasip1`. Various
steps in this journey so far have been:
* 2023-04-03: rust-lang/compiler-team#607 - initial proposal for this rename
* 2024-11-27: rust-lang/compiler-team#695 - amended schedule/procedure for rename
* 2024-01-29: rust-lang/rust#120468 - initial introduction of `wasm32-wasip1`
* 2024-06-18: rust-lang/rust#126662 - warn on usage of `wasm32-wasi`
* 2024-11-08: this PR - remove the `wasm32-wasi` target
The full transition schedule is in [this comment][comment] and is
summarized with:
* 2024-05-02: Rust 1.78 released with `wasm32-wasip1` target
* 2024-09-05: Rust 1.81 released warning on usage of `wasm32-wasi`
* 2025-01-09: Rust 1.84 to be released without the `wasm32-wasi` target
This means that support on stable for the replacement target of
`wasm32-wasip1` has currently been available for 6 months. Users have
already seen warnings on stable for 2 months about usage of
`wasm32-wasi` and stable users have another 2 months of warnings before
the target is removed from stable.
This commit is intended to be the final step in this transition so the
source tree should no longer mention `wasm32-wasi` except in historical
reference to the older name of the `wasm32-wasip1` target.
[comment]: https://github.com/rust-lang/rust/pull/120468#issuecomment-1977878747
AddressSanitizer adds instrumentation to global variables unless the
[`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes)
attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to
set this; previously it only had the desired effect on functions.
Add a Few Codegen Tests
Closes#86109Closes#64219
Those issues somehow got fixed over time.
So, this PR adds a couple of codegen tests to ensure we don't regress in the future.
Mark `simplify_aggregate_to_copy` mir-opt as unsound
Mark the `simplify_aggregate_to_copy` mir-opt added in #128299 as unsound as it seems to miscompile the MCVE reported in https://github.com/rust-lang/rust/issues/132353. The mir-opt can be re-enabled once this case is fixed.
```rs
fn pop_min(mut score2head: Vec<Option<usize>>) -> Option<usize> {
loop {
if let Some(col) = score2head[0] {
score2head[0] = None;
return Some(col);
}
}
}
fn main() {
let min = pop_min(vec![Some(1)]);
println!("min: {:?}", min);
// panic happens here on beta in release mode
// but not in debug mode
min.unwrap();
}
```
This MCVE is included as a `run-pass` ui regression test in the first commit. I built the ui test with a nightly manually, and can reproduce the behavioral difference with `-C opt-level=0` and `-C opt-level=1`. Locally, this ui test will fail unless it was run on a compiler built with the second commit marking the mir-opt as unsound thus disabling it by default.
This PR **partially reverts** commit e7386b3, reversing changes made to 02b1be1. The mir-opt implementation is just marked as unsound but **not** reverted to make reland reviews easier. Test changes are **reverted if they were not pure additions**. Tests added by the original PR received `-Z unsound-mir-opts` compile-flags.
cc `@DianQK` `@cjgillot` (PR author and reviewer of #128299)
Add a new 'pc' option to -Z branch-protection for aarch64 that
enables the use of PC as a diversifier in PAC branch protection code.
When the pauth-lr target feature is enabled in combination
with -Z branch-protection=pac-ret,pc, the new 9.5-a instructions
(pacibsppc, retaasppc, etc) will be generated.
Rename Receiver -> LegacyReceiver
As part of the "arbitrary self types v2" project, we are going to replace the current `Receiver` trait with a new mechanism based on a new, different `Receiver` trait.
This PR renames the old trait to get it out the way. Naming is hard. Options considered included:
* HardCodedReceiver (because it should only be used for things in the standard library, and hence is sort-of hard coded)
* LegacyReceiver
* TargetLessReceiver
* OldReceiver
These are all bad names, but fortunately this will be temporary. Assuming the new mechanism proceeds to stabilization as intended, the legacy trait will be removed altogether.
Although we expect this trait to be used only in the standard library, we suspect it may be in use elsehwere, so we're landing this change separately to identify any surprising breakages.
It's known that this trait is used within the Rust for Linux project; a patch is in progress to remove their dependency.
This is a part of the arbitrary self types v2 project,
https://github.com/rust-lang/rfcs/pull/3519https://github.com/rust-lang/rust/issues/44874
r? `@wesleywiser`
Set `signext` or `zeroext` for integer arguments on RISC-V and LoongArch64
This PR contains 3 commits:
- the first one introduces a new function `adjust_for_rust_abi` in `rustc_target`, and moves the x86 specific adjustment code into it;
- the second one adds RISC-V specific adjustment code into it, which sets `signext` or `zeroext` attribute for integer arguments.
- **UPDATE**: added the 3rd commit to apply the same adjustment for LoongArch64.
Optimize `Rc<T>::default`
The missing piece of https://github.com/rust-lang/rust/pull/131460.
Also refactored `Arc<T>::default` by using a safe `NonNull::from(Box::leak(_))` to replace the unnecessarily unsafe call to `NonNull::new_unchecked(Box::into_raw(_))`. The remaining unsafety is coming from `[Rc|Arc]::from_inner`, which is safe from the construction of `[Rc|Arc]Inner`.
x86-32 float return for 'Rust' ABI: treat all float types consistently
This helps with https://github.com/rust-lang/rust/issues/131819: for our own ABI on x86-32, we want to *never* use the float registers. The previous logic only considered F32 and F64, but skipped F16 and F128. So I made the logic just apply to all float types.
try-job: i686-gnu
try-job: i686-gnu-nopt
As part of the "arbitrary self types v2" project, we are going to
replace the current `Receiver` trait with a new mechanism based on a
new, different `Receiver` trait.
This PR renames the old trait to get it out the way. Naming is hard.
Options considered included:
* HardCodedReceiver (because it should only be used for things in the
standard library, and hence is sort-of hard coded)
* LegacyReceiver
* TargetLessReceiver
* OldReceiver
These are all bad names, but fortunately this will be temporary.
Assuming the new mechanism proceeds to stabilization as intended, the
legacy trait will be removed altogether.
Although we expect this trait to be used only in the standard library,
we suspect it may be in use elsehwere, so we're landing this change
separately to identify any surprising breakages.
It's known that this trait is used within the Rust for Linux project; a
patch is in progress to remove their dependency.
This is a part of the arbitrary self types v2 project,
https://github.com/rust-lang/rfcs/pull/3519https://github.com/rust-lang/rust/issues/44874
r? @wesleywiser
Always specify `llvm_abiname` for RISC-V targets
For RISC-V targets, when `llvm_abiname` is not specified LLVM will infer the ABI from the target features, causing #116344 to occur. This PR adds the correct `llvm_abiname` to all RISC-V targets where it is missing (which are all soft-float targets), and adds a test to prevent future RISC-V targets from accidentally omitting `llvm_abiname`. The only affect of this PR is that `-Ctarget-feature=+f` (or similar) will no longer affect the ABI on the modified targets.
<!-- homu-ignore:start -->
r? `@RalfJung`
<!--- homu-ignore:end -->
rust_for_linux: -Zregparm=<N> commandline flag for X86 (#116972)
Command line flag `-Zregparm=<N>` for X86 (32-bit) for rust-for-linux: https://github.com/rust-lang/rust/issues/116972
Implemented in the similar way as fastcall/vectorcall support (args are marked InReg if fit).
Return values larger than 2 registers using a return area pointer
LLVM and Cranelift disagree about how to return values that don't fit in the registers designated for return values. LLVM will force the entire return value to be passed by return area pointer, while Cranelift will look at each IR level return value independently and decide to pass it in a register or not, which would result in the return value being passed partially in registers and partially through a return area pointer.
While Cranelift may need to be fixed as the LLVM behavior is generally more correct with respect to the surface language, forcing this behavior in rustc itself makes it easier for other backends to conform to the Rust ABI and for the C ABI rustc already handles this behavior anyway.
In addition LLVM's decision to pass the return value in registers or using a return area pointer depends on how exactly the return type is lowered to an LLVM IR type. For example `Option<u128>` can be lowered as `{ i128, i128 }` in which case the x86_64 backend would use a return area pointer, or it could be passed as `{ i32, i128 }` in which case the x86_64 backend would pass it in registers by taking advantage of an LLVM ABI extension that allows using 3 registers for the x86_64 sysv call conv rather than the officially specified 2 registers.
This adjustment is only necessary for the Rust ABI as for other ABI's the calling convention implementations in rustc_target already ensure any return value which doesn't fit in the available amount of return registers is passed in the right way for the current target.
Helps with https://github.com/rust-lang/rustc_codegen_cranelift/issues/1525
cc https://github.com/bytecodealliance/wasmtime/issues/9250
Avoid superfluous UB checks in `IndexRange`
`IndexRange::len` is justified as an overall invariant, and
`take_prefix` and `take_suffix` are justified by local branch
conditions. A few more UB-checked calls remain in cases that are only
supported locally by `debug_assert!`, which won't do anything in
distributed builds, so those UB checks may still be useful.
We generally expect core's `#![rustc_preserve_ub_checks]` to optimize
away in user's release builds, but the mere presence of that extra code
can sometimes inhibit optimization, as seen in #131563.
llvm/llvm-project#91101 propagates range information across inlining,
resulting in more metadata in this test. Tolerate the range metadata if
it appears.
Support clobber_abi in MSP430 inline assembly
This supports `clobber_abi` which is one of the requirements of stabilization mentioned in #93335.
Refs: Section 3.2 "Register Conventions" in [MSP430 Embedded Application Binary Interface](https://www.ti.com/lit/an/slaa534a/slaa534a.pdf)
cc ``@cr1901``
r? ``@Amanieu``
``@rustbot`` label +O-msp430
`IndexRange::len` is justified as an overall invariant, and
`take_prefix` and `take_suffix` are justified by local branch
conditions. A few more UB-checked calls remain in cases that are only
supported locally by `debug_assert!`, which won't do anything in
distributed builds, so those UB checks may still be useful.
We generally expect core's `#![rustc_preserve_ub_checks]` to optimize
away in user's release builds, but the mere presence of that extra code
can sometimes inhibit optimization, as seen in #131563.
Use Default visibility for rustc-generated C symbol declarations
Non-default visibilities should only be used for definitions, not declarations, otherwise linking can fail.
This is based on https://github.com/rust-lang/rust/pull/123994.
Issue https://github.com/rust-lang/rust/issues/123427
When I changed `default-hidden-visibility` to `default-visibility` in https://github.com/rust-lang/rust/pull/130005, I updated all places in the code that used `default-hidden-visibility`, replicating the hidden-visibility bug to also happen for protected visibility.
Without this change, trying to build rustc with `-Z default-visibility=protected` fails with a link error.
This hack was intended to handle the case where `-Clto=thin` causes the
compiler to emit multiple output files (when producing LLVM-IR or assembly).
The hack only affects 4 tests, of which 3 are just meta-tests for the hack
itself. The one remaining test that motivated the hack currently doesn't even
need it!
(`tests/codegen/issues/issue-81408-dllimport-thinlto-windows.rs`)
Non-default visibilities should only be used for definitions, not
declarations, otherwise linking can fail.
Co-authored-by: Collin Baker <collinbaker@chromium.org>
LLVM and Cranelift disagree about how to return values that don't fit
in the registers designated for return values. LLVM will force the
entire return value to be passed by return area pointer, while
Cranelift will look at each IR level return value independently and
decide to pass it in a register or not, which would result in the
return value being passed partially in registers and partially through
a return area pointer.
While Cranelift may need to be fixed as the LLVM behavior is generally
more correct with respect to the surface language, forcing this
behavior in rustc itself makes it easier for other backends to conform
to the Rust ABI and for the C ABI rustc already handles this behavior
anyway.
In addition LLVM's decision to pass the return value in registers or
using a return area pointer depends on how exactly the return type is
lowered to an LLVM IR type. For example `Option<u128>` can be lowered
as `{ i128, i128 }` in which case the x86_64 backend would use a return
area pointer, or it could be passed as `{ i32, i128 }` in which case
the x86_64 backend would pass it in registers by taking advantage of an
LLVM ABI extension that allows using 3 registers for the x86_64 sysv
call conv rather than the officially specified 2 registers.
This adjustment is only necessary for the Rust ABI as for other ABI's
the calling convention implementations in rustc_target already ensure
any return value which doesn't fit in the available amount of return
registers is passed in the right way for the current target.
The `Box<T: Default>` impl currently calls `T::default()` before allocating
the `Box`.
Most `Default` impls are trivial, which should in theory allow
LLVM to construct `T: Default` directly in the `Box` allocation when calling
`<Box<T>>::default()`.
However, the allocation may fail, which necessitates calling `T's` destructor if it has one.
If the destructor is non-trivial, then LLVM has a hard time proving that it's
sound to elide, which makes it construct `T` on the stack first, and then copy it into the allocation.
Create an uninit `Box` first, and then write `T::default` into it, so that LLVM now only needs to prove
that the `T::default` can't panic, which should be trivial for most `Default` impls.
Add missing module flags for `-Zfunction-return=thunk-extern`
This fixes a bug in the `-Zfunction-return=thunk-extern` flag. The flag needs to be passed onto LLVM to ensure that functions such as `asan.module_ctor` and `asan.module_dtor` that are created internally in LLVM have the mitigation applied to them.
This was originally discovered [in the Linux kernel](https://lore.kernel.org/all/CANiq72myZL4_poCMuNFevtpYYc0V0embjSuKb7y=C+m3vVA_8g@mail.gmail.com/).
Original flag PR: #116892
PR for similar issue: #129373
Tracking issue: #116853
cc ``@ojeda``
r? ``@wesleywiser``
- fix for divergence
- fix error message
- fix another cranelift test
- fix some cranelift things
- don't set the NORETURN option for naked asm
- fix use of naked_asm! in doc comment
- fix use of naked_asm! in run-make test
- use `span_bug` in unreachable branch
Refactor the code in the `convert_while_ascii` helper function to make
it more suitable for auto-vectorization and also process the full ascii
prefix of the string. The generic case conversion logic will only be
invoked starting from the first non-ascii character.
The runtime on microbenchmarks with ascii-only inputs improves between
1.5x for short and 4x for long inputs on x86_64 and aarch64.
The new implementation also encapsulates all unsafe inside the
`convert_while_ascii` function.
Fixes#123712
Don't alloca for unused locals
We already have a concept of mono-unreachable basic blocks; this is primarily useful for ensuring that we do not compile code under an `if false`. But since we never gave locals the same analysis, a large local only used under an `if false` will still have stack space allocated for it.
There are 3 places we traverse MIR during monomorphization: Inside the collector, `non_ssa_locals`, and the walk to generate code. Unfortunately, https://github.com/rust-lang/rust/pull/129283#issuecomment-2297925578 indicates that we cannot afford the expense of tracking reachable locals during the collector's traversal, so we do need at least two mono-reachable traversals. And of course caching is of no help here because the benchmarks that regress are incr-unchanged; they don't do any codegen.
This fixes the second problem in https://github.com/rust-lang/rust/issues/129282, and brings us anther step toward `const if` at home.
Remove macOS 10.10 dynamic linker bug workaround
Rust's current minimum macOS version is 10.12, so the hack can be removed. This PR also updates the `remove_dir_all` docs to reflect that all supported macOS versions are protected against TOCTOU race conditions (the fallback implementation was already removed in #127683).
try-job: dist-x86_64-apple
try-job: dist-aarch64-apple
try-job: dist-apple-various
try-job: aarch64-apple
try-job: x86_64-apple-1
Update the minimum external LLVM to 18
With this change, we'll have stable support for LLVM 18 and 19.
For reference, the previous increase to LLVM 17 was #122649.
cc `@rust-lang/wg-llvm`
r? nikic
tests: add repr/transparent test for aarch64
Fixes#74396.
Moves `transparent-struct-ptr.rs` to `transparent-byval-struct-ptr.rs` and then adds a new `transparent-opaque-ptr.rs` for aarch64.
On LLVM 20, these instructions already get eliminated, which at least
partially satisfies a TODO. I'm not talented enough at using FileCheck
to try and constrain this further, but if we really want to we could
copy an LLVM 20 specific version of this test that would restore it to
being CHECK-NEXT: insertvalue ...
@rustbot label: +llvm-main
Do not request sanitizers for naked functions
Naked functions can only contain inline asm, so any instrumentation inserted by sanitizers is illegal. Don't request it.
Fixes https://github.com/rust-lang/rust/issues/129224.
Don't emit `expect`/`assume` in opt-level=0
LLVM does not make use of expect/assume calls in `opt-level=0`, so we can simplify IR by not emitting them in this case.
rustc_target: Add various aarch64 features
Add various aarch64 features already supported by LLVM and Linux.
Additionally include some comment fixes to ensure consistency of feature names with the Arm ARM.
Compiler support for features added to stdarch by https://github.com/rust-lang/stdarch/pull/1614.
Tracking issue for unstable aarch64 features is https://github.com/rust-lang/rust/issues/127764.
List of added features:
- FEAT_CSSC
- FEAT_ECV
- FEAT_FAMINMAX
- FEAT_FLAGM2
- FEAT_FP8
- FEAT_FP8DOT2
- FEAT_FP8DOT4
- FEAT_FP8FMA
- FEAT_HBC
- FEAT_LSE128
- FEAT_LSE2
- FEAT_LUT
- FEAT_MOPS
- FEAT_LRCPC3
- FEAT_SVE_B16B16
- FEAT_SVE2p1
- FEAT_WFxT
- FEAT_SME
- FEAT_SME_F16F16
- FEAT_SME_F64F64
- FEAT_SME_F8F16
- FEAT_SME_F8F32
- FEAT_SME_FA64
- FEAT_SME_I16I64
- FEAT_SME_LUTv2
- FEAT_SME2
- FEAT_SME2p1
- FEAT_SSVE_FP8DOT2
- FEAT_SSVE_FP8DOT4
- FEAT_SSVE_FP8FMA
FEAT_FPMR is added in the first commit and then removed in a separate one to highlight it being removed from upstream LLVM 19. The intention is for it to be detectable at runtime through stdarch but not have a corresponding Rust compile-time feature.
add repr to the allowlist for naked functions
Fixes#129412 (combining unstable features #90957 (`#![feature(naked_functions)]`) and #82232 (`#![feature(fn_align)]`)
Set the cfi-normalize-integers and kcfi-offset module flags when
Control-Flow Integrity sanitizers are used, so functions generated by
the LLVM backend use the same CFI/KCFI options as rustc.
cfi-normalize-integers tells LLVM to also use integer normalization
for generated functions when -Zsanitizer-cfi-normalize-integers is
used.
kcfi-offset specifies the number of prefix nops between the KCFI
type hash and the function entry when -Z patchable-function-entry is
used. Note that LLVM assumes all indirectly callable functions use the
same number of prefix NOPs with -Zsanitizer=kcfi.
