mark some target features as 'forbidden' so they cannot be (un)set with -Ctarget-feature
The context for this is https://github.com/rust-lang/rust/issues/116344: some target features change the way floats are passed between functions. Changing those target features is unsound as code compiled for the same target may now use different ABIs.
So this introduces a new concept of "forbidden" target features (on top of the existing "stable " and "unstable" categories), and makes it a hard error to (un)set such a target feature. For now, the x86 and ARM feature `soft-float` is on that list. We'll have to make some effort to collect more relevant features, and similar features from other targets, but that can happen after the basic infrastructure for this landed. (These features are being collected in https://github.com/rust-lang/rust/issues/131799.)
I've made this a warning for now to give people some time to speak up if this would break something.
MCP: https://github.com/rust-lang/compiler-team/issues/780
Support clobber_abi and vector registers (clobber-only) in PowerPC inline assembly
This supports `clobber_abi` which is one of the requirements of stabilization mentioned in #93335.
This basically does a similar thing I did in https://github.com/rust-lang/rust/pull/130630 to implement `clobber_abi` for s390x, but for powerpc/powerpc64/powerpc64le.
- This also supports vector registers (as `vreg`) as clobber-only, which need to support clobbering of them to implement `clobber_abi`.
- `vreg` should be able to accept `#[repr(simd)]` types as input/output if the unstable `altivec` target feature is enabled, but `core::arch::{powerpc,powerpc64}` vector types, `#[repr(simd)]`, and `core::simd` are all unstable, so the fact that this is currently a clobber-only should not be considered a blocker of clobber_abi implementation or stabilization. So I have not implemented it in this PR.
- See https://github.com/rust-lang/rust/pull/131551 (which is based on this PR) for a PR to implement this.
- (I'm not sticking to whether that PR should be a separate PR or part of this PR, so I can merge that PR into this PR if needed.)
Refs:
- PPC32 SysV: Section "Function Calling Sequence" in [System V Application Binary Interface PowerPC Processor Supplement](https://refspecs.linuxfoundation.org/elf/elfspec_ppc.pdf)
- PPC64 ELFv1: Section 3.2 "Function Calling Sequence" in [64-bit PowerPC ELF Application Binary Interface Supplement](https://refspecs.linuxfoundation.org/ELF/ppc64/PPC-elf64abi.html#FUNC-CALL)
- PPC64 ELFv2: Section 2.2 "Function Calling Sequence" in [64-Bit ELF V2 ABI Specification](https://openpowerfoundation.org/specifications/64bitelfabi/)
- AIX: [Register usage and conventions](https://www.ibm.com/docs/en/aix/7.3?topic=overview-register-usage-conventions), [Special registers in the PowerPC®](https://www.ibm.com/docs/en/aix/7.3?topic=overview-special-registers-in-powerpc), [AIX vector programming](https://www.ibm.com/docs/en/aix/7.3?topic=concepts-aix-vector-programming)
- Register definition in LLVM: https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0/llvm/lib/Target/PowerPC/PPCRegisterInfo.td#L189
If I understand the above four ABI documentations correctly, except for the PPC32 SysV's VR (Vector Registers) and 32-bit AIX (currently not supported by rustc)'s r13, there does not appear to be important differences in terms of implementing `clobber_abi`:
- The above four ABIs are consistent about FPR (0-13: volatile, 14-31: nonvolatile), CR (0-1,5-7: volatile, 2-4: nonvolatile), XER (volatile), and CTR (volatile).
- As for GPR, only the registers we are treating as reserved are slightly different
- r0, r3-r12 are volatile
- r1(sp, reserved), r14-31 are nonvolatile
- r2(reserved) is TOC pointer in PPC64 ELF/AIX, system-reserved register in PPC32 SysV (AFAIK used as thread pointer in Linux/BSDs)
- r13(reserved for non-32-bit-AIX) is thread pointer in PPC64 ELF, small data area pointer register in PPC32 SysV, "reserved under 64-bit environment; not restored across system calls[^r13]" in AIX)
- As for FPSCR, volatile in PPC64 ELFv1/AIX, some fields are volatile only in certain situations (rest are volatile) in PPC32 SysV/PPC64 ELFv2.
- As for VR (Vector Registers), it is not mentioned in PPC32 SysV, v0-v19 are volatile in both in PPC64 ELF/AIX, v20-v31 are nonvolatile in PPC64 ELF, reserved or nonvolatile depending on the ABI ([vec-extabi vs vec-default in LLVM](https://reviews.llvm.org/D89684), we are [using vec-extabi](https://github.com/rust-lang/rust/pull/131341#discussion_r1797693299)) in AIX:
> When the default Vector enabled mode is used, these registers are reserved and must not be used.
> In the extended ABI vector enabled mode, these registers are nonvolatile and their values are preserved across function calls
I left [FIXME comment about PPC32 SysV](https://github.com/rust-lang/rust/pull/131341#discussion_r1790496095) and added ABI check for AIX.
- As for VRSAVE, it is not mentioned in PPC32 SysV, nonvolatile in PPC64 ELFv1, reserved in PPC64 ELFv2/AIX
- As for VSCR, it is not mentioned in PPC32 SysV/PPC64 ELFv1, some fields are volatile only in certain situations (rest are volatile) in PPC64 ELFv2, volatile in AIX
We are currently treating r1-r2, r13 (non-32-bit-AIX), r29-r31, LR, CTR, and VRSAVE as reserved.
We are currently not processing anything about FPSCR and VSCR, but I feel those are things that should be processed by `preserves_flags` rather than `clobber_abi` if we need to do something about them. (However, PPCRegisterInfo.td in LLVM does not seem to define anything about them.)
Replaces #111335 and #124279
cc `@ecnelises` `@bzEq` `@lu-zero`
r? `@Amanieu`
`@rustbot` label +O-PowerPC +A-inline-assembly
[^r13]: callee-saved, according to [LLVM](6a6af0246b/llvm/lib/Target/PowerPC/PPCCallingConv.td (L322)) and [GCC](a9173a50e7/gcc/config/rs6000/rs6000.h (L859)).
Reduce dependence on the target name
The target name can be anything with custom target specs. Matching on fields inside the target spec is much more robust than matching on the target name.
Also remove the unused is_builtin target spec field.
The target name can be anything with custom target specs. Matching on
fields inside the target spec is much more robust than matching on the
target name.
Port most of `--print=target-cpus` to Rust
The logic and formatting needed by `--print=target-cpus` has historically been carried out in C++ code. Originally it used `printf` to write directly to the console, but later it switched over to writing to a `std::ostringstream` and then passing its buffer to a callback function pointer.
This PR replaces that C++ code with a very simple function that writes a list of CPU names to a `&RustString`, with the rest of the logic and formatting being handled by ordinary safe Rust code.
AddressSanitizer adds instrumentation to global variables unless the
[`no_sanitize_address`](https://llvm.org/docs/LangRef.html#global-attributes)
attribute is set on them.
This commit extends the existing `#[no_sanitize(address)]` attribute to
set this; previously it only had the desired effect on functions.
Move versioned Apple LLVM targets from `rustc_target` to `rustc_codegen_ssa`
Fully specified LLVM targets contain the OS version on macOS/iOS/tvOS/watchOS/visionOS, and this version depends on the deployment target environment variables like `MACOSX_DEPLOYMENT_TARGET`, `IPHONEOS_DEPLOYMENT_TARGET` etc.
We would like to move this to later in the compilation pipeline, both because it feels impure to access environment variables when fetching target information, but mostly because we need access to more information from https://github.com/rust-lang/rust/pull/130883 to do https://github.com/rust-lang/rust/issues/118204. See also https://github.com/rust-lang/rust/pull/129342#issuecomment-2335156119 for some discussion.
The first and second commit does the actual refactor, it should be a non-functional change, the third commit adds diagnostics for invalid deployment targets, which are now possible to do because we have access to the session.
Tested with the same commands as in https://github.com/rust-lang/rust/pull/130435.
r? ``````@petrochenkov``````
Remove support for `-Zprofile` (gcov-style coverage instrumentation)
Tracking issue: #42524
MCP: https://github.com/rust-lang/compiler-team/issues/798
---
This PR removes the unstable `-Zprofile` flag, which enables ”gcov-style” coverage instrumentation, along with its associated `-Zprofile-emit` configuration flag.
(The profile flag predates and is almost entirely separate from the stable `-Cinstrument-coverage` flag.)
Notably, the `-Zprofile` flag:
- Is largely untested in-tree, having only one run-make test that does not check whether its output is correct or useful.
- Has no known maintainer.
- Has seen no push towards stabilization.
- Has at least one severe regression reported in 2022 that apparently remains unaddressed.
- #100125
- Is confusingly named, since it appears to be more about coverage than performance profiling, and has nothing to do with PGO.
- Is fundamentally limited by relying on counters auto-inserted by LLVM, with no knowledge of Rust beyond debuginfo.
The OS version depends on the deployment target environment variables,
the access of which we want to move to later in the compilation pipeline
that has access to more information, for example `env_depinfo`.
LLVM continues to align more 128-bit integers to 128-bits in the data
layout rather than relying on the high level language to do it. Update
SPARC target files to match and add a backcompat replacement for current
LLVMs.
See llvm/llvm-project#106951 for details
Add a new 'pc' option to -Z branch-protection for aarch64 that
enables the use of PC as a diversifier in PAC branch protection code.
When the pauth-lr target feature is enabled in combination
with -Z branch-protection=pac-ret,pc, the new 9.5-a instructions
(pacibsppc, retaasppc, etc) will be generated.
Rename `rustc_abi::Abi` to `BackendRepr`
Remove the confabulation of `rustc_abi::Abi` with what "ABI" actually means by renaming it to `BackendRepr`, and rename `Abi::Aggregate` to `BackendRepr::Memory`. The type never actually represented how things are passed, as that has to have `PassMode` considered, at minimum, but rather it just is how we represented some things to the backend. This conflation arose because LLVM, the primary backend at the time, would lower certain IR forms using certain ABIs. Even that only somewhat was true, as it broke down when one ventured significantly afield of what is described by the System V AMD64 ABI either by using different architectures, ABI-modifying IR annotations, the same architecture **with different ISA extensions enabled**, or other... unexpected delights.
Unfortunately both names are still somewhat of a misnomer right now, as people have written code for years based on this misunderstanding. Still, their original names are even moreso, and for better or worse, this backend code hasn't received as much maintenance as the rest of the compiler, lately. Actually arriving at a correct end-state will simply require us to disentangle a lot of code in order to fix, much of it pointlessly repeated in several places. Thus this is not an "actual fix", just a way to deflect further misunderstandings.
cg_llvm: Clean up FFI calls for operand bundles
All of these FFI functions have equivalents in the stable LLVM-C API, though `LLVMBuildCallBr` requires a temporary polyfill on LLVM 18.
This PR also creates a clear split between `OperandBundleOwned` and `OperandBundle`, and updates the internals of the owner to be a little less terrifying.
The initial naming of "Abi" was an awful mistake, conveying wrong ideas
about how psABIs worked and even more about what the enum meant.
It was only meant to represent the way the value would be described to
a codegen backend as it was lowered to that intermediate representation.
It was never meant to mean anything about the actual psABI handling!
The conflation is because LLVM typically will associate a certain form
with a certain ABI, but even that does not hold when the special cases
that actually exist arise, plus the IR annotations that modify the ABI.
Reframe `rustc_abi::Abi` as the `BackendRepr` of the type, and rename
`BackendRepr::Aggregate` as `BackendRepr::Memory`. Unfortunately, due to
the persistent misunderstandings, this too is now incorrect:
- Scattered ABI-relevant code is entangled with BackendRepr
- We do not always pre-compute a correct BackendRepr that reflects how
we "actually" want this value to be handled, so we leave the backend
interface to also inject various special-cases here
- In some cases `BackendRepr::Memory` is a "real" aggregate, but in
others it is in fact using memory, and in some cases it is a scalar!
Our rustc-to-backend lowering code handles this sort of thing right now.
That will eventually be addressed by lifting duplicated lowering code
to either rustc_codegen_ssa or rustc_target as appropriate.
cg_llvm: Clean up FFI calls for setting module flags
This is a combination of several inter-related changes to how module flags are set:
- Remove some unnecessary code for setting an `"LTOPostLink"` flag, which has been obsolete since LLVM 17.
- Define our own enum instead of relying on enum values defined by LLVM's unstable C++ API.
- Use safe wrapper functions to set module flags, instead of direct `unsafe` calls.
- Consistently pass pointer/length strings instead of C strings.
- Remove or shrink some `unsafe` blocks.
- Don't rely on enum values defined by LLVM's C++ API
- Use safe wrapper functions instead of direct `unsafe` calls
- Consistently pass pointer/length strings instead of C strings
correct LLVMRustCreateThinLTOData arg types
`LLVMRustCreateThinLTOData` defined in rust as
```rust
pub fn LLVMRustCreateThinLTOData(
Modules: *const ThinLTOModule,
NumModules: c_uint,
PreservedSymbols: *const *const c_char,
PreservedSymbolsLen: c_uint,
) -> Option<&'static mut ThinLTOData>;
```
but in cpp as
```cpp
extern "C" LLVMRustThinLTOData *
LLVMRustCreateThinLTOData(LLVMRustThinLTOModule *modules, int num_modules,
const char **preserved_symbols, int num_symbols) {
```
(note `c_unit` vs `int` types). Let it be actually `size_t`.
Also fixes return type of `LLVMRustDIBuilderCreateOpLLVMFragment` to uint64_t as other similar functions around, which should be correct, i assume.
cg_llvm: Use a type-safe helper to cast `&str` and `&[u8]` to `*const c_char`
In `rustc_codegen_llvm` there are many uses of `.as_ptr().cast()` to convert a string or byte-slice to `*const c_char`, which then gets passed through FFI.
This works, but is fragile, because there's nothing constraining the pointer cast to actually be from `u8` to `c_char`. If the original value changes to something else that has an `as_ptr` method, or the context changes to expect something other than `c_char`, the cast will silently do the wrong thing.
By making the cast more explicit via a helper method, we can be sure that it will either perform the intended cast, or fail at compile time.
This commit adds a new rustc target feature named `wide-arithmetic` for
WebAssembly targets. This corresponds to the [wide-arithmetic] proposal
for WebAssembly which adds new instructions catered towards accelerating
integer arithmetic larger than 64-bits. This proposal to WebAssembly is
not standard yet so this new feature is flagged as an unstable target
feature. Additionally Rust's LLVM version doesn't support this new
feature yet since support will first be added in LLVM 20, so the
feature filtering logic for LLVM is updated to handle this.
I'll also note that I'm not currently planning to add wasm-specific
intrinsics to `std::arch::wasm32` at this time. The currently proposed
instructions are all accessible through `i128` or `u128`-based
operations which Rust already supports, so intrinsic shouldn't be
necessary to get access to these new instructions.
[wide-arithmetic]: https://github.com/WebAssembly/wide-arithmetic
Replace some LLVMRust wrappers with calls to the LLVM C API
This PR removes the LLVMRust wrapper functions for getting/setting linkage and visibility, and replaces them with direct calls to the corresponding functions in LLVM's C API.
To make this convenient and sound, two pieces of supporting code have also been added:
- A simple proc-macro that derives `TryFrom<u32>` for fieldless enums
- A wrapper type for C enum values returned by LLVM functions, to ensure soundness if LLVM returns an enum value we don't know about
In a few places, the use of safe wrapper functions means that an `unsafe` block is no longer needed, so the affected code has changed its indentation level.
rustc_target: Add pauth-lr aarch64 target feature
Add the pauth-lr target feature, corresponding to aarch64 FEAT_PAuth_LR. This feature has been added in LLVM 19.
It is currently not supported by the Linux hwcap and so we cannot add runtime feature detection for it at this time.
r? `@Amanieu`
coverage: Consolidate creation of covmap/covfun records
This code for creating covmap/covfun records during codegen was split across multiple functions and files for dubious historical reasons. Having it all in one place makes it easier to follow.
This PR also includes two semi-related cleanups:
- Getting the codegen context's `coverage_cx` state is made infallible, since it should always exist when running the code paths that need it.
- The value of `covfun_section_name` is saved in the codegen context, since it never changes at runtime, and the code that needs it has access to the context anyway.
---
Background: Coverage instrumentation generates two kinds of metadata that are embedded in the final binary. There is per-CGU information that goes in the `__llvm_covmap` linker section, and per-function information that goes in the `__llvm_covfun` section (except on Windows, where slightly different section names are used).
- removed extra bits from predicates queries that are no longer needed in the new system
- removed the need for `non_erasable_generics` to take in tcx and DefId, removed unused arguments in callers
Adding an extra `OnceCell` to `CrateCoverageContext` is much nicer than trying
to thread this string through multiple layers of function calls that already
have access to the context.
coverage: Pass coverage mappings to LLVM as separate structs
Instead of trying to cram *N* different kinds of coverage mapping data into a single list for FFI, pass *N* different lists of simpler structs.
This avoids the need to fill unused fields with dummy values, and avoids the need to tag structs with their underlying kind. It also lets us call the dedicated LLVM constructors for each different mapping type, instead of having to go through the complex general-purpose constructor.
Even though this adds multiple new structs to the FFI surface area, the resulting C++ code is simpler and shorter.
---
I've structured this mostly as a single atomic patch, rather than a series of incremental changes, because that avoids the need to make fiddly fixes to code that is about to be deleted anyway.
Continue to get rid of `ty::Const::{try_}eval*`
This PR mostly does:
* Removes all of the `try_eval_*` and `eval_*` helpers from `ty::Const`, and replace their usages with `try_to_*`.
* Remove `ty::Const::eval`.
* Rename `ty::Const::normalize` to `ty::Const::normalize_internal`. This function is still used in the normalization code itself.
* Fix some weirdness around the `TransmuteFrom` goal.
I'm happy to split it out further; for example, I could probably land the first part which removes the helpers, or the changes to codegen which are more obvious than the changes to tools.
r? BoxyUwU
Part of https://github.com/rust-lang/rust/issues/130704
Migrate `llvm::set_comdat` and `llvm::SetUniqueComdat` to LLVM-C FFI.
Note, now we can call `llvm::set_comdat` only when the target actually
supports adding comdat. As this has no convenient LLVM-C API, we
implement this as `TargetOptions::supports_comdat`.
Co-authored-by: Stuart Cook <Zalathar@users.noreply.github.com>
Add the pauth-lr target feature, corresponding to aarch64 FEAT_PAuth_LR.
This feature has been added in LLVM 19.
It is currently not supported by the Linux hwcap and so we cannot add
runtime feature detection for it at this time.
LLVM has added 3 new address spaces to support special Windows use
cases. These shouldn't trouble us for now, but LLVM requires matching
data layouts.
See llvm/llvm-project#111879 for details
intrinsics fmuladdf{32,64}: expose llvm.fmuladd.* semantics
Add intrinsics `fmuladd{f32,f64}`. This computes `(a * b) + c`, to be fused if the code generator determines that (i) the target instruction set has support for a fused operation, and (ii) that the fused operation is more efficient than the equivalent, separate pair of `mul` and `add` instructions.
https://llvm.org/docs/LangRef.html#llvm-fmuladd-intrinsic
The codegen_cranelift uses the `fma` function from libc, which is a correct implementation, but without the desired performance semantic. I think this requires an update to cranelift to expose a suitable instruction in its IR.
I have not tested with codegen_gcc, but it should behave the same way (using `fma` from libc).
---
This topic has been discussed a few times on Zulip and was suggested, for example, by `@workingjubilee` in [Effect of fma disabled](https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/Effect.20of.20fma.20disabled/near/274179331).
Add intrinsics `fmuladd{f16,f32,f64,f128}`. This computes `(a * b) +
c`, to be fused if the code generator determines that (i) the target
instruction set has support for a fused operation, and (ii) that the
fused operation is more efficient than the equivalent, separate pair
of `mul` and `add` instructions.
https://llvm.org/docs/LangRef.html#llvm-fmuladd-intrinsic
MIRI support is included for f32 and f64.
The codegen_cranelift uses the `fma` function from libc, which is a
correct implementation, but without the desired performance semantic. I
think this requires an update to cranelift to expose a suitable
instruction in its IR.
I have not tested with codegen_gcc, but it should behave the same
way (using `fma` from libc).
Use Default visibility for rustc-generated C symbol declarations
Non-default visibilities should only be used for definitions, not declarations, otherwise linking can fail.
This is based on https://github.com/rust-lang/rust/pull/123994.
Issue https://github.com/rust-lang/rust/issues/123427
When I changed `default-hidden-visibility` to `default-visibility` in https://github.com/rust-lang/rust/pull/130005, I updated all places in the code that used `default-hidden-visibility`, replicating the hidden-visibility bug to also happen for protected visibility.
Without this change, trying to build rustc with `-Z default-visibility=protected` fails with a link error.
Non-default visibilities should only be used for definitions, not
declarations, otherwise linking can fail.
Co-authored-by: Collin Baker <collinbaker@chromium.org>
rustc_target: Add sme-b16b16 as an explicit aarch64 target feature
LLVM 20 split out what used to be called b16b16 and correspond to aarch64
FEAT_SVE_B16B16 into sve-b16b16 and sme-b16b16.
Add sme-b16b16 as an explicit feature and update the codegen accordingly.
Resolves https://github.com/rust-lang/rust/pull/129894.
codegen_ssa: consolidate tied target checks
Fixes#105110.
Fixes#105111.
`rustc_codegen_llvm` and `rustc_codegen_gcc` duplicated logic for checking if tied target features were partially enabled. This PR consolidates these checks into `rustc_codegen_ssa` in the `codegen_fn_attrs` query, which also is run pre-monomorphisation for each function, which ensures that this check is run for unused functions, as would be expected.
Also adds a test confirming that enabling one tied feature doesn't imply another - the appropriate error for this was already being emitted. I did a bisect and narrowed it down to two patches it was likely to be - something in #128796, probably #128221 or #128679.
LLVM 20 split out what used to be called b16b16 and correspond to aarch64
FEAT_SVE_B16B16 into sve-b16b16 and sme-b16b16.
Add sme-b16b16 as an explicit feature and update the codegen accordingly.
compiler: Stop reexporting enum-globs from `rustc_target::abi`
Three enums had **all** their variants glob-exported into a distressingly large amount of the tree. Cease to do that, and also cease to glob import the contents of the module that contained them. Redirect relevant imports to their actual source, the `rustc_abi` crate.
No functional changes.
[Coverage][MCDC] Adapt mcdc to llvm 19
Related issue: #126672
Also finish task 4 at #124144
[llvm #82448](https://github.com/llvm/llvm-project/pull/82448) has introduced some break changes into mcdc, causing incompatibility between llvm 18 and 19. This draft adapts to that change and gives up supporting for llvm-18.
Add missing module flags for `-Zfunction-return=thunk-extern`
This fixes a bug in the `-Zfunction-return=thunk-extern` flag. The flag needs to be passed onto LLVM to ensure that functions such as `asan.module_ctor` and `asan.module_dtor` that are created internally in LLVM have the mitigation applied to them.
This was originally discovered [in the Linux kernel](https://lore.kernel.org/all/CANiq72myZL4_poCMuNFevtpYYc0V0embjSuKb7y=C+m3vVA_8g@mail.gmail.com/).
Original flag PR: #116892
PR for similar issue: #129373
Tracking issue: #116853
cc ``@ojeda``
r? ``@wesleywiser``
Like #130865 did for the standard library, we can use `&raw` in the
compiler now that stage0 supports it. Also like the other issue, I did
not make any doc or test changes at this time.
