add `-Zmin-function-alignment`
tracking issue: https://github.com/rust-lang/rust/issues/82232
This PR adds the `-Zmin-function-alignment=<align>` flag, that specifies a minimum alignment for all* functions.
### Motivation
This feature is requested by RfL [here](https://github.com/rust-lang/rust/issues/128830):
> i.e. the equivalents of `-fmin-function-alignment` ([GCC](https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-fmin-function-alignment_003dn), Clang does not support it) / `-falign-functions` ([GCC](https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#index-falign-functions), [Clang](https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang1-falign-functions)).
>
> For the Linux kernel, the behavior wanted is that of GCC's `-fmin-function-alignment` and Clang's `-falign-functions`, i.e. align all functions, including cold functions.
>
> There is [`feature(fn_align)`](https://github.com/rust-lang/rust/issues/82232), but we need to do it globally.
### Behavior
The `fn_align` feature does not have an RFC. It was decided at the time that it would not be necessary, but maybe we feel differently about that now? In any case, here are the semantics of this flag:
- `-Zmin-function-alignment=<align>` specifies the minimum alignment of all* functions
- the `#[repr(align(<align>))]` attribute can be used to override the function alignment on a per-function basis: when `-Zmin-function-alignment` is specified, the attribute's value is only used when it is higher than the value passed to `-Zmin-function-alignment`.
- the target may decide to use a higher value (e.g. on x86_64 the minimum that LLVM generates is 16)
- The highest supported alignment in rust is `2^29`: I checked a bunch of targets, and they all emit the `.p2align 29` directive for targets that align functions at all (some GPU stuff does not have function alignment).
*: Only with `build-std` would the minimum alignment also be applied to `std` functions.
---
cc `@ojeda`
r? `@workingjubilee` you were active on the tracking issue
Adds `#[rustc_force_inline]` which is similar to always inlining but
reports an error if the inlining was not possible, and which always
attempts to inline annotated items, regardless of optimisation levels.
It can only be applied to free functions to guarantee that the MIR
inliner will be able to resolve calls.
rustc_codegen_llvm: Add a new 'pc' option to branch-protection
Add a new 'pc' option to -Z branch-protection for aarch64 that enables the use of PC as a diversifier in PAC branch protection code.
When the pauth-lr target feature is enabled in combination with -Z branch-protection=pac-ret,pc, the new 9.5-a instructions (pacibsppc, retaasppc, etc) will be generated.
Add a new 'pc' option to -Z branch-protection for aarch64 that
enables the use of PC as a diversifier in PAC branch protection code.
When the pauth-lr target feature is enabled in combination
with -Z branch-protection=pac-ret,pc, the new 9.5-a instructions
(pacibsppc, retaasppc, etc) will be generated.
`rustc_codegen_llvm` and `rustc_codegen_gcc` duplicated logic for
checking if tied target features were partially enabled. This commit
consolidates these checks into `rustc_codegen_ssa` in the
`codegen_fn_attrs` query, which also is run pre-monomorphisation for
each function, which ensures that this check is run for unused functions,
as would be expected.
Convert to_llvm_features to return Option<LLVMFeature> so that it can
return None if the requested feature is not available for the current
LLVM version.
Add match rules to filter out aarch64 features not available in LLVM 17.
FEAT_FPMR has been removed from upstream LLVM as of LLVM 19.
Remove the feature from the target features list and temporarily hack
the LLVM codegen to always enable it until the minimum LLVM version is
bumped to 19.
codegen: better centralize function declaration attribute computation
For some reason, the codegen backend has two functions that compute which attributes a function declaration gets: `apply_attrs_llfn` and `attributes::from_fn_attrs`. They are called in different places, on entirely different layers of abstraction.
To me the code seems cleaner if we centralize this entirely in `apply_attrs_llfn`, so that's what this PR does.
Set branch protection function attributes
Since LLVM 19, it is necessary to set not only module flags, but also function attributes for branch protection on aarch64. See e15d67cfc2 for the relevant LLVM change.
Fixes https://github.com/rust-lang/rust/issues/127829.
Since LLVM 19, it is necessary to set not only module flags, but
also function attributes for branch protection on aarch64. See
e15d67cfc2
for the relevant LLVM change.
... this is a special attribute that was made to be a target-feature in
LLVM 18+, but in all previous versions, this "feature" is a naked
attribute. We will have to handle this situation differently than all
other target-features.
Remove the unstable `extern "wasm"` ABI (`wasm_abi` feature tracked
in #83788).
As discussed in https://github.com/rust-lang/rust/pull/127513#issuecomment-2220410679
and following, this ABI is a failed experiment that did not end
up being used for anything. Keeping support for this ABI in LLVM 19
would require us to switch wasm targets to the `experimental-mv`
ABI, which we do not want to do.
It should be noted that `Abi::Wasm` was internally used for two
things: The `-Z wasm-c-abi=legacy` ABI that is still used by
default on some wasm targets, and the `extern "wasm"` ABI. Despite
both being `Abi::Wasm` internally, they were not the same. An
explicit `extern "wasm"` additionally enabled the `+multivalue`
feature.
I've opted to remove `Abi::Wasm` in this patch entirely, instead
of keeping it as an ABI with only internal usage. Both
`-Z wasm-c-abi` variants are now treated as part of the normal
C ABI, just with different different treatment in
adjust_for_foreign_abi.
`-Z patchable-function-entry` works like `-fpatchable-function-entry`
on clang/gcc. The arguments are total nop count and function offset.
See MCP rust-lang/compiler-team#704
Add -Zuse-sync-unwind
Currently Rust uses async unwind by default, but async unwind will bring non-negligible size overhead. it would be nice to allow users to choose this.
In addition, async unwind currently prevents LLVM from generate compact unwind for MachO, if one wishes to generate compact unwind for MachO, then also needs this flag.
This is intended to be used for Linux kernel RETHUNK builds.
With this commit (optionally backported to Rust 1.73.0), plus a
patched Linux kernel to pass the flag, I get a RETHUNK build with
Rust enabled that is `objtool`-warning-free and is able to boot in
QEMU and load a sample Rust kernel module.
Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Replace the \01__gnu_mcount_nc to LLVM intrinsic for ARM
Current `-Zinstrument-mcount` for ARM32 use the `\01__gnu_mcount_nc` directly for its instrumentation function.
However, the LLVM does not use this mcount function directly, but it wraps it to intrinsic, `llvm.arm.gnu.eabi.mcount` and the transform pass also only handle the intrinsic.
As a result, current `-Zinstrument-mcount` not work on ARM32. Refer: https://github.com/namhyung/uftrace/issues/1764
This commit replaces the mcount name from native function to the LLVM intrinsic so that the transform pass can handle it.
Current `-Zinstrument-mcount` for ARM32 use the `\01__gnu_mcount_nc`
directly for its instrumentation function.
However, the LLVM does not use this mcount function directly, but it wraps
it to intrinsic, `llvm.arm.gnu.eabi.mcount` and the transform pass also
only handle the intrinsic.
As a result, current `-Zinstrument-mcount` not work on ARM32.
Refer: https://github.com/namhyung/uftrace/issues/1764
This commit replaces the mcount name from native function to the
LLVM intrinsic so that the transform pass can handle it.
Signed-off-by: ChoKyuWon <kyuwoncho18@gmail.com>
I don't know why `SmallStr` was used here; some ad hoc profiling showed
this code is not that hot, the string is usually empty, and when it's
not empty it's usually very short. However, the use of a
`SmallStr<1024>` does result in 1024 byte `memcpy` call on each
execution, which shows up when I do `memcpy` profiling. So using a
normal string makes the code both simpler and very slightly faster.
Adds support for LLVM [SafeStack] which provides backward edge control
flow protection by separating the stack into two parts: data which is
only accessed in provable safe ways is allocated on the normal stack
(the "safe stack") and all other data is placed in a separate allocation
(the "unsafe stack").
SafeStack support is enabled by passing `-Zsanitizer=safestack`.
[SafeStack]: https://clang.llvm.org/docs/SafeStack.html
Add the attributes to functions according to the settings.
"xray-always" overrides "xray-never", and they both override
"xray-ignore-loops" and "xray-instruction-threshold", but we'll
let lints deal with warnings about silly attribute combinations.
