Allow more top-down inlining for single-BB callees
This means that things like `<usize as Step>::forward_unchecked` and `<PartialOrd for f32>::le` will inline even if
we've already done a bunch of inlining to find the calls to them.
Fixes#138136
~~Draft as it's built atop #138135, which adds a mir-opt test that's a nice demonstration of this. To see just this change, look at <48f63e3be5>~~ Rebased to be just the inlining change, as the other existing tests show it great.
Don't `alloca` just to look at a discriminant
Today we're making LLVM do a bunch of extra work when you match on trivial stuff like `Option<bool>` or `ControlFlow<u8>`.
This PR changes that so that simple types like `Option<u32>` or `Result<(), Box<Error>>` can stay as `OperandValue::ScalarPair` and we can still read the discriminant from them, rather than needing to write them into memory to have a `PlaceValue` just to get the discriminant out.
Fixes#137503
naked functions: on windows emit `.endef` without the symbol name
tracking issue: https://github.com/rust-lang/rust/issues/90957
fixes https://github.com/rust-lang/rust/issues/138320
The `.endef` directive does not take the name as an argument. Apparently the LLVM x86_64 parser does accept this, but on i686 it's rejected. In general `i686` does some special name mangling stuff, so it's good to include it in the naked function tests.
r? ````@ChrisDenton```` (because windows)
This means that things like `<usize as Step>::forward_unchecked` and `<PartialOrd for f32>::le` will inline even if we've already done a bunch of inlining to find the calls to them.
Apply dllimport in ThinLTO
This partially reverts https://github.com/rust-lang/rust/pull/103353 by properly applying `dllimport` if `-Z dylib-lto` is passed. That PR should probably fully be reverted as it looks quite sketchy. We don't know locally if the entire crate graph would be statically linked.
This should hopefully be sufficient to make ThinLTO work for rustc on Windows.
r? ``@wesleywiser``
---
Edit: This PR is changed to just generally revert https://github.com/rust-lang/rust/pull/103353.
Don't re-`assume` in `transmute`s that don't change niches
I noticed in nightly 2025-02-21 that `transmute` is emitting way more `assume`s than necessary for newtypes.
For example, the three transmutes in <https://rust.godbolt.org/z/fW1KaTc4o> emits
```rust
define noundef range(i32 1, 0) i32 `@repeatedly_transparent_transmute(i32` noundef range(i32 1, 0) %_1) unnamed_addr {
start:
%0 = sub i32 %_1, 1
%1 = icmp ule i32 %0, -2
call void `@llvm.assume(i1` %1)
%2 = sub i32 %_1, 1
%3 = icmp ule i32 %2, -2
call void `@llvm.assume(i1` %3)
%4 = sub i32 %_1, 1
%5 = icmp ule i32 %4, -2
call void `@llvm.assume(i1` %5)
%6 = sub i32 %_1, 1
%7 = icmp ule i32 %6, -2
call void `@llvm.assume(i1` %7)
%8 = sub i32 %_1, 1
%9 = icmp ule i32 %8, -2
call void `@llvm.assume(i1` %9)
%10 = sub i32 %_1, 1
%11 = icmp ule i32 %10, -2
call void `@llvm.assume(i1` %11)
ret i32 %_1
}
```
But those are all just newtypes that don't change size or niches, so none of it's needed.
After this PR it's down to just
```rust
define noundef range(i32 1, 0) i32 `@repeatedly_transparent_transmute(i32` noundef range(i32 1, 0) %_1) unnamed_addr {
start:
ret i32 %_1
}
```
because none of those `assume`s in the original actually did anything.
(Transmuting to something with a difference niche, though, still has the assumes -- the other tests continue to pass checking that.)
Don't include global asm in `mir_keys`, fix error body synthesis
r? oli-obk
Fixes#137470Fixes#137471Fixes#137472Fixes#137473
try-job: test-various
try-job: x86_64-apple-2
Break critical edges in inline asm before code generation
An inline asm terminator defines outputs along its target edges -- a
fallthrough target and labeled targets. Code generation implements this
by inserting code directly into the target blocks. This approach works
only if the target blocks don't have other predecessors.
Establish required invariant by extending existing code that breaks
critical edges before code generation.
Fixes#137867.
r? ``@bjorn3``
An inline asm terminator defines outputs along its target edges -- a
fallthrough target and labeled targets. Code generation implements this
by inserting code directly into the target blocks. This approach works
only if the target blocks don't have other predecessors.
Establish required invariant by extending existing code that breaks
critical edges before code generation.
Rollup of 12 pull requests
Successful merges:
- #135767 (Future incompatibility warning `unsupported_fn_ptr_calling_conventions`: Also warn in dependencies)
- #137852 (Remove layouting dead code for non-array SIMD types.)
- #137863 (Fix pretty printing of unsafe binders)
- #137882 (do not build additional stage on compiler paths)
- #137894 (Revert "store ScalarPair via memset when one side is undef and the other side can be memset")
- #137902 (Make `ast::TokenKind` more like `lexer::TokenKind`)
- #137921 (Subtree update of `rust-analyzer`)
- #137922 (A few cleanups after the removal of `cfg(not(parallel))`)
- #137939 (fix order on shl impl)
- #137946 (Fix docker run-local docs)
- #137955 (Always allow rustdoc-json tests to contain long lines)
- #137958 (triagebot.toml: Don't label `test/rustdoc-json` as A-rustdoc-search)
r? `@ghost`
`@rustbot` modify labels: rollup
test(codegen): add looping_over_ne_bytes test for #133528
Adds test for #133528.
I renamed the function to `looping_over_ne_bytes` to better reflect that it is doing.
I also set the min llvm version to 20 as this was presumably a llvm bug that was fixed in version 20.
I didn't tie the test to any specific architecture, as we are testing llvm output.
tests: adapt for LLVM 21 changes
Per discussion in #137799 we don't really need this readonly attribute, so let's just drop it so the test passes on LLVM 21.
Fixes#137799.
Rollup of 8 pull requests
Successful merges:
- #136503 (Tweak output of const panic diagnostic)
- #137390 (tests: fix up new test for nocapture -> capture(none) change)
- #137617 (Introduce `feature(generic_const_parameter_types)`)
- #137719 (Add missing case explanation for doc inlined re-export of doc hidden item)
- #137763 (Use `mk_ty_from_kind` a bit less, clean up lifetime handling in borrowck)
- #137769 (Do not yeet `unsafe<>` from type when formatting unsafe binder)
- #137776 (Some `rustc_transmute` cleanups)
- #137800 (Remove `ParamEnv::without_caller_bounds`)
r? `@ghost`
`@rustbot` modify labels: rollup
tests: fix up new test for nocapture -> capture(none) change
Same change as #136287, but for a newly introduced test.
``@rustbot`` label llvm-main
r? ``@nikic``
The embedded bitcode should always be prepared for LTO/ThinLTO
Fixes#115344. Fixes#117220.
There are currently two methods for generating bitcode that used for LTO. One method involves using `-C linker-plugin-lto` to emit object files as bitcode, which is the typical setting used by cargo. The other method is through `-C embed-bitcode=yes`.
When using with `-C embed-bitcode=yes -C lto=no`, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module.
This PR is doing something similar to LLVM's `buildFatLTODefaultPipeline`, obtaining the bitcode for embedding after running `buildThinLTOPreLinkDefaultPipeline`.
r? nikic
tests: use minicore more
minicore makes it much easier to add new language items to all of the existing `no_core` tests.
Most of the remaining tests that *could* use minicore either fail because..
1. LLVM IR output changes and doesn't pass the test as written. I didn't look into these further.
2. The test has revisions w/ different compilation flags, expecting some to fail, and when using minicore, minicore is compiled with those flags and fails in the expected way because of the flags rather than the test, and that's considered a failure.
But these tests can be changed and make adding new language items a lot easier.
r? ```@jieyouxu```
import `simd_` intrinsics
In most cases, we can import the simd intrinsics rather than redeclare them. Apparently, most of these tests were written before `std::intrinsics::simd` existed.
There are a couple of exceptions where we can't yet import:
- the intrinsics are not declared as `const fn` in the standard library, causing issues in the `const-eval` tests
- the `simd_shuffle_generic` function is not exposed from `std::intrinsics`
- the `simd_fpow` and `simd_fpowi` functions are not exposed from `std::intrinsics` (removed in https://github.com/rust-lang/rust/pull/137595)
- some tests use `no_core`, and therefore cannot use `std::intrinsics`
r? ```@RalfJung```
cc ```@workingjubilee``` do you have context on why some intrinsics are not exposed?
Update some comparison codegen tests now that they pass in LLVM20
Fixes#106107
Needed one tweak to the default `PartialOrd::le` to get the test to pass. Everything but the derived 2-field `le` test passes even without the change to the defaults in the trait.
Same motivation as #136287, but for a newly introduced test. Rather than
over-constraining here, we just match the sret and accept pretty much
all other attributes.
@rustbot label llvm-main
r? @nikic
remove `simd_fpow` and `simd_fpowi`
Discussed in https://github.com/rust-lang/rust/issues/137555
These functions are not exposed from `std::intrinsics::simd`, and not used anywhere outside of the compiler. They also don't lower to particularly good code at least on the major ISAs (I checked x86_64, aarch64, s390x, powerpc), where the vector is just spilled to the stack and scalar functions are used for the actual logic.
r? `@RalfJung`
Tighten `str-to-string-128690.rs``CHECK{,-NOT}`s to make it less likely to incorrectly fail with symbol name mangling
The `invoke` to match on to `CHECK` or `CHECK-NOT` (latest master) looks like
```llvm
%_0.i.i.i.i.i.i.i.i.i.i.i.i.i1.i = invoke noundef zeroext i1 ``@"_ZN42_$LT$str$u20$as$u20$core..fmt..Display$GT$3fmt17ha18033e7fb4f14fcE"(ptr`` noalias noundef nonnull readonly align 1 %_3.val.i.i.i.i.i.i.i.i.i.i.i.i.i, i64 noundef %_3.val1.i.i.i.i.i.i.i.i.i.i.i.i.i, ptr noalias noundef nonnull align 8 dereferenceable(64) %formatter.i)
to label %bb1.i unwind label %cleanup.i, !noalias !80
```
in the local `.ll` output.
This test incorrectly failed in https://github.com/rust-lang/rust/pull/137483#issuecomment-2676925819 due to
```
// CHECK-NOT: {{(call|invoke).*}}fmt
```
matching against the unrelated call
```llvm
tail call void ``@_RNvNtCseLfmtnDCoTB_5alloc7raw_vec12handle_error``
```
It's not pretty by any means, but...
r? ``@saethlin``
Emit getelementptr inbounds nuw for pointer::add()
Lower pointer::add (via intrinsic::offset with unsigned offset) to getelementptr inbounds nuw on LLVM versions that support it. This lets LLVM make use of the pre-condition that the offset addition does not wrap in an unsigned sense. Together with inbounds, this also implies that the offset is non-negative.
Fixes https://github.com/rust-lang/rust/issues/137217.
intrinsics: unify rint, roundeven, nearbyint in a single round_ties_even intrinsic
LLVM has three intrinsics here that all do the same thing (when used in the default FP environment). There's no reason Rust needs to copy that historically-grown mess -- let's just have one intrinsic and leave it up to the LLVM backend to decide how to lower that.
Suggested by `@hanna-kruppe` in https://github.com/rust-lang/rust/issues/136459; Cc `@tgross35`
try-job: test-various
Reduce `Box::default` stack copies in debug mode
The `Box::new(T::default())` implementation of `Box::default` only
had two stack copies in debug mode, compared to the current version,
which has four. By avoiding creating any `MaybeUninit<T>`'s and just writing
`T` directly to the `Box` pointer, the stack usage in debug mode remains
the same as the old version.
Another option would be to mark `Box::write` as `#[inline(always)]`,
and change it's implementation to to avoid calling `MaybeUninit::write`
(which creates a `MaybeUninit<T>` on the stack) and to use `ptr::write` instead.
Fixes: #136043
Do not ignore uninhabited types for function-call ABI purposes. (Remove BackendRepr::Uninhabited)
Accepted MCP: https://github.com/rust-lang/compiler-team/issues/832Fixes#135802
Do not consider the inhabitedness of a type for function call ABI purposes.
* Remove the [`rustc_abi::BackendRepr::Uninhabited`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_abi/enum.BackendRepr.html) variant
* Instead calculate the `BackendRepr` of uninhabited types "normally" (as though they were not uninhabited "at the top level", but still considering inhabitedness of variants to determine enum layout, etc)
* Add an `uninhabited: bool` field to [`rustc_abi::LayoutData`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_abi/struct.LayoutData.html) so inhabitedness of a `LayoutData` can still be queried when necessary (e.g. when determining if an enum variant needs a tag value allocated to it).
This should not affect type layouts (size/align/field offset); this should only affect function call ABI, and only of uninhabited types.
cc ``@RalfJung``
Create a generic AVR target: avr-none
This commit removes the `avr-unknown-gnu-atmega328` target and replaces it with a more generic `avr-none` variant that must be specialized using `-C target-cpu` (e.g. `-C target-cpu=atmega328p`).
Seizing the day, I'm adding myself as the maintainer of this target - I've been already fixing the bugs anyway, might as well make it official 🙂
Related discussions:
- https://github.com/rust-lang/rust/pull/131171
- https://github.com/rust-lang/compiler-team/issues/800
try-job: x86_64-gnu-debug
Fix codegen of uninhabited PassMode::Indirect return types.
Add codegen test for uninhabited PassMode::Indirect return types.
Enable optimizations for uninhabited return type codegen test
