Commit Graph

548 Commits

Author SHA1 Message Date
Matthias Krüger
9570ac4d28
Rollup merge of #123564 - scottmcm:step-by-div-zero, r=joboet
Don't emit divide-by-zero panic paths in `StepBy::len`

I happened to notice today that there's actually two such calls emitted in the assembly: <https://rust.godbolt.org/z/1Wbbd3Ts6>

Since they're impossible, hopefully telling LLVM that will also help optimizations elsewhere.
2024-04-08 22:06:22 +02:00
Matthias Krüger
ecfc3384f1
Rollup merge of #122781 - nikic:ppc-abi-fix, r=cuviper
Fix argument ABI for overaligned structs on ppc64le

When passing a 16 (or higher) aligned struct by value on ppc64le, it needs to be passed as an array of `i128` rather than an array of `i64`. This will force the use of an even starting doubleword.

For the case of a 16 byte struct with alignment 16 it is important that `[1 x i128]` is used instead of `i128` -- apparently, the latter will get treated similarly to `[2 x i64]`, not exhibiting the correct ABI. Add a `force_array` flag to `Uniform` to support this.

The relevant clang code can be found here:
fe2119a7b0/clang/lib/CodeGen/Targets/PPC.cpp (L878-L884)
fe2119a7b0/clang/lib/CodeGen/Targets/PPC.cpp (L780-L784)

I think the corresponding psABI wording is this:

> Fixed size aggregates and unions passed by value are mapped to as
> many doublewords of the parameter save area as the value uses in
> memory. Aggregrates and unions are aligned according to their
> alignment requirements. This may result in doublewords being
> skipped for alignment.

In particular the last sentence. Though I didn't find any wording for Clang's behavior of clamping the alignment to 16.

Fixes https://github.com/rust-lang/rust/issues/122767.

r? `@cuviper`
2024-04-08 22:06:20 +02:00
bors
537aab7a2e Auto merge of #120131 - oli-obk:pattern_types_syntax, r=compiler-errors
Implement minimal, internal-only pattern types in the type system

rebase of https://github.com/rust-lang/rust/pull/107606

You can create pattern types with `std::pat::pattern_type!(ty is pat)`. The feature is incomplete and will panic on you if you use any pattern other than integral range patterns. The only way to create or deconstruct a pattern type is via `transmute`.

This PR's implementation differs from the MCP's text. Specifically

> This means you could implement different traits for different pattern types with the same base type. Thus, we just forbid implementing any traits for pattern types.

is violated in this PR. The reason is that we do need impls after all in order to make them usable as fields. constants of type `std::time::Nanoseconds` struct are used in patterns, so the type must be structural-eq, which it only can be if you derive several traits on it. It doesn't need to be structural-eq recursively, so we can just manually implement the relevant traits on the pattern type and use the pattern type as a private field.

Waiting on:

* [x] move all unrelated commits into their own PRs.
* [x] fix niche computation (see 2db07f94f44f078daffe5823680d07d4fded883f)
* [x] add lots more tests
* [x] T-types MCP https://github.com/rust-lang/types-team/issues/126 to finish
* [x] some commit cleanup
* [x] full self-review
* [x] remove 61bd325da19a918cc3e02bbbdce97281a389c648, it's not necessary anymore I think.
* [ ] ~~make sure we never accidentally leak pattern types to user code (add stability checks or feature gate checks and appopriate tests)~~ we don't even do this for the new float primitives
* [x] get approval that [the scope expansion to trait impls](https://rust-lang.zulipchat.com/#narrow/stream/326866-t-types.2Fnominated/topic/Pattern.20types.20types-team.23126/near/427670099) is ok

r? `@BoxyUwU`
2024-04-08 16:25:23 +00:00
Oli Scherer
84acfe86de Actually create ranged int types in the type system. 2024-04-08 12:02:19 +00:00
Philippe-Cholet
7a2678de7d Add invariant to VecDeque::pop_* that len < cap if pop successful
Similar to #114370 for VecDeque instead of Vec. It now uses `core::hint::assert_unchecked`.
2024-04-08 12:12:13 +02:00
Kai Luo
d8d1e6ce21 Limited to little endian target 2024-04-08 11:11:11 +08:00
Nikita Popov
009280c5e3 Fix argument ABI for overaligned structs on ppc64le
When passing a 16 (or higher) aligned struct by value on ppc64le,
it needs to be passed as an array of `i128` rather than an array
of `i64`. This will force the use of an even starting register.

For the case of a 16 byte struct with alignment 16 it is important
that `[1 x i128]` is used instead of `i128` -- apparently, the
latter will get treated similarly to `[2 x i64]`, not exhibiting
the correct ABI. Add a `force_array` flag to `Uniform` to support
this.

The relevant clang code can be found here:
fe2119a7b0/clang/lib/CodeGen/Targets/PPC.cpp (L878-L884)
fe2119a7b0/clang/lib/CodeGen/Targets/PPC.cpp (L780-L784)

I think the corresponding psABI wording is this:

> Fixed size aggregates and unions passed by value are mapped to as
> many doublewords of the parameter save area as the value uses in
> memory. Aggregrates and unions are aligned according to their
> alignment requirements. This may result in doublewords being
> skipped for alignment.

In particular the last sentence.

Fixes https://github.com/rust-lang/rust/issues/122767.
2024-04-08 11:15:36 +09:00
bors
4e431fad67 Auto merge of #123561 - saethlin:str-unchecked-sub-index, r=scottmcm
Use unchecked_sub in str indexing

https://github.com/rust-lang/rust/pull/108763 applied this logic to indexing for slices, but of course `str` has its own separate impl.

Found this by skimming over the codegen for https://github.com/oxidecomputer/hubris/; their dist builds enable overflow checks so the lack of `unchecked_sub` was producing an impossible-to-hit overflow check and also inhibiting some inlining.

r? scottmcm
2024-04-07 12:49:15 +00:00
bors
0e3235f85b Auto merge of #123555 - DianQK:update-llvm-18, r=cuviper
Update to LLVM 18.1.3

Fixes #122805.

This should work on all targets: https://rust.godbolt.org/z/svW8ha31z.

r? `@cuviper`
2024-04-07 06:33:58 +00:00
DianQK
5acfe772fa
Add the test case for #122805 2024-04-07 13:01:54 +08:00
Scott McMurray
00bd24766f Don't emit divide-by-zero panic paths in StepBy::len
I happened to notice today that there's actually two such calls emitted in the assembly: <https://rust.godbolt.org/z/1Wbbd3Ts6>

Since they're impossible, hopefully telling LLVM that will also help optimizations elsewhere.
2024-04-06 11:37:57 -07:00
Ben Kimock
712aab72df Use unchecked_sub in str indexing 2024-04-06 14:09:03 -04:00
Ben Kimock
a7912cb421 Put checks that detect UB under their own flag below debug_assertions 2024-04-06 11:21:47 -04:00
Matthias Krüger
ad3df4919d
Rollup merge of #123525 - maurer:no-id-dyn2, r=compiler-errors
CFI: Don't rewrite ty::Dynamic directly

Now that we're using a type folder, the arguments in predicates are processed automatically - we don't need to descend manually.

We also want to keep projection clauses around, and this does so.

r? `@compiler-errors`
2024-04-06 08:56:35 +02:00
Matthew Maurer
5083378f16 CFI: Don't rewrite ty::Dynamic directly
Now that we're using a type folder, the arguments in predicates are
processed automatically - we don't need to descend manually.

We also want to keep projection clauses around, and this does so.
2024-04-05 23:58:15 +00:00
Guillaume Gomez
5ceac29123
Rollup merge of #123487 - rcvalle:rust-cfi-restore-typeid-for-instance, r=compiler-errors
CFI: Restore typeid_for_instance default behavior

Restore typeid_for_instance default behavior of performing self type erasure, since it's the most common case and what it does most of the time. Using concrete self (or not performing self type erasure) is for assigning a secondary type id, and secondary type ids are only assigned when they're unique and to methods, and also are only tested for when methods are used as function pointers.
2024-04-05 22:33:27 +02:00
Ramon de C Valle
2498a9d464 CFI: Restore typeid_for_instance default behavior
Restore typeid_for_instance default behavior of performing self type
erasure, since it's the most common case and what it does most of the
time. Using concrete self (or not performing self type erasure) is for
assigning a secondary type id, and secondary type ids are only assigned
when they're unique and to methods, and also are only tested for when
methods are used as function pointers.
2024-04-04 21:19:33 -07:00
许杰友 Jieyou Xu (Joe)
476156aedf
Port issue-7349 to a codegen test 2024-04-04 21:59:08 +01:00
bors
29fe618f75 Auto merge of #123052 - maurer:addr-taken, r=compiler-errors
CFI: Support function pointers for trait methods

Adds support for both CFI and KCFI for function pointers to trait methods by attaching both concrete and abstract types to functions.

KCFI does this through generation of a `ReifyShim` on any function pointer for a method that could go into a vtable, and keeping this separate from `ReifyShim`s that are *intended* for vtable us by setting a `ReifyReason` on them.

CFI does this by setting both the concrete and abstract type on every instance.

This should land after #123024 or a similar PR, as it diverges the implementation of CFI vs KCFI.

r? `@compiler-errors`
2024-04-04 06:40:30 +00:00
Matthias Krüger
bc8415b9e6
Rollup merge of #122619 - erikdesjardins:cast, r=compiler-errors
Fix some unsoundness with PassMode::Cast ABI

Fixes #122617

Reviewable commit-by-commit. More info in each commit message.
2024-04-03 22:11:00 +02:00
bors
76cf07d5df Auto merge of #122225 - DianQK:nits-120268, r=cjgillot
Rename `UninhabitedEnumBranching` to `UnreachableEnumBranching`

Per [#120268](https://github.com/rust-lang/rust/pull/120268#discussion_r1517492060), I rename `UninhabitedEnumBranching` to `UnreachableEnumBranching` .

I solved some nits to add some comments.

I adjusted the workaround restrictions. This should be useful for `a <= b` and `if let Some/Ok(v)`. For enum with few variants, `early-tailduplication` should not cause compile time overhead.

r? RalfJung
2024-04-03 06:22:23 +00:00
bors
88c2f4f5f5 Auto merge of #123385 - matthiaskrgr:rollup-v69vjbn, r=matthiaskrgr
Rollup of 8 pull requests

Successful merges:

 - #123198 (Add fn const BuildHasherDefault::new)
 - #123226 (De-LLVM the unchecked shifts [MCP#693])
 - #123302 (Make sure to insert `Sized` bound first into clauses list)
 - #123348 (rustdoc: add a couple of regression tests)
 - #123362 (Check that nested statics in thread locals are duplicated per thread.)
 - #123368 (CFI: Support non-general coroutines)
 - #123375 (rustdoc: synthetic auto trait impls: accept unresolved region vars for now)
 - #123378 (Update sysinfo to 0.30.8)

Failed merges:

 - #123349 (Fix capture analysis for by-move closure bodies)

r? `@ghost`
`@rustbot` modify labels: rollup
2024-04-02 21:23:53 +00:00
bors
a77322c16f Auto merge of #118310 - scottmcm:three-way-compare, r=davidtwco
Add `Ord::cmp` for primitives as a `BinOp` in MIR

Update: most of this OP was written months ago.  See https://github.com/rust-lang/rust/pull/118310#issuecomment-2016940014 below for where we got to recently that made it ready for review.

---

There are dozens of reasonable ways to implement `Ord::cmp` for integers using comparison, bit-ops, and branches.  Those differences are irrelevant at the rust level, however, so we can make things better by adding `BinOp::Cmp` at the MIR level:

1. Exactly how to implement it is left up to the backends, so LLVM can use whatever pattern its optimizer best recognizes and cranelift can use whichever pattern codegens the fastest.
2. By not inlining those details for every use of `cmp`, we drastically reduce the amount of MIR generated for `derive`d `PartialOrd`, while also making it more amenable to MIR-level optimizations.

Having extremely careful `if` ordering to μoptimize resource usage on broadwell (#63767) is great, but it really feels to me like libcore is the wrong place to put that logic.  Similarly, using subtraction [tricks](https://graphics.stanford.edu/~seander/bithacks.html#CopyIntegerSign) (#105840) is arguably even nicer, but depends on the optimizer understanding it (https://github.com/llvm/llvm-project/issues/73417) to be practical.  Or maybe [bitor is better than add](https://discourse.llvm.org/t/representing-in-ir/67369/2?u=scottmcm)?  But maybe only on a future version that [has `or disjoint` support](https://discourse.llvm.org/t/rfc-add-or-disjoint-flag/75036?u=scottmcm)?  And just because one of those forms happens to be good for LLVM, there's no guarantee that it'd be the same form that GCC or Cranelift would rather see -- especially given their very different optimizers.  Not to mention that if LLVM gets a spaceship intrinsic -- [which it should](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Suboptimal.20inlining.20in.20std.20function.20.60binary_search.60/near/404250586) -- we'll need at least a rustc intrinsic to be able to call it.

As for simplifying it in Rust, we now regularly inline `{integer}::partial_cmp`, but it's quite a large amount of IR.  The best way to see that is with 8811efa88b (diff-d134c32d028fbe2bf835fef2df9aca9d13332dd82284ff21ee7ebf717bfa4765R113) -- I added a new pre-codegen MIR test for a simple 3-tuple struct, and this PR change it from 36 locals and 26 basic blocks down to 24 locals and 8 basic blocks.  Even better, as soon as the construct-`Some`-then-match-it-in-same-BB noise is cleaned up, this'll expose the `Cmp == 0` branches clearly in MIR, so that an InstCombine (#105808) can simplify that to just a `BinOp::Eq` and thus fix some of our generated code perf issues.  (Tracking that through today's `if a < b { Less } else if a == b { Equal } else { Greater }` would be *much* harder.)

---

r? `@ghost`
But first I should check that perf is ok with this
~~...and my true nemesis, tidy.~~
2024-04-02 19:21:44 +00:00
Matthew Maurer
93c2bace58 CFI: Switch sense of type erasure flag
Previously, we had `NO_SELF_TYPE_ERASURE`, a negative configuration. Now
we have `ERASE_SELF_TYPE`, a positive configuration.
2024-04-02 18:24:44 +00:00
Scott McMurray
0601f0c66d De-LLVM the unchecked shifts [MCP#693]
This is just one part of the MCP, but it's the one that IMHO removes the most noise from the standard library code.

Seems net simpler this way, since MIR already supported heterogeneous shifts anyway, and thus it's not more work for backends than before.
2024-03-30 03:32:11 -07:00
bors
877d36b192 Auto merge of #122976 - caibear:optimize_reserve_for_push, r=cuviper
Remove len argument from RawVec::reserve_for_push

Removes `RawVec::reserve_for_push`'s `len` argument since it's always the same as capacity.
Also makes `Vec::insert` use `RawVec::reserve_for_push`.
2024-03-30 00:29:24 +00:00
Cai Bear
4500c83c62 Fix test. 2024-03-29 15:37:43 -07:00
bors
58dcd1fdb9 Auto merge of #123071 - rcvalle:rust-cfi-fix-method-fn-ptr-cast, r=compiler-errors
CFI: Fix methods as function pointer cast

Fix casting between methods and function pointers by assigning a secondary type id to methods with their concrete self so they can be used as function pointers.

This was split off from #116404.

cc `@compiler-errors` `@workingjubilee`
2024-03-29 09:04:05 +00:00
bors
db2f9759f4 Auto merge of #122671 - Mark-Simulacrum:const-panic-msg, r=Nilstrieb
Codegen const panic messages as function calls

This skips emitting extra arguments at every callsite (of which there
can be many). For a librustc_driver build with overflow checks enabled,
this cuts 0.7MB from the resulting shared library (see [perf]).

A sample improvement from nightly:

```
        leaq    str.0(%rip), %rdi
        leaq    .Lalloc_d6aeb8e2aa19de39a7f0e861c998af13(%rip), %rdx
        movl    $25, %esi
        callq   *_ZN4core9panicking5panic17h17cabb89c5bcc999E@GOTPCREL(%rip)
```

to this PR:

```
        leaq    .Lalloc_d6aeb8e2aa19de39a7f0e861c998af13(%rip), %rdi
        callq   *_RNvNtNtCsduqIKoij8JB_4core9panicking11panic_const23panic_const_div_by_zero@GOTPCREL(%rip)
```

[perf]: https://perf.rust-lang.org/compare.html?start=a7e4de13c1785819f4d61da41f6704ed69d5f203&end=64fbb4f0b2d621ff46d559d1e9f5ad89a8d7789b&stat=instructions:u
2024-03-29 00:24:01 +00:00
DianQK
ec359f7d9f
Restore the test checks for wider_reduce_into_iter
The current minimum support is for LLVM 17.
2024-03-28 21:28:45 +08:00
Ramon de C Valle
8e6b4e91b6 CFI: Fix methods as function pointer cast
Fix casting between methods and function pointers by assigning a
secondary type id to methods with their concrete self so they can be
used as function pointers.
2024-03-27 16:19:17 -07:00
Matthias Krüger
6464e5b78c
Rollup merge of #123075 - rcvalle:rust-cfi-fix-drop-drop-in-place, r=compiler-errors
CFI: Fix drop and drop_in_place

Fix drop and drop_in_place by transforming self of drop and drop_in_place methods into a Drop trait objects.

This was split off from https://github.com/rust-lang/rust/pull/116404.

cc `@compiler-errors` `@workingjubilee`
2024-03-27 23:27:22 +01:00
Ramon de C Valle
0b860818e6 CFI: Fix drop and drop_in_place
Fix drop and drop_in_place by transforming self of drop and
drop_in_place methods into Drop trait objects.
2024-03-27 12:52:14 -07:00
clubby789
b500693ad7 Don't emit load metadata in debug mode 2024-03-25 18:32:45 +00:00
Scott McMurray
3da115a93b Add+Use mir::BinOp::Cmp 2024-03-23 23:23:41 -07:00
Jubilee
b9b65f816d
Rollup merge of #122875 - maurer:cfi-transparent-termination, r=workingjubilee
CFI: Support self_cell-like recursion

Current `transform_ty` attempts to avoid cycles when normalizing `#[repr(transparent)]` types to their interior, but runs afoul of this pattern used in `self_cell`:

```
struct X<T> {
  x: u8,
  p: PhantomData<T>,
}

 #[repr(transparent)]
struct Y(X<Y>);
```

When attempting to normalize Y, it will still cycle indefinitely. By using a types-visited list, this will instead get expanded exactly one layer deep to X<Y>, and then stop, not attempting to normalize `Y` any further.

This PR was split off from #121962 as part of fixing the larger vtable compatibility issues.

r? ``````@workingjubilee``````
2024-03-23 22:59:42 -07:00
bors
d6eb0f5a09 Auto merge of #122582 - scottmcm:swap-intrinsic-v2, r=oli-obk
Let codegen decide when to `mem::swap` with immediates

Making `libcore` decide this is silly; the backend has so much better information about when it's a good idea.

Thus this PR introduces a new `typed_swap` intrinsic with a fallback body, and replaces that fallback implementation when swapping immediates or scalar pairs.

r? oli-obk

Replaces #111744, and means we'll never need more libs PRs like #111803 or #107140
2024-03-23 13:57:55 +00:00
Matthew Maurer
dec36c3d6e CFI: Support self_cell-like recursion
Current `transform_ty` attempts to avoid cycles when normalizing
`#[repr(transparent)]` types to their interior, but runs afoul of this
pattern used in `self_cell`:

```
struct X<T> {
  x: u8,
  p: PhantomData<T>,
}

 #[repr(transparent)]
struct Y(X<Y>);
```

When attempting to normalize Y, it will still cycle indefinitely. By
using a types-visited list, this will instead get expanded exactly
one layer deep to X<Y>, and then stop, not attempting to normalize `Y`
any further.
2024-03-22 23:02:05 +00:00
Mark Rousskov
00f4daa276 Codegen const panic messages as function calls
This skips emitting extra arguments at every callsite (of which there
can be many). For a librustc_driver build with overflow checks enabled,
this cuts 0.7MB from the resulting binary.
2024-03-22 09:55:50 -04:00
bors
7762adccb2 Auto merge of #122456 - maurer:cfi-nonpassed, r=workingjubilee
CFI: Skip non-passed arguments

Rust will occasionally rely on fn((), X) -> Y being compatible with fn(X) -> Y, since () is a non-passed argument. Relax CFI by choosing not to encode non-passed arguments.

This PR was split off from #121962 as part of fixing the larger vtable compatibility issues.

r? `@workingjubilee`
2024-03-22 06:09:40 +00:00
Matthew Maurer
f2f0d255df CFI: Skip non-passed arguments
Rust will occasionally rely on fn((), X) -> Y being compatible with
fn(X) -> Y, since () is a non-passed argument. Relax CFI by choosing not
to encode non-passed arguments.
2024-03-21 22:26:26 +00:00
clubby789
5f254d8b66 Remove SpecOptionPartialEq 2024-03-19 16:32:01 +00:00
bors
148a41c6b5 Auto merge of #122375 - rcvalle:rust-cfi-break-tests-into-smaller-files, r=compiler-errors
CFI: Break tests into smaller files

Break type metadata identifiers tests into smaller set of tests/files, and move CFI (and KCFI) codegen tests to a cfi (and kcfi) subdirectory,
2024-03-19 02:17:52 +00:00
Scott McMurray
6d2cb39ac5 Stop whining, tidy 2024-03-17 12:51:58 -07:00
Scott McMurray
7d537106a1 Let codegen decide when to mem::swap with immediates
Making `libcore` decide this is silly; the backend has so much better information about when it's a good idea.

So introduce a new `typed_swap` intrinsic with a fallback body, but replace that implementation for immediates and scalar pairs.
2024-03-17 11:59:18 -07:00
Josh Stone
d9132de4ab Remove an obsolete ignore-llvm-version 2024-03-17 10:52:00 -07:00
Erik Desjardins
dec81ac223 disable crashing test on sparc 2024-03-17 13:40:27 -04:00
Josh Stone
29430554f6 Update the minimum external LLVM to 17 2024-03-17 10:11:04 -07:00
Erik Desjardins
8d5fd94e62 add tests for PassMode::Cast fixes
Tests added in cast-target-abi.rs, covering the single element, array,
and prefix cases in `CastTarget::llvm_type`, and the Rust-is-larger/smaller
cases in the Rust<->ABI copying code.

ffi-out-of-bounds-loads.rs was overhauled to be runnable on any
platform. Its alignment also increases due to the removal of a `min` in
the previous commit; this was probably an insufficient workaround for
this issue or similar. The higher alignment is fine, since the alloca is
actually aligned to 8 bytes, as the test checks now confirm.
2024-03-17 00:39:21 -04:00
bors
c563f2ee79 Auto merge of #122371 - oli-obk:visit_nested_body, r=tmiasko
Stop walking the bodies of statics for reachability, and evaluate them instead

cc `@saethlin` `@RalfJung`

cc #119214

This reuses the `DefIdVisitor` from `rustc_privacy`, because they basically try to do the same thing.

This PR's changes can probably be extended to constants, too, but let's tackle that separately, it's likely more involved.
2024-03-16 04:35:02 +00:00