Commit Graph

660 Commits

Author SHA1 Message Date
Jubilee Young
6d5d8b5071 cg_clif: rustc_abi::Abi => BackendRepr 2024-10-29 15:01:01 -07:00
Jubilee Young
9015c6deca cg_clif: Rename LayoutS to LayoutData 2024-10-27 22:32:40 -07:00
Deadbeef
f6fea83342 Effects cleanup
- removed extra bits from predicates queries that are no longer needed in the new system
- removed the need for `non_erasable_generics` to take in tcx and DefId, removed unused arguments in callers
2024-10-26 10:19:07 +08:00
Michael Goulet
5cf8107aa6 Fix tests 2024-10-19 18:07:35 +00:00
Michael Goulet
e3800a1a04 Allow dropping dyn principal 2024-10-17 20:43:31 +02:00
Jed Brown
0d8a978e8a intrinsics.fmuladdf{16,32,64,128}: expose llvm.fmuladd.* semantics
Add intrinsics `fmuladd{f16,f32,f64,f128}`. This computes `(a * b) +
c`, to be fused if the code generator determines that (i) the target
instruction set has support for a fused operation, and (ii) that the
fused operation is more efficient than the equivalent, separate pair
of `mul` and `add` instructions.

https://llvm.org/docs/LangRef.html#llvm-fmuladd-intrinsic

MIRI support is included for f32 and f64.

The codegen_cranelift uses the `fma` function from libc, which is a
correct implementation, but without the desired performance semantic. I
think this requires an update to cranelift to expose a suitable
instruction in its IR.

I have not tested with codegen_gcc, but it should behave the same
way (using `fma` from libc).
2024-10-11 15:32:56 -06:00
Jubilee Young
b3beb4efc7 cg_clif: Factor out rustc_target::abi 2024-10-08 18:24:56 -07:00
Folkert de Vries
5fc60d1e52 various fixes for naked_asm! implementation
- fix for divergence
- fix error message
- fix another cranelift test
- fix some cranelift things
- don't set the NORETURN option for naked asm
- fix use of naked_asm! in doc comment
- fix use of naked_asm! in run-make test
- use `span_bug` in unreachable branch
2024-10-06 19:00:09 +02:00
Urgau
018ba0528f Use wide pointers consistenly across the compiler 2024-10-04 14:06:48 +02:00
Michael Goulet
eb75d20a55 Relax a debug assertion in codegen 2024-09-30 12:18:02 -04:00
Michael Goulet
3209943604 Add a debug assertion in codegen that unsize casts of the same principal trait def id are truly NOPs 2024-09-25 11:13:59 -04:00
Lukas Markeffsky
bd31e3ed70 be even more precise about "cast" vs "coercion" 2024-09-24 23:12:02 +02:00
Lukas Markeffsky
46ecb23198 unify dyn* coercions with other pointer coercions 2024-09-24 22:17:55 +02:00
Michael Goulet
702a644b74 Check vtable projections for validity in miri 2024-09-23 19:38:26 -04:00
bjorn3
6b042f5b5d Rustfmt 2024-09-23 11:28:15 +00:00
bjorn3
b40fe1ee28 Merge commit '6d35b4c9a04580366fd800692a5b5db79d766530' into sync_cg_clif-2024-09-22 2024-09-23 11:20:46 +00:00
Michael Goulet
c682aa162b Reformat using the new identifier sorting from rustfmt 2024-09-22 19:11:29 -04:00
Folkert de Vries
1ddd67a79a add C-cmse-nonsecure-entry ABI 2024-09-21 13:04:14 +02:00
Michael Goulet
914193c8f4 Do not unnecessarily eval consts in codegen 2024-09-20 20:38:11 -04:00
Nicholas Nethercote
acb832d640 Use associative type defaults in {Layout,FnAbi}OfHelpers.
This avoids some repetitive boilerplate code.
2024-09-17 10:25:06 +10:00
Ralf Jung
60ee1b7ac6 simd_shuffle: require index argument to be a vector 2024-09-14 14:43:24 +02:00
Rémy Rakic
c646b46b52 introduce PrettyPrintMirOptions for cosmetic MIR dump options
initially starting with `-Z mir-include-spans` because we want them in
the NLL mir dump pass
2024-08-30 07:07:28 +00:00
Trevor Gross
d2ff033302
Rollup merge of #128731 - RalfJung:simd-shuffle-vector, r=workingjubilee
simd_shuffle intrinsic: allow argument to be passed as vector

See https://github.com/rust-lang/rust/issues/128738 for context.

I'd like to get rid of [this hack](6c0b89dfac/compiler/rustc_codegen_ssa/src/mir/block.rs (L922-L935)). https://github.com/rust-lang/rust/pull/128537 almost lets us do that since constant SIMD vectors will then be passed as immediate arguments. However, simd_shuffle for some reason actually takes an *array* as argument, not a vector, so the hack is still required to ensure that the array becomes an immediate (which then later stages of codegen convert into a vector, as that's what LLVM needs).

This PR prepares simd_shuffle to also support a vector as the `idx` argument. Once this lands, stdarch can hopefully be updated to pass `idx` as a vector, and then support for arrays can be removed, which finally lets us get rid of that hack.
2024-08-27 01:46:50 -05:00
Ralf Jung
35709be02d rename AddressOf -> RawBorrow inside the compiler 2024-08-18 19:46:53 +02:00
Chris Denton
0156eb57a1
Always use ar_archive_writer for import libs 2024-08-17 19:10:46 +00:00
bors
e9c965df7b Auto merge of #128812 - nnethercote:shrink-TyKind-FnPtr, r=compiler-errors
Shrink `TyKind::FnPtr`.

By splitting the `FnSig` within `TyKind::FnPtr` into `FnSigTys` and `FnHeader`, which can be packed more efficiently. This reduces the size of the hot `TyKind` type from 32 bytes to 24 bytes on 64-bit platforms. This reduces peak memory usage by a few percent on some benchmarks. It also reduces cache misses and page faults similarly, though this doesn't translate to clear cycles or wall-time improvements on CI.

r? `@compiler-errors`
2024-08-14 00:56:53 +00:00
Ralf Jung
194baa820d simd_shuffle intrinsic: allow argument to be passed as vector (not just as array) 2024-08-13 07:51:17 +02:00
Guillaume Gomez
095ca33bb6
Rollup merge of #128149 - RalfJung:nontemporal_store, r=jieyouxu,Amanieu,Jubilee
nontemporal_store: make sure that the intrinsic is truly just a hint

The `!nontemporal` flag for stores in LLVM *sounds* like it is just a hint, but actually, it is not -- at least on x86, non-temporal stores need very special treatment by the programmer or else the Rust memory model breaks down. LLVM still treats these stores as-if they were normal stores for optimizations, which is [highly dubious](https://github.com/llvm/llvm-project/issues/64521). Let's avoid all that dubiousness by making our own non-temporal stores be truly just a hint, which is possible on some targets (e.g. ARM). On all other targets, non-temporal stores become regular stores.

~~Blocked on https://github.com/rust-lang/stdarch/pull/1541 propagating to the rustc repo, to make sure the `_mm_stream` intrinsics are unaffected by this change.~~

Fixes https://github.com/rust-lang/rust/issues/114582
Cc `@Amanieu` `@workingjubilee`
2024-08-12 17:09:14 +02:00
bjorn3
6e5873c08c Use ar_archive_writer from sysroot for cg_clif 2024-08-09 17:23:50 +00:00
bjorn3
af7ea3135d Merge commit '69b3f5a426a5c1c05236a45b36f6679d95fbe01b' into sync_cg_clif-2024-08-09 2024-08-09 17:18:46 +00:00
Nicholas Nethercote
c4717cc9d1 Shrink TyKind::FnPtr.
By splitting the `FnSig` within `TyKind::FnPtr` into `FnSigTys` and
`FnHeader`, which can be packed more efficiently. This reduces the size
of the hot `TyKind` type from 32 bytes to 24 bytes on 64-bit platforms.
This reduces peak memory usage by a few percent on some benchmarks. It
also reduces cache misses and page faults similarly, though this doesn't
translate to clear cycles or wall-time improvements on CI.
2024-08-09 14:33:25 +10:00
Michael Goulet
b916431976 Rename struct_tail_erasing_lifetimes to struct_tail_for_codegen 2024-08-08 12:15:16 -04:00
Ralf Jung
28e0907111 nontemporal_store: make sure that the intrinsic is truly just a hint 2024-08-05 10:57:14 +02:00
bjorn3
3c987cbe02 Move computation of decorated names out of the create_dll_import_lib method 2024-07-30 10:32:32 +00:00
bjorn3
ee89db9b17 Move temp file name generation out of the create_dll_import_lib method 2024-07-30 10:10:41 +00:00
Nicholas Nethercote
84ac80f192 Reformat use declarations.
The previous commit updated `rustfmt.toml` appropriately. This commit is
the outcome of running `x fmt --all` with the new formatting options.
2024-07-29 08:26:52 +10:00
GnomedDev
db8cdc5d37
Use Cow<'static, str> for InlineAsmTemplatePiece::String 2024-07-24 21:11:55 +01:00
Matthias Krüger
9a6f8ccf3a
Rollup merge of #127779 - momvart:should_codegen_hook, r=cjgillot
Add a hook for `should_codegen_locally`

This PR lifts the module-local function `should_codegen_locally` to `TyCtxt` as a hook.
In addition to monomorphization, this function is used for checking the dependency of `compiler_builtins` on other libraries. Moving this function to the hooks also makes overriding it possible for the tools that use the rustc interface.
2024-07-20 13:24:52 +02:00
Yuri Astrakhan
aef0e346de Avoid ref when using format! in compiler
Clean up a few minor refs in `format!` macro, as it has a performance cost. Apparently the compiler is unable to inline `format!("{}", &variable)`, and does a run-time double-reference instead (format macro already does one level referencing).  Inlining format args prevents accidental `&` misuse.
2024-07-19 14:52:07 -04:00
Trevor Gross
63f239c89f
Rollup merge of #124033 - bjorn3:ar_archive_writer_0_3_0, r=davidtwco
Sync ar_archive_writer to LLVM 18.1.3

From LLVM 15.0.0-rc3. This adds support for COFF archives containing Arm64EC object files and has various fixes for AIX big archive files.
2024-07-16 16:15:13 -05:00
Mohammad Omidvar
9b80250abb Move compiler_builtin check to the use case 2024-07-15 23:43:52 +00:00
bjorn3
6ff363f0ff Merge commit '659243d85c7489412bd0faa1c068d904a6042941' into sync_cg_clif-2024-07-13 2024-07-13 18:39:03 +00:00
bors
9af6fee87d Auto merge of #113128 - WaffleLapkin:become_trully_unuwuable, r=oli-obk,RalfJung
Support tail calls in mir via `TerminatorKind::TailCall`

This is one of the interesting bits in tail call implementation — MIR support.

This adds a new `TerminatorKind` which represents a tail call:
```rust
    TailCall {
        func: Operand<'tcx>,
        args: Vec<Operand<'tcx>>,
        fn_span: Span,
    },
```

*Structurally* this is very similar to a normal `Call` but is missing a few fields:
- `destination` — tail calls don't write to destination, instead they pass caller's destination to the callee (such that eventual `return` will write to the caller of the function that used tail call)
- `target` — similarly to `destination` tail calls pass the caller's return address to the callee, so there is nothing to do
- `unwind` — I _think_ this is applicable too, although it's a bit confusing
- `call_source` — `become` forbids operators and is not created as a lowering of something else; tail calls always come from HIR (at least for now)

It might be helpful to read the interpreter implementation to understand what `TailCall` means exactly, although I've tried documenting it too.

-----

There are a few `FIXME`-questions still left, ideally we'd be able to answer them during review ':)

-----

r? `@oli-obk`
cc `@scottmcm` `@DrMeepster` `@JakobDegen`
2024-07-08 04:35:04 +00:00
bjorn3
58e551433d Sync ar_archive_writer to LLVM 18.1.3
From LLVM 15.0.0-rc3. This adds support for COFF archives containing
Arm64EC object files and has various fixes for AIX big archive files.
2024-07-07 16:56:35 +00:00
Maybe Waffle
484152d562 Support tail calls in mir via TerminatorKind::TailCall 2024-07-07 17:11:04 +02:00
bors
489233170a Auto merge of #123781 - RalfJung:miri-fn-identity, r=oli-obk
Miri function identity hack: account for possible inlining

Having a non-lifetime generic is not the only reason a function can be duplicated. Another possibility is that the function may be eligible for cross-crate inlining. So also take into account the inlining attribute in this Miri hack for function pointer identity.

That said, `cross_crate_inlinable` will still sometimes return true even for `inline(never)` functions:
- when they are `DefKind::Ctor(..) | DefKind::Closure` -- I assume those cannot be `InlineAttr::Never` anyway?
- when `cross_crate_inline_threshold == InliningThreshold::Always`

so maybe this is still not quite the right criterion to use for function pointer identity.
2024-07-04 23:45:56 +00:00
Michael Goulet
3273ccea4b Fix spans 2024-07-02 15:48:48 -04:00
Michael Goulet
9dc129ae82 Give Instance::expect_resolve a span 2024-07-02 15:48:48 -04:00
Ralf Jung
41b98da42d Miri function identity hack: account for possible inlining 2024-07-02 21:05:30 +02:00
bjorn3
9ec6a02ab3 Merge commit '49cd5dd454d0115cfbe9e39102a8b3ba4616aa40' into sync_cg_clif-2024-06-30 2024-06-30 11:28:14 +00:00