nordic-dev.net/rust
nordic-dev.net/rust
2
0
Fork 0
mirror of https://github.com/rust-lang/rust.git synced 2024-11-26 08:44:35 +00:00
Code Issues Packages Projects Releases Wiki Activity
256,578 Commits 7 Branches 139 Tags 1.9 GiB
22d0073d47
Commit Graph

616 Commits

Author SHA1 Message Date
Ramon de C Valle
dee4e02102 Add initial support for DataFlowSanitizer 
Adds initial support for DataFlowSanitizer to the Rust compiler. It
currently supports `-Zsanitizer-dataflow-abilist`. Additional options
for it can be passed to LLVM command line argument processor via LLVM
arguments using `llvm-args` codegen option (e.g.,
`-Cllvm-args=-dfsan-combine-pointer-labels-on-load=false`).
 2024-03-01 18:50:40 -08:00
Kornel
78fb977d6b try_with_capacity for Vec, VecDeque, String 
#91913
 2024-03-01 18:24:02 +00:00
Guillaume Gomez
36bd9ef5a8
Rollup merge of #120820 - CKingX:cpu-base-minimum, r=petrochenkov,ChrisDenton 
Enable CMPXCHG16B, SSE3, SAHF/LAHF and 128-bit Atomics (in nightly) in Windows x64

As Rust plans to set Windows 10 as the minimum supported OS for target x86_64-pc-windows-msvc, I have added the cmpxchg16b and sse3 feature. Windows 10 requires CMPXCHG16B, LAHF/SAHF, and PrefetchW as stated in the requirements [here](https://download.microsoft.com/download/c/1/5/c150e1ca-4a55-4a7e-94c5-bfc8c2e785c5/Windows%2010%20Minimum%20Hardware%20Requirements.pdf). Furthermore, CPUs that meet these requirements also have SSE3 ([see](https://walbourn.github.io/directxmath-sse3-and-ssse3/))
 2024-02-29 17:08:36 +01:00
Guillaume Gomez
b2c3279984
Rollup merge of #121700 - rcvalle:rust-cfi-dont-compress-user-defined-builtin-types, r=compiler-errors 
CFI: Don't compress user-defined builtin types

Doesn't compress user-defined builtin types (see https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-builtin and https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-compression).
 2024-02-29 14:33:51 +01:00
Erik Desjardins
401651015d test merging of multiple match branches that access fields of the same offset 2024-02-27 23:14:36 -05:00
Erik Desjardins
c1017d4828 use non-inbounds GEP for ZSTs, add fixmes 2024-02-27 23:00:54 -05:00
Ramon de C Valle
8f7b921f52 CFI: Don't compress user-defined builtin types 
Doesn't compress user-defined builtin types (see
https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-builtin and
https://itanium-cxx-abi.github.io/cxx-abi/abi.html#mangling-compression).
 2024-02-27 12:23:48 -08:00
Erik Desjardins
4dabbcb23b allow using scalarpair with a common prim of ptr/ptr-sized-int 2024-02-27 00:09:12 -05:00
Erik Desjardins
123015e722 always use gep inbounds i8 (ptradd) for field offsets 2024-02-26 22:28:09 -05:00
bors
71ffdf7ff7 Auto merge of #121655 - matthiaskrgr:rollup-qpx3kks, r=matthiaskrgr 
Rollup of 4 pull requests

Successful merges:

 - #121598 (rename 'try' intrinsic to 'catch_unwind')
 - #121639 (Update books)
 - #121648 (Update Vec and String `{from,into}_raw_parts`-family docs)
 - #121651 (Properly emit `expected ;` on `#[attr] expr`)

r? `@ghost`
`@rustbot` modify labels: rollup
 2024-02-27 00:55:14 +00:00
Matthias Krüger
d95c321062
Rollup merge of #121598 - RalfJung:catch_unwind, r=oli-obk 
rename 'try' intrinsic to 'catch_unwind'

The intrinsic has nothing to do with `try` blocks, and corresponds to the stable `catch_unwind` function, so this makes a lot more sense IMO.

Also rename Miri's special function while we are at it, to reflect the level of abstraction it works on: it's an unwinding mechanism, on which Rust implements panics.
 2024-02-27 00:40:00 +01:00
bors
5c786a7fe3 Auto merge of #121516 - RalfJung:platform-intrinsics-begone, r=oli-obk 
remove platform-intrinsics ABI; make SIMD intrinsics be regular intrinsics

`@Amanieu` `@workingjubilee` I don't think there is any reason these need to be "special"? The [original RFC](https://rust-lang.github.io/rfcs/1199-simd-infrastructure.html) indicated eventually making them stable, but I think that is no longer the plan, so seems to me like we can clean this up a bit.

Blocked on https://github.com/rust-lang/stdarch/pull/1538, https://github.com/rust-lang/rust/pull/121542.
 2024-02-26 22:24:16 +00:00
Tim Neumann
05a6f65d81 Update a test to support Symbol Mangling V0 2024-02-26 18:12:07 +01:00
Ralf Jung
b4ca582b89 rename 'try' intrinsic to 'catch_unwind' 2024-02-26 11:10:18 +01:00
Guillaume Gomez
0e08be5360
Rollup merge of #120656 - Zalathar:filecheck-flags, r=wesleywiser 
Allow tests to specify a `//@ filecheck-flags:` header

This allows individual codegen/assembly/mir-opt tests to pass extra flags to the LLVM `filecheck` tool as needed.

---

The original motivation was noticing that `tests/run-make/instrument-coverage` was very close to being an ordinary codegen test, except that it needs some extra logic to set up platform-specific variables to be passed into filecheck.

I then saw the comment in `verify_with_filecheck` indicating that a `filecheck-flags` header might be useful for other purposes as well.
 2024-02-26 10:27:41 +01:00
Markus Reiter
b2fbb8a053
Use generic NonZero in tests. 2024-02-25 12:03:48 +01:00
Ralf Jung
c1d0e489e5 fix use of platform_intrinsics in tests 2024-02-25 08:15:44 +01:00
bors
89d8e3116c Auto merge of #120650 - clubby789:switchint-const, r=saethlin 
Use `br` instead of a conditional when switching on a constant boolean

r? `@ghost`
 2024-02-25 01:27:44 +00:00
Gary Guo
4677a71369 Add tests for asm goto 2024-02-24 19:49:16 +00:00
Ben Kimock
2f3c0b9859 Ignore less tests in debug builds 2024-02-23 18:04:01 -05:00
clubby789
7159aed51e Use br instead of conditional when branching on constant 2024-02-23 10:52:55 +00:00
Zalathar
e56cc8408d Remove unhelpful DEFINE_INTERNAL from filecheck flags 
This define was copied over from the run-make version of the test, but doesn't
seem to serve any useful purpose.
 2024-02-23 11:29:01 +11:00
Zalathar
0c19c632ab Convert tests/run-make/instrument-coverage to an ordinary codegen test 
This test was already very close to being an ordinary codegen test, except that
it needed some extra logic to set a few variables based on (target) platform
characteristics.

Now that we have support for `//@ filecheck-flags:`, we can instead set those
variables using the normal test revisions mechanism.
 2024-02-23 11:28:59 +11:00
Zalathar
c1889b549b Move existing coverage codegen tests into a subdirectory 
This makes room for migrating over `tests/run-make/instrument-coverage`,
without increasing the number of top-level items in the codegen test directory.
 2024-02-23 11:28:09 +11:00
Zalathar
baec3076db Allow tests to specify a //@ filecheck-flags: header 
Any flags specified here will be passed to LLVM's `filecheck` tool, in tests
that use that tool.
 2024-02-23 11:28:06 +11:00
Zalathar
36f298c93d Add some simple meta-tests for the handling of filecheck flags 2024-02-23 11:27:38 +11:00
许杰友 Jieyou Xu (Joe)
6e48b96692
[AUTO_GENERATED] Migrate compiletest to use ui_test-style //@ directives 2024-02-22 16:04:04 +00:00
bors
52dba5ffe7 Auto merge of #121225 - RalfJung:simd-extract-insert-const-idx, r=oli-obk,Amanieu 
require simd_insert, simd_extract indices to be constants

As discussed in https://github.com/rust-lang/rust/issues/77477 (see in particular [here](https://github.com/rust-lang/rust/issues/77477#issuecomment-703149102)). This PR doesn't touch codegen yet -- the first step is to ensure that the indices are always constants; the second step is to then make use of this fact in backends.

Blocked on https://github.com/rust-lang/stdarch/pull/1530 propagating to the rustc repo.
 2024-02-22 09:59:41 +00:00
Ralf Jung
07b6240947 remove simd_reduce_{min,max}_nanless 2024-02-21 20:50:47 +01:00
bors
bb8b11e67d Auto merge of #120718 - saethlin:reasonable-fast-math, r=nnethercote 
Add "algebraic" fast-math intrinsics, based on fast-math ops that cannot return poison

Setting all of LLVM's fast-math flags makes our fast-math intrinsics very dangerous, because some inputs are UB. This set of flags permits common algebraic transformations, but according to the [LangRef](https://llvm.org/docs/LangRef.html#fastmath), only the flags `nnan` (no nans) and `ninf` (no infs) can produce poison.

And this uses the algebraic float ops to fix https://github.com/rust-lang/rust/issues/120720

cc `@orlp`
 2024-02-21 09:43:33 +00:00
Ben Kimock
cc73b71e8e Add "algebraic" versions of the fast-math intrinsics 2024-02-20 12:39:03 -05:00
Ralf Jung
e19f89b5ff delete a test that no longer makes sense 2024-02-20 08:37:47 +01:00
CKingX
2d25c3b369
Updated test to account for added previous features (thanks erikdesjardins!) 2024-02-19 21:59:13 -08:00
bors
158f00a1c5 Auto merge of #118264 - lukas-code:optimized-draining, r=the8472 
Optimize `VecDeque::drain` for (half-)open ranges

The most common use cases of `VecDeque::drain` consume either the entire queue or elements from the front or back.[^1] This PR makes these operations faster by optimizing the generated code of the destructor of the drain:

* `.drain(..)` is now the same as `.clear()`.
* `.drain(n..)` is now (almost[^2]) the same as `.truncate(n)`.
* `.drain(..n)` is now an efficient "advance" function. This operation is not provided by a dedicated function and optimizing it is my main motivation for this PR.

Previously, all of these cases generated a function call to the destructor of the `DropGuard`, emitting a lot of unused machine code as well as unnecessary branches and loads/stores of stack variables.

There are no algorithmic changes in this PR, but it simplifies the code enough to allow LLVM to recognize the special cases and optimize accordingly. Most notably, it allows elimination of the rather large [`wrap_copy`] function.

Some [rudimentary microbenchmarks][benches] show a performance improvement of **~3x-4x** on my machine for the special cases and roughly equal performance for the general case.

Best reviewed commit by commit.

[^1]: source: GitHub code search: [full range `drain(..)` = 7.5k results][full], [from front `drain(..n)` = 3.2k results][front], [from back `drain(n..)` = 1.6k results][back], [from middle `drain(n..m)` = <500 results][middle]

[^2]: `.drain(0..)` and `.clear()` reset the head to 0, but `.truncate(0)` does not.

[full]: https://github.com/search?type=code&q=%2FVecDeque%28.%7C%5Cn%29%2B%5C.drain%5C%280%3F%5C.%5C.%5C%29%2F+lang%3ARust
[front]: https://github.com/search?type=code&q=%2FVecDeque%28.%7C%5Cn%29%2B%5C.drain%5C%280%3F%5C.%5C.%5B%5E%29%5D.*%5C%29%2F+lang%3ARust
[back]: https://github.com/search?type=code&q=%2FVecDeque%28.%7C%5Cn%29%2B%5C.drain%5C%28%5B%5E0%5D.*%5C.%5C.%5C%29%2F+lang%3ARust
[middle]: https://github.com/search?type=code&q=%2FVecDeque%28.%7C%5Cn%29%2B%5C.drain%5C%28%5B%5E0%5D.*%5C.%5C.%5B%5E%29%5D.*%5C%29%2F+lang%3ARust
[`wrap_copy`]: 4fd68eb47b/library/alloc/src/collections/vec_deque/mod.rs (L262-L391)
[benches]: https://gist.github.com/lukas-code/c97bd707d074c4cc31f241edbc7fd2a2

<details>
<summary>generated assembly</summary>

before:
```asm
clear:
	sub rsp, 40
	mov rax, qword ptr [rdi + 24]
	mov qword ptr [rdi + 24], 0
	mov qword ptr [rsp], rdi
	mov qword ptr [rsp + 8], rax
	xorps xmm0, xmm0
	movups xmmword ptr [rsp + 16], xmm0
	mov qword ptr [rsp + 32], rax
	test rax, rax
	je .LBB1_2
	mov rcx, qword ptr [rdi]
	mov rdx, qword ptr [rdi + 16]
	xor esi, esi
	cmp rdx, rcx
	cmovae rsi, rcx
	sub rdx, rsi
	mov rsi, rcx
	sub rsi, rdx
	lea rdi, [rdx + rax]
	cmp rsi, rax
	cmovb rdi, rcx
	sub rdi, rdx
	mov qword ptr [rsp + 16], rdi
	mov qword ptr [rsp + 32], 0
.LBB1_2:
	mov rdi, rsp
	call core::ptr::drop_in_place<<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<i32,alloc::alloc::Global>>
	add rsp, 40
	ret

truncate:
	mov rax, qword ptr [rdi + 24]
	sub rax, rsi
	jbe .LBB2_2
	sub rsp, 40
	mov qword ptr [rdi + 24], rsi
	mov qword ptr [rsp], rdi
	mov qword ptr [rsp + 8], rax
	mov rcx, qword ptr [rdi]
	mov rdx, qword ptr [rdi + 16]
	add rdx, rsi
	xor edi, edi
	cmp rdx, rcx
	cmovae rdi, rcx
	mov qword ptr [rsp + 24], 0
	sub rdx, rdi
	mov rdi, rcx
	sub rdi, rdx
	lea r8, [rdx + rax]
	cmp rdi, rax
	cmovb r8, rcx
	sub rsi, rdx
	add rsi, r8
	mov qword ptr [rsp + 16], rsi
	mov qword ptr [rsp + 32], 0
	mov rdi, rsp
	call core::ptr::drop_in_place<<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<i32,alloc::alloc::Global>>
	add rsp, 40

advance:
	mov rcx, qword ptr [rdi + 24]
	mov rax, rcx
	sub rax, rsi
	jbe .LBB3_1
	sub rsp, 40
	mov qword ptr [rdi + 24], 0
	mov qword ptr [rsp], rdi
	mov qword ptr [rsp + 8], rsi
	mov qword ptr [rsp + 16], 0
	mov qword ptr [rsp + 24], rax
	mov qword ptr [rsp + 32], rsi
	test rsi, rsi
	je .LBB3_6
	mov rax, qword ptr [rdi]
	mov rcx, qword ptr [rdi + 16]
	xor edx, edx
	cmp rcx, rax
	cmovae rdx, rax
	sub rcx, rdx
	mov rdx, rax
	sub rdx, rcx
	lea rdi, [rcx + rsi]
	cmp rdx, rsi
	cmovb rdi, rax
	sub rdi, rcx
	mov qword ptr [rsp + 16], rdi
	mov qword ptr [rsp + 32], 0
.LBB3_6:
	mov rdi, rsp
	call core::ptr::drop_in_place<<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<i32,alloc::alloc::Global>>
	add rsp, 40
	ret
.LBB3_1:
	test rcx, rcx
	je .LBB3_3
	mov qword ptr [rdi + 24], 0
.LBB3_3:
	mov qword ptr [rdi + 16], 0
	ret

remove:
	sub rsp, 40
	cmp rdx, rsi
	jb .LBB4_5
	mov rax, qword ptr [rdi + 24]
	mov rcx, rax
	sub rcx, rdx
	jb .LBB4_6
	mov qword ptr [rdi + 24], rsi
	mov qword ptr [rsp], rdi
	sub rdx, rsi
	mov qword ptr [rsp + 8], rdx
	mov qword ptr [rsp + 16], rsi
	mov qword ptr [rsp + 24], rcx
	mov qword ptr [rsp + 32], rdx
	je .LBB4_4
	mov rax, qword ptr [rdi]
	mov rcx, qword ptr [rdi + 16]
	add rcx, rsi
	xor edi, edi
	cmp rcx, rax
	cmovae rdi, rax
	sub rcx, rdi
	mov rdi, rax
	sub rdi, rcx
	lea r8, [rcx + rdx]
	cmp rdi, rdx
	cmovb r8, rax
	sub rsi, rcx
	add rsi, r8
	mov qword ptr [rsp + 16], rsi
	mov qword ptr [rsp + 32], 0
.LBB4_4:
	mov rdi, rsp
	call core::ptr::drop_in_place<<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<i32,alloc::alloc::Global>>
	add rsp, 40
	ret
.LBB4_5:
	lea rax, [rip + .L__unnamed_2]
	mov rdi, rsi
	mov rsi, rdx
	mov rdx, rax
	call qword ptr [rip + core::slice::index::slice_index_order_fail@GOTPCREL]
.LBB4_6:
	lea rcx, [rip + .L__unnamed_2]
	mov rdi, rdx
	mov rsi, rax
	mov rdx, rcx
	call qword ptr [rip + core::slice::index::slice_end_index_len_fail@GOTPCREL]

core::ptr::drop_in_place<<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<i32,alloc::alloc::Global>>:
	push rbp
	push r15
	push r14
	push r13
	push r12
	push rbx
	sub rsp, 24
	mov rsi, qword ptr [rdi + 32]
	test rsi, rsi
	je .LBB0_2
	mov rax, qword ptr [rdi + 16]
	add rsi, rax
	jb .LBB0_45
.LBB0_2:
	mov r13, qword ptr [rdi]
	mov rbp, qword ptr [rdi + 8]
	mov rbx, qword ptr [r13 + 24]
	lea r12, [rbx + rbp]
	mov r15, qword ptr [rdi + 24]
	lea rsi, [r15 + r12]
	test rbx, rbx
	je .LBB0_10
	test r15, r15
	je .LBB0_42
	cmp rbx, r15
	jbe .LBB0_12
	mov r14, qword ptr [r13]
	mov rax, qword ptr [r13 + 16]
	add r12, rax
	xor ecx, ecx
	cmp r12, r14
	mov rdx, r14
	cmovb rdx, rcx
	sub r12, rdx
	add rbx, rax
	cmp rbx, r14
	cmovae rcx, r14
	sub rbx, rcx
	mov rcx, rbx
	sub rcx, r12
	je .LBB0_42
	mov rdi, qword ptr [r13 + 8]
	mov rax, rcx
	add rax, r14
	cmovae rax, rcx
	mov r8, r14
	sub r8, r12
	mov rcx, r14
	sub rcx, rbx
	mov rdx, r15
	sub rdx, r8
	mov qword ptr [rsp + 16], rsi
	jbe .LBB0_18
	cmp rax, r15
	jae .LBB0_24
	mov rdx, r15
	sub rdx, r8
	shl rdx, 2
	cmp r15, rcx
	jbe .LBB0_30
	sub r8, rcx
	mov qword ptr [rsp], rdi
	mov rax, qword ptr [rsp]
	lea rdi, [rax + 4*r8]
	mov rsi, qword ptr [rsp]
	mov qword ptr [rsp + 8], rcx
	mov r15, r8
	call qword ptr [rip + memmove@GOTPCREL]
	sub r14, r15
	mov rax, qword ptr [rsp]
	lea rsi, [rax + 4*r14]
	shl r15, 2
	mov rdi, qword ptr [rsp]
	mov rdx, r15
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, qword ptr [rsp]
	lea rsi, [rdi + 4*r12]
	lea rdi, [rdi + 4*rbx]
	mov r15, qword ptr [rsp + 8]
	jmp .LBB0_36
.LBB0_10:
	test r15, r15
	je .LBB0_17
	mov rax, qword ptr [r13]
	sub rsi, rbp
	add rbp, qword ptr [r13 + 16]
	xor ecx, ecx
	cmp rbp, rax
	cmovae rcx, rax
	sub rbp, rcx
	mov qword ptr [r13 + 16], rbp
	jmp .LBB0_43
.LBB0_12:
	mov rdx, qword ptr [r13 + 16]
	mov r15, qword ptr [r13]
	lea rax, [rdx + rbp]
	xor ecx, ecx
	cmp rax, r15
	cmovae rcx, r15
	mov r12, rax
	sub r12, rcx
	mov rcx, r12
	sub rcx, rdx
	je .LBB0_41
	mov rdi, qword ptr [r13 + 8]
	mov rax, rcx
	add rax, r15
	cmovae rax, rcx
	mov r8, r15
	sub r8, rdx
	mov rcx, r15
	sub rcx, r12
	mov r14, rbx
	sub r14, r8
	mov qword ptr [rsp + 16], rsi
	jbe .LBB0_21
	cmp rax, rbx
	jae .LBB0_26
	mov qword ptr [rsp], rdx
	mov rdx, rbx
	sub rdx, r8
	shl rdx, 2
	cmp rbx, rcx
	jbe .LBB0_32
	sub r8, rcx
	mov rbx, rdi
	lea rdi, [rdi + 4*r8]
	mov rsi, rbx
	mov qword ptr [rsp + 8], rcx
	mov r14, r8
	call qword ptr [rip + memmove@GOTPCREL]
	sub r15, r14
	lea rsi, [rbx + 4*r15]
	shl r14, 2
	mov rdi, rbx
	mov rdx, r14
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, rbx
	mov rax, qword ptr [rsp]
	lea rsi, [rbx + 4*rax]
	lea rdi, [rbx + 4*r12]
	mov rbx, qword ptr [rsp + 8]
	jmp .LBB0_40
.LBB0_17:
	xorps xmm0, xmm0
	movups xmmword ptr [r13 + 16], xmm0
	jmp .LBB0_44
.LBB0_18:
	mov r14, r15
	sub r14, rcx
	jbe .LBB0_28
	cmp rax, r15
	jae .LBB0_33
	lea rax, [rcx + r12]
	sub r15, rcx
	lea rsi, [rdi + 4*rax]
	shl r15, 2
	mov r14, rdi
	mov rdx, r15
	mov r15, rcx
	jmp .LBB0_31
.LBB0_21:
	mov r14, rbx
	sub r14, rcx
	jbe .LBB0_29
	cmp rax, rbx
	jae .LBB0_34
	lea rax, [rcx + rdx]
	sub rbx, rcx
	lea rsi, [rdi + 4*rax]
	shl rbx, 2
	mov r14, rdi
	mov r15, rdx
	mov rdx, rbx
	mov rbx, rcx
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r14
	lea rsi, [r14 + 4*r15]
	lea rdi, [r14 + 4*r12]
	jmp .LBB0_40
.LBB0_24:
	sub r15, rcx
	jbe .LBB0_35
	sub rcx, r8
	mov qword ptr [rsp + 8], rcx
	lea rsi, [rdi + 4*r12]
	mov r12, rdi
	lea rdi, [rdi + 4*rbx]
	lea rdx, [4*r8]
	mov r14, r8
	call qword ptr [rip + memmove@GOTPCREL]
	add r14, rbx
	lea rdi, [r12 + 4*r14]
	mov rbx, qword ptr [rsp + 8]
	lea rdx, [4*rbx]
	mov rsi, r12
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r12
	lea rsi, [r12 + 4*rbx]
	jmp .LBB0_36
.LBB0_26:
	sub rbx, rcx
	jbe .LBB0_37
	sub rcx, r8
	lea rsi, [rdi + 4*rdx]
	mov r15, rdi
	lea rdi, [rdi + 4*r12]
	lea rdx, [4*r8]
	mov r14, rcx
	mov qword ptr [rsp], r8
	call qword ptr [rip + memmove@GOTPCREL]
	add r12, qword ptr [rsp]
	lea rdi, [r15 + 4*r12]
	lea rdx, [4*r14]
	mov rsi, r15
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r15
	lea rsi, [r15 + 4*r14]
	jmp .LBB0_40
.LBB0_28:
	lea rsi, [rdi + 4*r12]
	lea rdi, [rdi + 4*rbx]
	jmp .LBB0_36
.LBB0_29:
	lea rsi, [rdi + 4*rdx]
	lea rdi, [rdi + 4*r12]
	jmp .LBB0_40
.LBB0_30:
	lea rax, [r8 + rbx]
	mov r14, rdi
	lea rdi, [rdi + 4*rax]
	mov rsi, r14
	mov r15, r8
.LBB0_31:
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r14
	lea rsi, [r14 + 4*r12]
	lea rdi, [r14 + 4*rbx]
	jmp .LBB0_36
.LBB0_32:
	lea rax, [r12 + r8]
	mov rbx, rdi
	lea rdi, [rdi + 4*rax]
	mov rsi, rbx
	mov r14, r8
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, rbx
	mov rax, qword ptr [rsp]
	lea rsi, [rbx + 4*rax]
	jmp .LBB0_38
.LBB0_33:
	lea rsi, [rdi + 4*r12]
	mov r15, rdi
	lea rdi, [rdi + 4*rbx]
	lea rdx, [4*rcx]
	mov rbx, rcx
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r15
	add rbx, r12
	lea rsi, [r15 + 4*rbx]
	mov r15, r14
	jmp .LBB0_36
.LBB0_34:
	lea rsi, [rdi + 4*rdx]
	mov rbx, rdi
	lea rdi, [rdi + 4*r12]
	mov r15, rdx
	lea rdx, [4*rcx]
	mov r12, rcx
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, rbx
	add r12, r15
	lea rsi, [rbx + 4*r12]
	jmp .LBB0_39
.LBB0_35:
	lea rsi, [rdi + 4*r12]
	mov r14, rdi
	lea rdi, [rdi + 4*rbx]
	mov r12, rdx
	lea rdx, [4*r8]
	mov r15, r8
	call qword ptr [rip + memmove@GOTPCREL]
	add r15, rbx
	mov rsi, r14
	lea rdi, [r14 + 4*r15]
	mov r15, r12
.LBB0_36:
	shl r15, 2
	mov rdx, r15
	call qword ptr [rip + memmove@GOTPCREL]
	mov rsi, qword ptr [rsp + 16]
	jmp .LBB0_42
.LBB0_37:
	lea rsi, [rdi + 4*rdx]
	mov rbx, rdi
	lea rdi, [rdi + 4*r12]
	lea rdx, [4*r8]
	mov r15, r8
	call qword ptr [rip + memmove@GOTPCREL]
	add r12, r15
	mov rsi, rbx
.LBB0_38:
	lea rdi, [rbx + 4*r12]
.LBB0_39:
	mov rbx, r14
.LBB0_40:
	shl rbx, 2
	mov rdx, rbx
	call qword ptr [rip + memmove@GOTPCREL]
	mov r15, qword ptr [r13]
	mov rax, qword ptr [r13 + 16]
	add rax, rbp
	mov rsi, qword ptr [rsp + 16]
.LBB0_41:
	xor ecx, ecx
	cmp rax, r15
	cmovae rcx, r15
	sub rax, rcx
	mov qword ptr [r13 + 16], rax
.LBB0_42:
	sub rsi, rbp
.LBB0_43:
	mov qword ptr [r13 + 24], rsi
.LBB0_44:
	add rsp, 24
	pop rbx
	pop r12
	pop r13
	pop r14
	pop r15
	pop rbp
	ret
.LBB0_45:
	lea rdx, [rip + .L__unnamed_1]
	mov rdi, rax
	call qword ptr [rip + core::slice::index::slice_index_order_fail@GOTPCREL]
```

after:
```asm
clear:
	movups xmmword ptr [rdi + 16], xmm0
	ret

truncate:
	cmp qword ptr [rdi + 24], rsi
	jbe .LBB2_4
	test rsi, rsi
	jne .LBB2_3
	mov qword ptr [rdi + 16], 0
.LBB2_3:
	mov qword ptr [rdi + 24], rsi
.LBB2_4:
	ret

advance:
	mov rcx, qword ptr [rdi + 24]
	mov rax, rcx
	sub rax, rsi
	jbe .LBB3_1
	mov rcx, qword ptr [rdi]
	add rsi, qword ptr [rdi + 16]
	xor edx, edx
	cmp rsi, rcx
	cmovae rdx, rcx
	sub rsi, rdx
	mov qword ptr [rdi + 16], rsi
	mov qword ptr [rdi + 24], rax
	ret
.LBB3_1:
	test rcx, rcx
	je .LBB3_3
	mov qword ptr [rdi + 24], 0
.LBB3_3:
	mov qword ptr [rdi + 16], 0
	ret

remove:
	push rbp
	push r15
	push r14
	push r13
	push r12
	push rbx
	push rax
	mov r15, rsi
	mov r14, rdx
	sub r14, rsi
	jb .LBB4_9
	mov rbx, rdi
	mov r12, qword ptr [rdi + 24]
	mov r13, r12
	sub r13, rdx
	jb .LBB4_10
	mov qword ptr [rbx + 24], r15
	mov rbp, r12
	sub rbp, r14
	test r15, r15
	je .LBB4_4
	cmp rbp, r15
	jne .LBB4_11
.LBB4_4:
	cmp r12, r14
	jne .LBB4_6
.LBB4_5:
	mov qword ptr [rbx + 16], 0
	jmp .LBB4_8
.LBB4_11:
	mov rdi, rbx
	mov rsi, r14
	mov rdx, r15
	mov rcx, r13
	call <<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<T,A> as core::ops::drop::Drop>::drop::copy_data
	cmp r12, r14
	je .LBB4_5
.LBB4_6:
	cmp r13, r15
	jbe .LBB4_8
	mov rax, qword ptr [rbx]
	add r14, qword ptr [rbx + 16]
	xor ecx, ecx
	cmp r14, rax
	cmovae rcx, rax
	sub r14, rcx
	mov qword ptr [rbx + 16], r14
.LBB4_8:
	mov qword ptr [rbx + 24], rbp
	add rsp, 8
	pop rbx
	pop r12
	pop r13
	pop r14
	pop r15
	pop rbp
	ret
.LBB4_9:
	lea rax, [rip + .L__unnamed_1]
	mov rdi, r15
	mov rsi, rdx
	mov rdx, rax
	call qword ptr [rip + core::slice::index::slice_index_order_fail@GOTPCREL]
.LBB4_10:
	lea rax, [rip + .L__unnamed_1]
	mov rdi, rdx
	mov rsi, r12
	mov rdx, rax
	call qword ptr [rip + core::slice::index::slice_end_index_len_fail@GOTPCREL]

<<alloc::collections::vec_deque::drain::Drain<T,A> as core::ops::drop::Drop>::drop::DropGuard<T,A> as core::ops::drop::Drop>::drop::copy_data:
	push rbp
	push r15
	push r14
	push r13
	push r12
	push rbx
	push rax
	mov r14, rsi
	cmp rdx, rcx
	jae .LBB0_1
	mov r12, qword ptr [rdi]
	mov rax, qword ptr [rdi + 16]
	add r14, rax
	xor ecx, ecx
	cmp r14, r12
	cmovae rcx, r12
	sub r14, rcx
	mov r15, rdx
	mov r13, r14
	mov r14, rax
	mov rcx, r13
	sub rcx, r14
	je .LBB0_18
.LBB0_4:
	mov rdi, qword ptr [rdi + 8]
	mov rax, rcx
	add rax, r12
	cmovae rax, rcx
	mov rbx, r12
	sub rbx, r14
	mov rcx, r12
	sub rcx, r13
	mov rbp, r15
	sub rbp, rbx
	jbe .LBB0_5
	cmp rax, r15
	jae .LBB0_12
	mov rdx, r15
	sub rdx, rbx
	shl rdx, 2
	cmp r15, rcx
	jbe .LBB0_16
	sub rbx, rcx
	mov rbp, rdi
	lea rdi, [rdi + 4*rbx]
	mov r15, qword ptr [rip + memmove@GOTPCREL]
	mov rsi, rbp
	mov qword ptr [rsp], rcx
	call r15
	sub r12, rbx
	lea rsi, [4*r12]
	add rsi, rbp
	shl rbx, 2
	mov rdi, rbp
	mov rdx, rbx
	call r15
	mov rdi, rbp
	lea rsi, [4*r14]
	add rsi, rbp
	lea rdi, [4*r13]
	add rdi, rbp
	mov r15, qword ptr [rsp]
	jmp .LBB0_7
.LBB0_1:
	mov r15, rcx
	add r14, rdx
	mov r12, qword ptr [rdi]
	mov r13, qword ptr [rdi + 16]
	add r14, r13
	xor eax, eax
	cmp r14, r12
	mov rcx, r12
	cmovb rcx, rax
	sub r14, rcx
	add r13, rdx
	cmp r13, r12
	cmovae rax, r12
	sub r13, rax
	mov rcx, r13
	sub rcx, r14
	jne .LBB0_4
.LBB0_18:
	add rsp, 8
	pop rbx
	pop r12
	pop r13
	pop r14
	pop r15
	pop rbp
	ret
.LBB0_5:
	mov rbx, r15
	sub rbx, rcx
	jbe .LBB0_6
	cmp rax, r15
	jae .LBB0_9
	lea rax, [rcx + r14]
	sub r15, rcx
	lea rsi, [rdi + 4*rax]
	shl r15, 2
	mov rbx, rdi
	mov rdx, r15
	mov r15, rcx
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, rbx
	lea rsi, [rbx + 4*r14]
	lea rdi, [rbx + 4*r13]
	jmp .LBB0_7
.LBB0_12:
	sub r15, rcx
	jbe .LBB0_13
	sub rcx, rbx
	lea rsi, [rdi + 4*r14]
	mov r12, rdi
	lea rdi, [rdi + 4*r13]
	lea rdx, [4*rbx]
	mov r14, qword ptr [rip + memmove@GOTPCREL]
	mov rbp, rcx
	call r14
	add rbx, r13
	lea rdi, [r12 + 4*rbx]
	lea rdx, [4*rbp]
	mov rsi, r12
	call r14
	mov rdi, r12
	lea rsi, [r12 + 4*rbp]
	jmp .LBB0_7
.LBB0_6:
	lea rsi, [rdi + 4*r14]
	lea rdi, [rdi + 4*r13]
	jmp .LBB0_7
.LBB0_16:
	lea rax, [rbx + r13]
	mov r15, rdi
	lea rdi, [rdi + 4*rax]
	mov rsi, r15
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r15
	lea rsi, [r15 + 4*r14]
	lea rdi, [r15 + 4*r13]
	mov r15, rbx
	jmp .LBB0_7
.LBB0_9:
	lea rsi, [rdi + 4*r14]
	mov r15, rdi
	lea rdi, [rdi + 4*r13]
	lea rdx, [4*rcx]
	mov r12, rcx
	call qword ptr [rip + memmove@GOTPCREL]
	mov rdi, r15
	add r12, r14
	lea rsi, [r15 + 4*r12]
	mov r15, rbx
	jmp .LBB0_7
.LBB0_13:
	lea rsi, [rdi + 4*r14]
	mov r14, rdi
	lea rdi, [rdi + 4*r13]
	lea rdx, [4*rbx]
	call qword ptr [rip + memmove@GOTPCREL]
	add rbx, r13
	mov rsi, r14
	lea rdi, [r14 + 4*rbx]
	mov r15, rbp
.LBB0_7:
	shl r15, 2
	mov rdx, r15
	add rsp, 8
	pop rbx
	pop r12
	pop r13
	pop r14
	pop r15
	pop rbp
	jmp qword ptr [rip + memmove@GOTPCREL]
```

</details>
 2024-02-18 00:03:39 +00:00
Ben Kimock
7c2db703b0 Don't use mem::zeroed in vec::IntoIter 2024-02-16 10:44:39 -05:00
Lukas Markeffsky
8f259ade66 add codegen test 2024-02-16 13:11:05 +01:00
bors
dfa88b328f Auto merge of #120500 - oli-obk:intrinsics2.0, r=WaffleLapkin 
Implement intrinsics with fallback bodies

fixes #93145 (though we can port many more intrinsics)
cc #63585

The way this works is that the backend logic for generating custom code for intrinsics has been made fallible. The only failure path is "this intrinsic is unknown". The `Instance` (that was `InstanceDef::Intrinsic`) then gets converted to `InstanceDef::Item`, which represents the fallback body. A regular function call to that body is then codegenned. This is currently implemented for

* codegen_ssa (so llvm and gcc)
* codegen_cranelift

other backends will need to adjust, but they can just keep doing what they were doing if they prefer (though adding new intrinsics to the compiler will then require them to implement them, instead of getting the fallback body).

cc `@scottmcm` `@WaffleLapkin`

### todo

* [ ] miri support
* [x] default intrinsic name to name of function instead of requiring it to be specified in attribute
* [x] make sure that the bodies are always available (must be collected for metadata)
 2024-02-16 09:53:01 +00:00
Augie Fackler
a6ee72df91 tests: LLVM 18 infers an extra noalias here 
This test started failing on LLVM 18 after change
61118ffd04. As far as I can tell, it's
just good fortune that LLVM is able to sniff out the new noalias here,
and it's correct.
 2024-02-13 10:33:40 +01:00
Oli Scherer
f35a2bd401 Support safe intrinsics with fallback bodies 
Turn `is_val_statically_known` into such an intrinsic to demonstrate. It is perfectly safe to call after all.
 2024-02-12 17:55:36 +00:00
Matthias Krüger
1843dfd0d5
Rollup merge of #118307 - scottmcm:tuple-eq-simpler, r=joshtriplett 
Remove an unneeded helper from the tuple library code

Thanks to https://github.com/rust-lang/rust/pull/107022, this is just what `==` does, so we don't need the helper here anymore.
 2024-02-11 08:25:41 +01:00
Michael Goulet
34ed554d81 Build DebugInfo for coroutine-closure 2024-02-09 16:01:29 +00:00
Guillaume Boisseau
7954c28cf9
Rollup merge of #119162 - heiher:direct-access-external-data, r=petrochenkov 
Add unstable `-Z direct-access-external-data` cmdline flag for `rustc`

The new flag has been described in the Major Change Proposal at https://github.com/rust-lang/compiler-team/issues/707

Fixes #118053
 2024-02-07 18:24:41 +01:00
Matthias Krüger
59ba8024af
Rollup merge of #120502 - clubby789:remove-ffi-returns-twice, r=compiler-errors 
Remove `ffi_returns_twice` feature

The [tracking issue](https://github.com/rust-lang/rust/issues/58314) and [RFC](https://github.com/rust-lang/rfcs/pull/2633) have been closed for a couple of years.

There is also an attribute gate in R-A which should be removed if this lands.
 2024-02-06 22:45:42 +01:00
bors
268dbbbc4b Auto merge of #120624 - matthiaskrgr:rollup-3gvcl20, r=matthiaskrgr 
Rollup of 8 pull requests

Successful merges:

 - #120484 (Avoid ICE when is_val_statically_known is not of a supported type)
 - #120516 (pattern_analysis: cleanup manual impls)
 - #120517 (never patterns: It is correct to lower `!` to `_`.)
 - #120523 (Improve `io::Read::read_buf_exact` error case)
 - #120528 (Store SHOULD_CAPTURE as AtomicU8)
 - #120529 (Update data layouts in custom target tests for LLVM 18)
 - #120531 (Remove a bunch of `has_errors` checks that have no meaningful or the wrong effect)
 - #120533 (Correct paths for hexagon-unknown-none-elf platform doc)

r? `@ghost`
`@rustbot` modify labels: rollup
 2024-02-04 20:51:28 +00:00
Matthias Krüger
6f24836a5b
Rollup merge of #120484 - Teapot4195:issue-120480-fix, r=compiler-errors 
Avoid ICE when is_val_statically_known is not of a supported type

2 ICE with 1 stone!
1. Implement `llvm.is.constant.ptr` to avoid first ICE in linked issue.
2. return `false` when the argument is not one of `i*`/`f*`/`ptr` to avoid second ICE.

fixes #120480
 2024-02-03 22:25:14 +01:00
Oli Scherer
6ac035df44 Revert unsound libcore changes of #119911 2024-02-01 22:53:25 +00:00
clubby789
7331315898 Remove ffi_returns_twice feature 2024-01-30 22:09:09 +00:00
Alex Huang
a97ff2a750 Add additional test cases for is_val_statically_known 2024-01-30 14:37:59 -05:00
Guillaume Gomez
6a1d34f32a
Rollup merge of #120310 - krasimirgg:jan-v0-sym, r=Mark-Simulacrum 
adapt test for v0 symbol mangling

No functional changes intended.

Adapts the test to also work under `new-symbol-mangling = true`.
 2024-01-30 16:57:48 +01:00
Nikita Popov
bdf7404b43 Update codegen test for LLVM 18 2024-01-26 15:03:23 +01:00
bors
039d887928 Auto merge of #119911 - NCGThompson:is-statically-known, r=oli-obk 
Replacement of #114390: Add new intrinsic `is_var_statically_known` and optimize pow for powers of two

This adds a new intrinsic `is_val_statically_known` that lowers to [``@llvm.is.constant.*`](https://llvm.org/docs/LangRef.html#llvm-is-constant-intrinsic).` It also applies the intrinsic in the int_pow methods to recognize and optimize the idiom `2isize.pow(x)`. See #114390 for more discussion.

While I have extended the scope of the power of two optimization from #114390, I haven't added any new uses for the intrinsic. That can be done in later pull requests.

Note: When testing or using the library, be sure to use `--stage 1` or higher. Otherwise, the intrinsic will be a noop and the doctests will be skipped. If you are trying out edits, you may be interested in [`--keep-stage 0`](https://rustc-dev-guide.rust-lang.org/building/suggested.html#faster-builds-with---keep-stage).

Fixes #47234
Resolves #114390
`@Centri3`
 2024-01-25 05:16:53 +00:00
Krasimir Georgiev
e23937c6d3 adapt test for v0 symbol mangling 
No functional changes intended.

Adapts the test to also work under new-symbol-mangling = true.
 2024-01-24 14:57:21 +00:00
Nicholas Thompson
9dccd5dce1 Further Implement Power of Two Optimization 2024-01-23 12:03:50 -05:00
Nicholas Thompson
971e37ff7e Further Implement is_val_statically_known 2024-01-23 12:02:31 -05:00
Nikita Popov
31f5f033e9 Remove uses of no-system-llvm 
It looks like none of these are actually needed.
 2024-01-23 10:31:07 +01:00
Nikita Popov
823e8b041a Allow disjoint flag in codegen test 2024-01-23 10:12:36 +01:00
bors
e35a56d96f Auto merge of #119892 - joboet:libs_use_assert_unchecked, r=Nilstrieb,cuviper 
Use `assert_unchecked` instead of `assume` intrinsic in the standard library

Now that a public wrapper for the `assume` intrinsic exists, we can use it in the standard library.

CC #119131
 2024-01-23 06:45:58 +00:00
joboet
638439a440
update codegen tests 2024-01-22 15:46:32 +01:00
AngelicosPhosphoros
60208a0517 Tweak the threshold for chunked swapping 
Thanks to 98892 for the tests I brought in here, as it demonstrated that 3×usize is currently suboptimal.
 2024-01-19 23:00:34 -08:00
Catherine Flores
5a4561749a Add new intrinsic is_constant and optimize pow 
Fix overflow check

Make MIRI choose the path randomly and rename the intrinsic

Add back test

Add miri test and make it operate on `ptr`

Define `llvm.is.constant` for primitives

Update MIRI comment and fix test in stage2

Add const eval test

Clarify that both branches must have the same side effects

guaranteed non guarantee

use immediate type instead

Co-Authored-By: Ralf Jung <post@ralfj.de>
 2024-01-19 13:46:27 -05:00
Nikita Popov
ce2d91dccd Directly use volatile_load intrinsic 
This makes the test work if libstd is compiled with debug assertions.
 2024-01-19 10:52:01 +01:00
Nikita Popov
7a0415ce37 Add codegen test for ScalarPair with i128 on LLVM 17 2024-01-19 10:52:01 +01:00
bors
bf2637f4e8 Auto merge of #119954 - scottmcm:option-unwrap-failed, r=WaffleLapkin 
Split out `option::unwrap_failed` like we have `result::unwrap_failed`

...and like `option::expect_failed`
 2024-01-16 15:32:39 +00:00
WANG Rui
06a41687b1 Add unstable -Z direct-access-external-data cmdline flag for rustc 
The new flag has been described in the Major Change Proposal at
https://github.com/rust-lang/compiler-team/issues/707
 2024-01-16 19:15:06 +08:00
bors
1ead4761e9 Auto merge of #119878 - scottmcm:inline-always-unwrap, r=workingjubilee 
Tune the inlinability of `unwrap`

Fixes #115463
cc `@thomcc`

This tweaks `unwrap` on ~~`Option` &~~ `Result` to be two parts:
- `#[inline(always)]` for checking the discriminant
- `#[cold]` for actually panicking

The idea here is that checking the discriminant on a `Result` ~~or `Option`~~ should always be trivial enough to be worth inlining, even in `opt-level=z`, especially compared to passing it to a function.

As seen in the issue and codegen test, this will hopefully help particularly for things like `.try_into().unwrap()`s that are actually infallible, but in a way that's only visible with the inlining.

EDIT: I've restricted this to `Result` to avoid combining effects
 2024-01-15 09:20:46 +00:00
Scott McMurray
23483664a2 Split out option::unwrap_failed like we have result::unwrap_failed 
...and like `option::expect_failed`
 2024-01-14 12:45:01 -08:00
bors
2319be8e26 Auto merge of #119452 - AngelicosPhosphoros:make_nonzeroint_get_assume_nonzero, r=scottmcm 
Add assume into `NonZeroIntX::get`

LLVM currently don't support range metadata for function arguments so it fails to optimize non zero integers using their invariant if they are provided using by-value function arguments.

Related to https://github.com/rust-lang/rust/issues/119422
Related to https://github.com/llvm/llvm-project/issues/76628
Related to https://github.com/rust-lang/rust/issues/49572
 2024-01-12 20:18:04 +00:00
Scott McMurray
b858c591dd Tune the inlinability of Result::unwrap 2024-01-12 10:57:58 -08:00
The 8472
93b34a5ffa mark vec::IntoIter pointers as !nonnull 2024-01-07 03:44:04 +01:00
AngelicosPhosphoros
8f432d4ae6 Add assume into NonZeroIntX::get 
LLVM currently don't support range metadata for function arguments so it fails to optimize non zero integers using their invariant if they are provided using by-value function arguments.

Related to https://github.com/rust-lang/rust/issues/119422
Related to https://github.com/llvm/llvm-project/issues/76628
Related to https://github.com/rust-lang/rust/issues/49572
 2024-01-06 14:26:37 +01:00
bors
432fffa8af Auto merge of #118991 - nikic:scalar-pair, r=nagisa 
Separate immediate and in-memory ScalarPair representation

Currently, we assume that ScalarPair is always represented using a two-element struct, both as an immediate value and when stored in memory.

This currently works fairly well, but runs into problems with https://github.com/rust-lang/rust/pull/116672, where a ScalarPair involving an i128 type can no longer be represented as a two-element struct in memory. For example, the tuple `(i32, i128)` needs to be represented in-memory as `{ i32, [3 x i32], i128 }` to satisfy alignment requirements. Using `{ i32, i128 }` instead will result in the second element being stored at the wrong offset (prior to LLVM 18).

Resolve this issue by no longer requiring that the immediate and in-memory type for ScalarPair are the same. The in-memory type will now look the same as for normal struct types (and will include padding filler and similar), while the immediate type stays a simple two-element struct type. This also means that booleans in immediate ScalarPair are now represented as i1 rather than i8, just like we do everywhere else.

The core change here is to llvm_type (which now treats ScalarPair as a normal struct) and immediate_llvm_type (which returns the two-element struct that llvm_type used to produce). The rest is fixing things up to no longer assume these are the same. In particular, this switches places that try to get pointers to the ScalarPair elements to use byte-geps instead of struct-geps.
 2024-01-05 14:31:56 +00:00
Nikita Popov
3cd6cde0be Make test compatible with 32-bit as well 2024-01-05 11:45:57 +01:00
Matthias Krüger
c505d760a6
Rollup merge of #119555 - Kobzol:maybeuninit-rvo-codegen-test, r=nikic 
Add codegen test for RVO on MaybeUninit

Codegen test for https://github.com/rust-lang/rust/issues/90595. Currently, this only works with `-Cpanic=abort`, but hopefully in the [future](https://www.npopov.com/2024/01/01/This-year-in-LLVM-2023.html#writable-and-dead_on_unwind) it should also work in the presence of panics.

r? ``@nikic``
 2024-01-04 08:33:26 +01:00
Jakub Beránek
0c56ccff04
Add codegen test for RVO on MaybeUninit 
Currently, this only works with `-Cpanic=abort`.
 2024-01-03 21:18:07 +01:00
León Orell Valerian Liehr
fcec407f4a
Rollup merge of #119523 - maurer:fix-sparc-llvm-18, r=nikic 
llvm: Allow `noundef` in codegen tests

LLVM 18 will automatically infer `noundef` in some situations. Adjust codegen tests to accept this.

See llvm/llvm-project#76553 for why `noundef` is being generated now.

``@rustbot`` label:+llvm-main
 2024-01-03 16:08:32 +01:00
Matthew Maurer
ee86b1f84c llvm: Allow noundef in codegen tests 
LLVM 18 will automatically infer `noundef` in some situations.
Adjust codegen tests to accept this.

See llvm/llvm-project#76553 for why `noundef` is being generated now.
 2024-01-02 18:02:17 +00:00
Nikita Popov
8e64fc94d8 Address review comments 2024-01-02 15:03:14 +01:00
Camille GILLOT
6dfda0d32f Revert codegen test change. 2023-12-24 20:08:58 +00:00
Camille GILLOT
2837727471 Replace legacy ConstProp by GVN. 2023-12-24 20:08:57 +00:00
Camille GILLOT
a03c972816 Enable GVN by default. 2023-12-24 20:08:57 +00:00
Augie Fackler
58fdbd1479 tests: fix overaligned-constant to not over-specify getelementptr instr 
On LLVM 18 we get slightly different arguments here, so it's easier to
just regex those away. The important details are all still asserted as I
understand things.

Fixes #119193.

@rustbot label: +llvm-main
 2023-12-21 15:53:28 -05:00
bors
920e0051cf Auto merge of #119056 - cjgillot:codegen-overalign, r=wesleywiser 
Tolerate overaligned MIR constants for codegen.

Fixes https://github.com/rust-lang/rust/issues/117761

cc `@saethlin`
 2023-12-21 04:01:36 +00:00
bors
51c0db6a91 Auto merge of #106790 - the8472:rawvec-niche, r=scottmcm 
add more niches to rawvec

Previously RawVec only had a single niche in its `NonNull` pointer. With this change it now has `isize::MAX` niches since half the value-space of the capacity field is never needed, we can't have a capacity larger than isize::MAX.
 2023-12-20 02:19:10 +00:00
Camille GILLOT
503af0deb2 Fortify test. 2023-12-17 23:31:58 +00:00
Camille GILLOT
3ea5cfaa11 Tolerate overaligned MIR constants for codegen. 2023-12-17 22:56:42 +00:00
Nikita Popov
c2fd26a115 Separate immediate and in-memory ScalarPair representation 
Currently, we assume that ScalarPair is always represented using
a two-element struct, both as an immediate value and when stored
in memory.

This currently works fairly well, but runs into problems with
https://github.com/rust-lang/rust/pull/116672, where a ScalarPair
involving an i128 type can no longer be represented as a two-element
struct in memory. For example, the tuple `(i32, i128)` needs to be
represented in-memory as `{ i32, [3 x i32], i128 }` to satisfy
alignment requirement. Using `{ i32, i128 }` instead will result in
the second element being stored at the wrong offset (prior to
LLVM 18).

Resolve this issue by no longer requiring that the immediate and
in-memory type for ScalarPair are the same. The in-memory type
will now look the same as for normal struct types (and will include
padding filler and similar), while the immediate type stays a
simple two-element struct type. This also means that booleans in
immediate ScalarPair are now represented as i1 rather than i8,
just like we do everywhere else.

The core change here is to llvm_type (which now treats ScalarPair
as a normal struct) and immediate_llvm_type (which returns the
two-element struct that llvm_type used to produce). The rest is
fixing things up to no longer assume these are the same. In
particular, this switches places that try to get pointers to the
ScalarPair elements to use byte-geps instead of struct-geps.
 2023-12-15 17:42:05 +01:00
Wesley Wiser
ce290514df
Adapt debug-accessibility tests for msvc-style enums 2023-12-15 11:45:03 +00:00
David Wood
07931c5a08
codegen_llvm: set DW_AT_accessibility 
Sets the accessibility of types and fields in DWARF using
`DW_AT_accessibility` attribute.

`DW_AT_accessibility` (public/protected/private) isn't exactly right for
Rust,  but neither is `DW_AT_visibility` (local/exported/qualified), and
there's no way to set `DW_AT_visbility` in LLVM's API.

Signed-off-by: David Wood <david@davidtw.co>
 2023-12-15 11:36:41 +00:00
bors
9d49eb76c4 Auto merge of #118417 - anforowicz:default-hidden-visibility, r=TaKO8Ki 
Add unstable `-Zdefault-hidden-visibility` cmdline flag for `rustc`.

The new flag has been described in the Major Change Proposal at
https://github.com/rust-lang/compiler-team/issues/656
 2023-12-14 09:16:15 +00:00
bors
e6d1b0ec98 Auto merge of #118491 - cuviper:aarch64-stack-probes, r=wesleywiser 
Enable stack probes on aarch64 for LLVM 18

I tested this on `aarch64-unknown-linux-gnu` with LLVM main (~18).

cc #77071, to be closed once we upgrade our LLVM submodule.
 2023-12-14 02:01:13 +00:00
Lukasz Anforowicz
981c4e3ce6 Add unstable -Zdefault-hidden-visibility cmdline flag for rustc. 
The new flag has been described in the Major Change Proposal at
https://github.com/rust-lang/compiler-team/issues/656
 2023-12-13 21:14:23 +00:00
Jakub Okoński
95b5a80f47
Fix alignment passed down to LLVM for simd_masked_load 2023-12-12 13:11:59 +01:00
The 8472
502df1b7d4 add more niches to rawvec 2023-12-11 23:38:48 +01:00
Jakub Okoński
97ae5095f5
Add simd_masked_{load,store} platform-intrinsics 
This maps to the LLVM intrinsics: llvm.masked.load and llvm.masked.store
 2023-12-09 12:36:08 +01:00
Josh Stone
b99b5e5752 Enable stack probes on aarch64 for LLVM 18 2023-12-07 17:17:00 -08:00
Ramon de C Valle
97032d63bd CFI: Add char to CFI integer normalization 
Adds char to CFI integer normalization to conform to #118032 for
cross-language CFI support.
 2023-12-07 11:28:16 -08:00
bendn
73afc00cf9
use assume(idx < self.len()) in [T]::get_unchecked 2023-12-04 06:00:12 +07:00
bors
3f1e30a0a5 Auto merge of #118077 - calebzulawski:sync-portable-simd-2023-11-19, r=workingjubilee 
Portable SIMD subtree update

Syncs nightly to the latest changes from rust-lang/portable-simd

r? `@rust-lang/libs`
 2023-12-02 18:04:01 +00:00
bors
f45631b10f Auto merge of #116892 - ojeda:rethunk, r=wesleywiser 
Add `-Zfunction-return={keep,thunk-extern}` option

This is intended to be used for Linux kernel RETHUNK builds.

With this commit (optionally backported to Rust 1.73.0), plus a patched Linux kernel to pass the flag, I get a RETHUNK build with Rust enabled that is `objtool`-warning-free and is able to boot in QEMU and load a sample Rust kernel module.

Issue: https://github.com/rust-lang/rust/issues/116853.
 2023-11-30 22:10:30 +00:00
Miguel Ojeda
2d476222e8 Add -Zfunction-return={keep,thunk-extern} option 
This is intended to be used for Linux kernel RETHUNK builds.

With this commit (optionally backported to Rust 1.73.0), plus a
patched Linux kernel to pass the flag, I get a RETHUNK build with
Rust enabled that is `objtool`-warning-free and is able to boot in
QEMU and load a sample Rust kernel module.

Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
 2023-11-30 20:21:31 +01:00
bors
07921b50ba Auto merge of #118036 - DianQK:thinlto-tests, r=tmiasko 
Add thinlto support to codegen, assembly and coverage tests

Using `--emit=llvm-ir` with thinlto usually result in multiple IR files.
Resolve test case failure issue reported in #113923.
 2023-11-30 13:33:32 +00:00
DianQK
c41bf96039
Add thinlto support to codegen, assembly and coverage tests 2023-11-30 18:48:03 +08:00
Krasimir Georgiev
81cd7c5b11 update test for new LLVM 18 codegen 
LLVM at HEAD now emits `or disjoint`: https://buildkite.com/llvm-project/rust-llvm-integrate-prototype/builds/24076#018c1596-8153-488e-b622-951266a02f6c/741-774
 2023-11-28 12:10:59 +00:00
bors
49b3924bd4 Auto merge of #117947 - Dirbaio:drop-llvm-15, r=cuviper 
Update the minimum external LLVM to 16.

With this change, we'll have stable support for LLVM 16 and 17.
For reference, the previous increase to LLVM 15 was #114148

[Relevant zulip discussion](https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/riscv.20forced-atomics)
 2023-11-27 21:54:03 +00:00
Caleb Zulawski
4d9607869a Update std::simd usage and test outputs 2023-11-26 09:02:25 -05:00
Scott McMurray
4b3f11523d Remove an unneeded helper from the tuple library code 2023-11-25 22:25:00 -08:00
Arlie Davis
9429d68842 convert ehcont-guard to an unstable option 2023-11-21 14:24:23 -08:00
Arlie Davis
e11d8d147b Add support for generating the EHCont section 
In the future Windows will enable Control-flow Enforcement Technology
(CET aka Shadow Stacks). To protect the path where the context is
updated during exception handling, the binary is required to enumerate
valid unwind entrypoints in a dedicated section which is validated when
the context is being set during exception handling.

The required support for EHCONT has already been merged into LLVM,
long ago. This change adds the Rust codegen option to enable it.

Reference:

* https://reviews.llvm.org/D40223

This also adds a new `ehcont-guard` option to the bootstrap config which
enables EHCont Guard when building std.
 2023-11-21 13:41:23 -08:00
Dario Nieuwenhuis
7de6d04bc8 Update the minimum external LLVM to 16. 2023-11-21 22:40:16 +01:00
bors
0b24479638 Auto merge of #116555 - paulmenage:llvm-module-flag, r=wesleywiser 
Add -Z llvm_module_flag

Allow adding values to the `!llvm.module.flags` metadata for a generated module.  The syntax is

`-Z llvm_module_flag=<name>:<type>:<value>:<behavior>`

Currently only u32 values are supported but the type is required to be specified for forward compatibility.  The `behavior` element must match one of the named LLVM metadata behaviors.viors.

This flag is expected to be perma-unstable.
 2023-11-15 16:54:31 +00:00
Augie Fackler
5d8d700fd3 tests: update check for inferred nneg on zext 
This was broken by upstream
llvm/llvm-project@dc6d077396. It's easy
enough to use a regex match to support both, so we do that.

r? @nikic
@rustbot label: +llvm-main
 2023-11-13 10:43:33 -05:00
Paul Menage
2e6b57541d Add -Z llvm_module_flag 
Allow adding values to the `!llvm.module.flags` metadata for a generated
module.  The syntax is

`-Z llvm_module_flag=<name>:<type>:<value>:<behavior>`

Currently only u32 values are supported but the type is required to be
specified for forward compatibility.  The `behavior` element must match
one of the named LLVM metadata behaviors.viors.

This flag is expected to be perma-unstable.
 2023-11-11 19:48:47 -08:00
Ben Kimock
d32d9238cf Emit #[inline] on derive(Debug) 2023-11-09 10:40:55 -05:00
Ben Kimock
fcdd99edca Add -Zcross-crate-inline-threshold=yes 2023-11-07 18:45:11 -05:00
bors
f5ca57e153 Auto merge of #117503 - kornelski:hint-try-reserved, r=workingjubilee 
Hint optimizer about try-reserved capacity

This is #116568, but limited only to the less-common `try_reserve` functions to reduce bloat in debug binaries from debug info, while still addressing the main use-case #116570
 2023-11-05 00:03:41 +00:00
Kornel
029fbd67ef Hint optimizer about reserved capacity 2023-11-02 00:52:06 +00:00
Matthias Krüger
260e07b0cb
Rollup merge of #115626 - clarfonthey:unchecked-math, r=thomcc 
Clean up unchecked_math, separate out unchecked_shifts

Tracking issue: #85122

Changes:

1. Remove `const_inherent_unchecked_arith` flag and make const-stability flags the same as the method feature flags. Given the number of other unsafe const fns already stabilised, it makes sense to just stabilise these in const context when they're stabilised.
2. Move `unchecked_shl` and `unchecked_shr` into a separate `unchecked_shifts` flag, since the semantics for them are unclear and they'll likely be stabilised separately as a result.
3. Add an `unchecked_neg` method exclusively to signed integers, under the `unchecked_neg` flag. This is because it's a new API and probably needs some time to marinate before it's stabilised, and while it *would* make sense to have a similar version for unsigned integers since `checked_neg` also exists for those there is absolutely no case where that would be a good idea, IMQHO.

The longer-term goal here is to prepare the `unchecked_math` methods for an FCP and stabilisation since they've existed for a while, their semantics are clear, and people seem in favour of stabilising them.
 2023-11-01 11:29:41 +01:00
okaneco
465ffc9ca7 Refactor some char, u8 ascii functions to be branchless 
Decompose singular `matches!` with or-patterns to individual `matches!`
statements to enable branchless code output. The following functions
were changed:
- `is_ascii_alphanumeric`
- `is_ascii_hexdigit`
- `is_ascii_punctuation`

Add codegen tests

Co-authored-by: George Bateman <george.bateman16@gmail.com>
Co-authored-by: scottmcm <scottmcm@users.noreply.github.com>
 2023-10-26 21:48:36 -04:00
Zalathar
f83f7966f5 coverage: Add UI tests for values accepted by -Cinstrument-coverage 2023-10-23 17:41:40 +11:00
Oli Scherer
af93c20c06 Rename lots of files that had generator in their name 2023-10-20 21:14:02 +00:00
Oli Scherer
e96ce20b34 s/generator/coroutine/ 2023-10-20 21:14:01 +00:00
Oli Scherer
60956837cf s/Generator/Coroutine/ 2023-10-20 21:10:38 +00:00
Ben Kimock
33b0e4be06 Automatically enable cross-crate inlining for small functions 2023-10-17 19:53:51 -04:00
Arthur Carcano
0bcac8a7f2 Add invariant to Vec::pop that len < cap if pop successful 
Fixes: https://github.com/rust-lang/rust/issues/114334
 2023-10-16 18:49:25 +02:00
Matthias Krüger
a8cda30f32
Rollup merge of #116591 - Zalathar:flaky-hash, r=Mark-Simulacrum 
Don't accidentally detect the commit hash as an `fadd` instruction

I've seen some reports of `tests/codegen/target-feature-inline-closure.rs` spuriously failing because it thinks the hash in the rustc version number contains an `fadd` instruction.

https://github.com/rust-lang/rust/pull/116085#issuecomment-1751174916
https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Is.20.60tests.2Fcodegen.2Ftarget-feature-inline-closure.2Ers.60.20flakey
https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/Strange.20.5Cn.20in.20output.20of.20assert.20.23108341/near/395811335

This PR tries to make that not happen by adding a `CHECK-LABEL` directive that will match the line with the rustc version string, preventing the previous `CHECK-NOT` from seeing it.
 2023-10-14 19:22:17 +02:00
ltdk
91405ab74a Clean up unchecked_math, separate out unchecked_shifts 2023-10-13 02:17:08 -04:00
bors
df4379b4eb Auto merge of #116510 - scottmcm:no-1-simd-v2, r=compiler-errors 
Copy 1-element arrays as scalars, not vectors

For `[T; 1]` it's silly to copy as `<1 x T>` when we can just copy as `T`.

Inspired by https://github.com/rust-lang/rust/issues/101210#issuecomment-1732470941, which pointed out that `Option<[u8; 1]>` was codegenning worse than `Option<u8>`.

(I'm not sure *why* LLVM doesn't optimize out `<1 x u8>`, but might as well just not emit it in the first place in this codepath.)

---

I think I bit off too much in #116479; let me try just the scalar case first.

r? `@ghost`
 2023-10-12 18:45:01 +00:00
Zalathar
58d62fc271 Don't accidentally detect the commit hash as an fadd instruction 2023-10-10 16:59:49 +11:00
Camille GILLOT
9d211b044d Ignore MSVC in test. 2023-10-08 16:45:45 +00:00
Camille GILLOT
098fc9715e Make FnDef 1-ZST in LLVM debuginfo. 2023-10-08 16:42:45 +00:00
Scott McMurray
ae9cec5839 Copy 1-element arrays as scalars, not vectors 
For `[T; 1]` it's silly to copy as `<1 x T>` when we can just copy as `T`.
 2023-10-07 00:10:32 -07:00
bors
d4ba2b4c7c Auto merge of #116018 - DianQK:simd-wide-sum-test, r=scottmcm 
Increasing the SIMD size improves the vectorization possibilities

Change the `simd-wide-sum.rs` to pass tests based on the LLVM main branch.

For smaller lengths, we cannot expect to always get vectorized.

A related discussion at https://rust-lang.zulipchat.com/#narrow/stream/187780-t-compiler.2Fwg-llvm/topic/LLVM.20HEAD.3A.20codegen.2Fsimd.2Fsimd-wide-sum.2Ers.20newly.20failing.

r? scottmcm
 2023-10-06 08:04:53 +00:00
scottmcm
e300847864
Add a wishlist FIXME 2023-10-06 07:05:09 +00:00
Nikita Popov
5bcf4f26ac Limit to LLVM 17.0.2 to work around WinEH codegen bug 2023-10-02 11:06:38 +02:00
Nikita Popov
0608fca3ad Fix codegen tests on panic=abort targets 2023-10-02 10:37:56 +02:00
Erik Desjardins
31ee8b1818 Reapply: Mark drop calls in landing pads cold instead of noinline 
Co-authored-by: Max Fan <git@max.fan>
Co-authored-by: Nikita Popov <npopov@redhat.com>
 2023-10-02 10:37:53 +02:00
bors
42ca6e4e57 Auto merge of #104385 - BlackHoleFox:apple-minimum-bumps, r=petrochenkov 
Raise minimum supported Apple OS versions

This implements the proposal to raise the minimum supported Apple OS versions as laid out in the now-completed MCP (https://github.com/rust-lang/compiler-team/issues/556).

As of this PR, rustc and the stdlib now support these versions as the baseline:
- macOS: 10.12 Sierra
- iOS: 10
- tvOS: 10
- watchOS: 5 (Unchanged)

In addition to everything this breaks indirectly, these changes also erase the `armv7-apple-ios` target (currently tier 3) because the oldest supported iOS device now uses ARMv7s. Not sure what the policy around tier3 target removal is but shimming it is not an option due to the linker refusing.

[Per comment](https://github.com/rust-lang/compiler-team/issues/556#issuecomment-1297175073), this requires a FCP to merge. cc `@wesleywiser.`
 2023-09-24 02:35:05 +00:00
DianQK
910674f1c4
Only check for successful vectorization on wider_reduce_into_iter 
Different vectorization results are due to different LLVM versions.
 2023-09-24 09:49:39 +08:00
BlackHoleFox
58bbca958d Raise minimum supported macOS to 10.12 2023-09-23 19:14:25 -05:00
bors
13e6f24b9a Auto merge of #107421 - cjgillot:drop-tracking-mir, r=oli-obk 
Enable -Zdrop-tracking-mir by default

This PR enables the `drop-tracking-mir` flag by default. This flag was initially implemented in https://github.com/rust-lang/rust/pull/101692.

This flag computes auto-traits on generators based on their analysis MIR, instead of trying to compute on the HIR body. This removes the need for HIR-based drop-tracking, as we can now reuse the same code to compute generator witness types and to compute generator interior fields.
 2023-09-23 18:28:00 +00:00
bors
19c65022fc Auto merge of #116047 - a-lafrance:I80836-codegen-test, r=Mark-Simulacrum 
Add codegen test to guard against VecDeque optimization regression

Very small PR that adds a codegen test to guard against regression for the `VecDeque` optimization addressed in #80836. Ensures that Rustc optimizes away the panic when unwrapping the result of `.get(0)` because of the `!is_empty()` condition.
 2023-09-23 16:38:20 +00:00
Camille GILLOT
bffb3467e1 Make test more robust to opts. 2023-09-23 13:47:30 +00:00
bors
55b5c7bfde Auto merge of #115695 - tmiasko:compiletest-supported-sanitizers, r=oli-obk 
compiletest: load supported sanitizers from target spec
 2023-09-23 00:25:14 +00:00
Tomasz Miąsko
9090ed8119 Fix test on targets with crt-static default 2023-09-22 18:13:00 +02:00
Arthur Lafrance
d5ec9af09d Add test to guard against VecDeque optimization regression 2023-09-21 20:42:21 -07:00
Ralf Jung
c4ec12f4b7 adjust how closure/generator types and rvalues are printed 2023-09-21 22:20:58 +02:00
DianQK
d30f210e5d
Increasing the SIMD size improves the vectorization possibilities 
Change the simd-wide-sum.rs to pass the LLVM main branching test.
 2023-09-21 12:36:12 +08:00
bors
0e11725809 Auto merge of #115734 - tmiasko:kcfi-no-core, r=compiler-errors 
Use no_core for KCFI tests to exercise them in CI
 2023-09-20 05:24:34 +00:00
Matthias Krüger
7a4904cbdb
Rollup merge of #115591 - djkoloski:issue_115385, r=cuviper 
Add regression test for LLVM 17-rc3 miscompile

Closes #115385, see that issue for more details.
 2023-09-11 21:16:21 +02:00
Tomasz Miąsko
ce19bc3964 Use no_core for KCFI tests to exercise them in CI 2023-09-11 20:54:52 +02:00
bors
62ebe3a2b1 Auto merge of #115417 - dpaoliello:fixdi, r=wesleywiser 
Use the same DISubprogram for each instance of the same inlined function within a caller

# Issue Details:
The call to `panic` within a function like `Option::unwrap` is translated to LLVM as a `tail call` (as it will never return), when multiple calls to the same function like this are inlined LLVM will notice the common `tail call` block (i.e., loading the same panic string + location info and then calling `panic`) and merge them together.

When merging these instructions together, LLVM will also attempt to merge the debug locations as well, but this fails (i.e., debug info is dropped) as Rust emits a new `DISubprogram` at each inline site thus LLVM doesn't recognize that these are actually the same function and so thinks that there isn't a common debug location.

As an example of this, consider the following program:
```rust
#[no_mangle]
fn add_numbers(x: &Option<i32>, y: &Option<i32>) -> i32 {
    let x1 = x.unwrap();
    let y1 = y.unwrap();

    x1 + y1
}
```

 When building for x86_64 Windows using 1.72 it generates (note the lack of `.cv_loc` before the call to `panic`, thus it will be attributed to the same line at the `addq` instruction):

```llvm
	.cv_loc	0 1 3 0                        # src\lib.rs:3:0
	addq	$40, %rsp
	retq
	leaq	.Lalloc_f570dea0a53168780ce9a91e67646421(%rip), %rcx
	leaq	.Lalloc_629ace53b7e5b76aaa810d549cc84ea3(%rip), %r8
	movl	$43, %edx
	callq	_ZN4core9panicking5panic17h12e60b9063f6dee8E
	int3
```

# Fix Details:
Cache the `DISubprogram` emitted for each inlined function instance within a caller so that this can be reused if that instance is encountered again.

Ideally, we would also deduplicate child scopes and variables, however my attempt to do that with #114643 resulted in asserts when building for Linux (#115156) which would require some deep changes to Rust to fix (#115455).

Instead, when using an inlined function as a debug scope, we will also create a new child scope such that subsequent child scopes and variables do not collide (from LLVM's perspective).

After this change the above assembly now (with <https://reviews.llvm.org/D159226> as well) shows the `panic!` was inlined from `unwrap` in `option.rs` at line 935 into the current function in `lib.rs` at line 0 (line 0 is emitted since it is ambiguous which line to use as there were two inline sites that lead to this same code):

```llvm
	.cv_loc	0 1 3 0                        # src\lib.rs:3:0
	addq	$40, %rsp
	retq
	.cv_inline_site_id 6 within 0 inlined_at 1 0 0
	.cv_loc	6 2 935 0                       # library\core\src\option.rs:935:0
	leaq	.Lalloc_5f55955de67e57c79064b537689facea(%rip), %rcx
	leaq	.Lalloc_e741d4de8cb5801e1fd7a6c6795c1559(%rip), %r8
	movl	$43, %edx
	callq	_ZN4core9panicking5panic17hde1558f32d5b1c04E
	int3
```
 2023-09-08 20:56:01 +00:00
bors
cd71a37f32 Auto merge of #115372 - RalfJung:abi-assert-eq, r=davidtwco 
add rustc_abi(assert_eq) to test some guaranteed or at least highly expected ABI compatibility guarantees

This new repr(transparent) test is super useful, it would have found https://github.com/rust-lang/rust/issues/115336 and found https://github.com/rust-lang/rust/issues/115404, https://github.com/rust-lang/rust/issues/115481, https://github.com/rust-lang/rust/issues/115509.
 2023-09-08 11:56:08 +00:00
Ralf Jung
8922c0c541 add support for rustc_abi(assert_eq) and use it to test some repr(transparent) cases 2023-09-07 09:14:29 +02:00
David Koloski
ddd8878d69 Address feedback 2023-09-06 22:16:22 +00:00
bors
e3abbd4994 Auto merge of #114946 - anforowicz:generic-fix-for-asan-lto, r=tmiasko 
Preserve ASAN-related symbols during LTO.

Fixes https://github.com/rust-lang/rust/issues/113404
 2023-09-06 20:04:03 +00:00
David Koloski
c18da3ccd4 Add regression test for LLVM 17-rc3 miscompile 
See #115385 for more details.
 2023-09-06 02:23:48 +00:00
bors
c4f25777a0 Auto merge of #115273 - the8472:take-fold, r=cuviper 
Optimize Take::{fold, for_each} when wrapping TrustedRandomAccess iterators
 2023-09-02 12:40:16 +00:00
The 8472
f93e125828 restrict test to x86-64 2023-09-02 13:42:58 +02:00
Daniel Paoliello
06890774ab Deduplicate inlined function debug info, but create a new lexical scope to child subsequent scopes and variables from colliding 2023-09-01 14:27:21 -07:00
Ding Xiang Fei
67553e8a11
update tests that are ignored by debug 2023-09-01 04:01:54 +08:00
Lukasz Anforowicz
e6dddbda35 Preserve ___asan_globals_registered symbol during LTO. 
Fixes https://github.com/rust-lang/rust/issues/113404
 2023-08-29 19:02:33 +00:00
bors
f3284dc3ad Auto merge of #115260 - scottmcm:not-quite-so-cold, r=WaffleLapkin 
Use `preserve_mostcc` for `extern "rust-cold"`

As experimentation in #115242 has shown looks better than `coldcc`.  Notably, clang exposes `preserve_most` (https://clang.llvm.org/docs/AttributeReference.html#preserve-most) but not `cold`, so this change should put us on a better-supported path.

And *don't* use a different convention for cold on Windows, because that actually ends up making things worse. (See comment in the code.)

cc tracking issue #97544
 2023-08-29 02:23:43 +00:00
bors
9f48a85447 Auto merge of #115050 - khei4:khei4/codegen-move-before-nocapture, r=nikic 
add codegen test for the move before passing to nocapture, by shared-ref arg

This PR adds codegen test for https://github.com/rust-lang/rust/issues/107436#issuecomment-1685792517 (It seems like this works from llvm-16?)

Fixes #107436
 2023-08-28 15:30:28 +00:00
bors
668bf8c593 Auto merge of #115231 - saethlin:dont-ignore-wasm, r=Mark-Simulacrum 
Remove some wasm/emscripten ignores

I'm planning on landing a few PRs like this that remove ignores that aren't required. This just covers mir-opt and codegen tests.
 2023-08-27 17:51:50 +00:00
bors
f0727758d1 Auto merge of #115139 - cjgillot:llvm-fragment, r=nikic 
Do not forget to pass DWARF fragment information to LLVM.

Fixes https://github.com/rust-lang/rust/issues/115113 for the rustc part
 2023-08-27 14:06:57 +00:00
The 8472
72b01d5cca Optimize Take::{fold, for_each} when wrapping TrustedRandomAccess iterators 2023-08-27 15:32:34 +02:00
Matthias Krüger
ce7993670b
Rollup merge of #114957 - loongarch-rs:fix-tests, r=Mark-Simulacrum 
tests: Fix tests for LoongArch64

This PR fixes `lp64d abi` tests for LoongArch64.
 2023-08-27 09:45:18 +02:00
Scott McMurray
754f488d46 Use preserve_mostcc for extern "rust-cold" 
As experimentation in 115242 has shown looks better than `coldcc`.

And *don't* use a different convention for cold on Windows, because that actually ends up making things worse.

cc tracking issue 97544
 2023-08-26 17:42:59 -07:00
Camille GILLOT
5529e2f893 Restrict test to x86_64. 2023-08-26 22:55:52 +00:00
Camille GILLOT
930b2e72ee Do not produce fragment for ZST. 2023-08-26 16:54:28 +00:00
Camille GILLOT
f49494ecce Add test with non-ZST. 2023-08-26 14:21:10 +00:00
Camille GILLOT
b3bbc22cb7 Do not forget to pass DWARF fragment information to LLVM. 2023-08-26 14:21:10 +00:00
khei4
d88c80f5de add codegen test for #107436 
remove trailing whitespace, add trailing newline

fix llvm version and function name
 2023-08-26 18:14:47 +09:00
bors
42857db66d Auto merge of #115232 - wesleywiser:revert_114643, r=tmiasko 
Revert "Use the same DISubprogram for each instance of the same inline function within the caller"

This reverts commit 687bffa493.

Reverting to resolve ICEs reported on nightly.

cc `@dpaoliello`

Fixes #115156
 2023-08-26 07:47:26 +00:00
Scott McMurray
84e305dd93 Stop emitting non-power-of-two vectors in basic LLVM codegen 2023-08-25 20:06:57 -07:00
Wesley Wiser
d0b2c4f727 Revert "Use the same DISubprogram for each instance of the same inlined function within the caller" 
This reverts commit 687bffa493.

Reverting to resolve ICEs reported on nightly.
 2023-08-25 19:49:10 -04:00
Ben Kimock
b678d40826 Remove some wasm/emscripten ignores 2023-08-25 19:48:20 -04:00
Ramon de C Valle
5d6e2d7050 Fix CFI: f32 and f64 are encoded incorrectly for c 
Fix #115150 by encoding f32 and f64 correctly for cross-language CFI. I
missed changing the encoding for f32 and f64 when I introduced the
integer normalization option in #105452 as integer normalization does
not include floating point. `f32` and `f64` should be always encoded as
`f` and `d` since they are both FFI safe when their representation are
the same (i.e., IEEE 754) for both the Rust compiler and Clang.
 2023-08-24 21:02:06 -07:00
bors
97fff1f2ed Auto merge of #114790 - taiki-e:asm-maybe-uninit, r=Amanieu 
Allow MaybeUninit in input and output of inline assembly

**Motivation:**

As part of the work to remove UBs from crossbeam's AtomicCell, I'm writing a library to implement atomic operations on MaybeUnint using inline assembly ([atomic-maybe-uninit](https://github.com/taiki-e/atomic-maybe-uninit), https://github.com/crossbeam-rs/crossbeam/pull/1015).

However, currently, MaybeUnint cannot be used in input&output of inline assembly, so when processing MaybeUninit, values must be [passed through memory](https://github.com/taiki-e/atomic-maybe-uninit/blob/main/src/arch/aarch64.rs#L121-L122). It is inefficient and microbenchmarks have [actually shown significant performance degradation](https://github.com/crossbeam-rs/crossbeam/pull/1015#issuecomment-1676549870).

It would be nice if we could allow MaybeUninit in input and output of inline assembly.

---

This PR changed the type check in rustc_hir_analysis to allow `MaybeUnint<int | float | ptr | fn ptr | simd vector>` in input and output of inline assembly and added a simple test.

To be honest, I'm not sure that this is the correct way to do it, because this is like doing transmute to integers/floats/etc from MaybeUninit on the compiler side. EDIT: [this seems fine](https://rust-lang.zulipchat.com/#narrow/stream/216763-project-inline-asm/topic/MaybeUninit.20in.20asm!/near/384662900)

r? `@Amanieu`
cc `@thomcc` (because you [had previously proposed this](https://rust-lang.zulipchat.com/#narrow/stream/216763-project-inline-asm/topic/MaybeUninit.20in.20asm!))
 2023-08-23 13:40:41 +00:00
Taiki Endo
03fd2d4379 Allow MaybeUninit in input and output of inline assembly 2023-08-23 21:57:18 +09:00
Dylan DPC
391cbdaa7c
Rollup merge of #115096 - kadiwa4:no_memcpy_padding, r=cjgillot 
Add regression test for not `memcpy`ing padding bytes

Closes #56297

See this comparison: https://rust.godbolt.org/z/jjzfonfcE

I don't have any experience with codegen tests, I hope this is correct
 2023-08-23 05:35:17 +00:00
bors
154ae32a55 Auto merge of #114643 - dpaoliello:inlinedebuginfo, r=wesleywiser 
Use the same DISubprogram for each instance of the same inlined function within a caller

# Issue Details:
The call to `panic` within a function like `Option::unwrap` is translated to LLVM as a `tail call` (as it will never return), when multiple calls to the same function like this is inlined LLVM will notice the common `tail call` block (i.e., loading the same panic string + location info and then calling `panic`) and merge them together.

When merging these instructions together, LLVM will also attempt to merge the debug locations as well, but this fails (i.e., debug info is dropped) as Rust emits a new `DISubprogram` at each inline site thus LLVM doesn't recognize that these are actually the same function and so thinks that there isn't a common debug location.

As an example of this when building for x86_64 Windows (note the lack of `.cv_loc` before the call to `panic`, thus it will be attributed to the same line at the `addq` instruction):

```
	.cv_loc	0 1 23 0                        # src\lib.rs:23:0
	addq	$40, %rsp
	retq
	leaq	.Lalloc_f570dea0a53168780ce9a91e67646421(%rip), %rcx
	leaq	.Lalloc_629ace53b7e5b76aaa810d549cc84ea3(%rip), %r8
	movl	$43, %edx
	callq	_ZN4core9panicking5panic17h12e60b9063f6dee8E
	int3
```

# Fix Details:
Cache the `DISubprogram` emitted for each inlined function instance within a caller so that this can be reused if that instance is encountered again, this also requires caching the `DILexicalBlock` and `DIVariable` objects to avoid creating duplicates.

After this change the above assembly now looks like:

```
	.cv_loc	0 1 23 0                        # src\lib.rs:23:0
	addq	$40, %rsp
	retq
	.cv_inline_site_id 5 within 0 inlined_at 1 0 0
	.cv_inline_site_id 6 within 5 inlined_at 1 12 0
	.cv_loc	6 2 935 0                       # library\core\src\option.rs:935:0
	leaq	.Lalloc_5f55955de67e57c79064b537689facea(%rip), %rcx
	leaq	.Lalloc_e741d4de8cb5801e1fd7a6c6795c1559(%rip), %r8
	movl	$43, %edx
	callq	_ZN4core9panicking5panic17hde1558f32d5b1c04E
	int3
```
 2023-08-22 20:15:29 +00:00
kadiwa
265c1b5d50
add regression test for not memcpying padding bytes 2023-08-22 15:10:56 +02:00
WANG Rui
126f4abd8b tests: Fix tests for LoongArch64 2023-08-18 14:56:53 +08:00
Camille GILLOT
3798bca605 Bless codegen tests. 2023-08-17 18:28:33 +00:00
bors
1ec628d7fa Auto merge of #114850 - khei4:khei4/trailing_zero_codegen, r=nikic 
add codegen test for `trailing_zeros` comparison

This PR add codegen test for
https://github.com/rust-lang/rust/issues/107554#issuecomment-1677369236

Fixes #107554.
 2023-08-16 11:07:13 +00:00
khei4
8d514f2e98 add codegen test for issue 107554 
specify llvm-version and bit width for int arg

add missing percent simbol
 2023-08-16 14:04:05 +09:00
DianQK
c12c0841ad Cherry-pick test for issue #114312 2023-08-15 11:33:45 +02:00
DianQK
6f5b4e3581
Add test for method debuginfo declaration. 
We've investigated one reason why debugging information often goes wrong at https://reviews.llvm.org/D152095.
> LLVM can't handle IR where subprogram definitions are nested within DICompositeType when doing LTO builds,
> because there's no good way to cross the CU boundary to insert a nested DISubprogram definition in one CU into a type defined in another CU.
 2023-08-12 21:27:46 +08:00
Daniel Paoliello
687bffa493 Use the same DISubprogram for each instance of the same inlined function within the caller 2023-08-11 10:21:52 -07:00
Scott McMurray
ab6e2bc3d0 Tell LLVM that the negation in <*const T>::sub cannot overflow 
Today it's just `sub` <https://rust.godbolt.org/z/8EzEPnMr5>; with this PR it's `sub nsw`.
 2023-08-10 23:00:39 -07:00
Matthias Krüger
06daa9e263
Rollup merge of #114562 - Trolldemorted:thiscall, r=oli-obk 
stabilize abi_thiscall

Closes https://github.com/rust-lang/rust/issues/42202, stabilizing the use of the "thiscall" ABI.

FCP was substituted by a poll, and the poll has been accepted.
 2023-08-07 16:47:57 +02:00
Benedikt Radtke
3f3262e592 stabilize abi_thiscall 2023-08-07 14:11:03 +02:00
Matthias Krüger
cbe2522652
Rollup merge of #114382 - scottmcm:compare-bytes-intrinsic, r=cjgillot 
Add a new `compare_bytes` intrinsic instead of calling `memcmp` directly

As discussed in #113435, this lets the backends be the place that can have the "don't call the function if n == 0" logic, if it's needed for the target.  (I didn't actually *add* those checks, though, since as I understood it we didn't actually need them on known targets?)

Doing this also let me make it `const` (unstable), which I don't think `extern "C" fn memcmp` can be.

cc `@RalfJung` `@Amanieu`
 2023-08-07 05:29:12 +02:00
Matthias Krüger
fe1c3a1a5e
Rollup merge of #114230 - workingjubilee:codegen-tests-that-nest, r=Mark-Simulacrum 
Nest other codegen test topics

This PR is like rust-lang/rust#114229 in that it mostly pushes codegen tests around, shoving them into their own directories, but because all of the changes are very simple cleanups I pulled them into a separate PR. The other PR might involve actually evaluating the correctness of the test after changes, but here it is mostly a matter of taste. The only "functional" change is deleting a few tests that... hinge on a version of LLVM that we don't support (as of rust-lang/rust#114148 anyways).

I considered a few different ways to group other topics but I feel the question of whether `tests/codegen/{vec,array,slice}` should exist is more subtle than these choices, as it might be better to group such related tests by other topics like bounds check elision, thus I avoided making it.
 2023-08-07 05:29:11 +02:00
Matthias Krüger
137177386b
Rollup merge of #114229 - workingjubilee:nest-sanitizer-dir, r=Mark-Simulacrum 
Nest tests/codegen/sanitizer*.rs tests in sanitizer dir

The sanitizer tests are the largest and most meticulously tested set of tests in tests/codegen. That's good! They all clearly belong to a subject and thus could go in a directory, but are not, instead being placed simply in tests/codegen. That's bad! Fix this by placing them in their own directory and renaming them to be less repetitive after that move.

A few tests are brittle, and embed their filename in the test's checks. This is acceptable for the ones where it is used only two times, but one test embeds the test's mangled filename in the test *over 50 times*! This may have been one of the things discouraging anyone from moving it, and thus from moving the set. Fortunately, I have some knowledge of Itanium mangling (involuntarily), regex, and the FileCheck syntax. With a capturing variable, FileCheck allows us to now move this test around again without diffing it on ~50 lines, while still guaranteeing that the mangled substring is the same each time.

This also clarifies why the substring is repeated a zillion times, instead of being cryptic. They don't call it mangling because the result is pretty and easy to understand, but now it is slightly easier! Yay descriptive variables!
 2023-08-07 05:29:10 +02:00
Scott McMurray
502af03445 Add a new compare_bytes intrinsic instead of calling memcmp directly 2023-08-06 15:47:40 -07:00
bors
abd3637e42 Auto merge of #105545 - erikdesjardins:ptrclean, r=bjorn3 
cleanup: remove pointee types

This can't be merged until the oldest LLVM version we support uses opaque pointers, which will be the case after #114148. (Also note `-Cllvm-args="-opaque-pointers=0"` can technically be used in LLVM 15, though I don't think we should support that configuration.)

I initially hoped this would provide some minor perf win, but in https://github.com/rust-lang/rust/pull/105412#issuecomment-1341224450 it had very little impact, so this is only valuable as a cleanup.

As a followup, this will enable #96242 to be resolved.

r? `@ghost`

`@rustbot` label S-blocked
 2023-08-01 19:44:17 +00:00
Jubilee Young
c81d3e23d1 Remove LLVM 14 codegen tests 
We raised our LLVM minimum to 15, so these tests seem pointless.
 2023-07-29 18:34:41 -07:00
Jubilee Young
f03b31591c tests/codegen/c-variadic* -> cffi/c-variadic* 2023-07-29 18:34:41 -07:00