Nilstrieb
0f4925e436
Make intrinsic fallback bodies cross-crate inlineable
...
This change was prompted by the stage1 compiler spending 4% of its time
when compiling the polymorphic-recursion MIR opt test in `unlikely`.
Intrinsic fallback bodies like `unlikely` should always be inlined, it's
very silly if they are not. To do this, we enable the fallback bodies to
be cross-crate inlineable. Not that this matters for our workloads since
the compiler never actually _uses_ the "fallback bodies", it just uses
whatever was cfg(bootstrap)ped, so I've also added `#[inline]` to those.
2024-02-19 19:25:20 +01:00
Michael Goulet
9789e88cfe
Check that the ABI of the instance we are inlining is correct
2024-02-11 19:17:42 +00:00
zetanumbers
da4ec6ffa7
Deduplicate tcx.instance_mir(instance)
calls in try_instance_mir
2024-02-08 15:47:14 +03:00
Michael Goulet
ca44416023
Fix drop shim for AsyncFnOnce closure, AsyncFnMut shim for AsyncFn closure
2024-02-06 02:22:58 +00:00
Michael Goulet
427896dd7e
Construct body for by-move coroutine closure output
2024-02-06 02:22:58 +00:00
Michael Goulet
fc4fff4038
Build a shim to call async closures with different AsyncFn trait kinds
2024-02-06 02:22:58 +00:00
Martin Nordholts
16ba56c242
compiler: Lower fn call arg spans down to MIR
...
To enable improved accuracy of diagnostics in upcoming commits.
2024-01-15 19:07:11 +01:00
Camille GILLOT
4071572cb4
Merge dead bb pruning and unreachable bb deduplication.
2024-01-07 15:12:10 +00:00
surechen
40ae34194c
remove redundant imports
...
detects redundant imports that can be eliminated.
for #117772 :
In order to facilitate review and modification, split the checking code and
removing redundant imports code into two PR.
2023-12-10 10:56:22 +08:00
Nilstrieb
21a870515b
Fix clippy::needless_borrow
in the compiler
...
`x clippy compiler -Aclippy::all -Wclippy::needless_borrow --fix`.
Then I had to remove a few unnecessary parens and muts that were exposed
now.
2023-11-21 20:13:40 +01:00
Tomasz Miąsko
1be1c2ebcf
Fix insertion of statements to be executed along return edge in inlining
...
Inlining creates additional statements to be executed along the return
edge: an assignment to the destination, storage end for temporaries.
Previously those statements where inserted directly into a call target,
but this is incorrect when the target has other predecessors.
Avoid the issue by creating a new dedicated block for those statements.
When the block happens to be redundant it will be removed by CFG
simplification that follows inlining.
Fixes #117355
2023-11-20 09:27:30 +01:00
Tomasz Miąsko
525a64fa81
Use let else to reduce indentation
2023-11-19 22:55:31 +01:00
Ralf Jung
f7985afe4f
explain why we don't inline when target features differ
2023-10-28 16:50:40 +02:00
Tomasz Miąsko
011b260cc8
Require target features to match exactly during inlining
...
In general it is not correct to inline a callee with a target features
that are subset of the callee. Require target features to match exactly
during inlining.
The exact match could be potentially relaxed, but this would require
identifying specific feature that are allowed to differ, those that need
to match, and those that can be present in caller but not in callee.
This resolves MIR part of #116573 . For other concerns with respect to
the previous implementation also see areInlineCompatible in LLVM.
2023-10-24 22:49:15 +02:00
Camille GILLOT
3cb0c2e385
Make instance an option in CostChecker.
2023-10-21 06:58:00 +00:00
Camille GILLOT
8252ad02c4
Extract cost checker from inliner.
2023-10-21 06:57:49 +00:00
Oli Scherer
e96ce20b34
s/generator/coroutine/
2023-10-20 21:14:01 +00:00
Oli Scherer
60956837cf
s/Generator/Coroutine/
2023-10-20 21:10:38 +00:00
Zalathar
7d38f4a611
Remove unused TyCtxt
from remove_dead_blocks
...
This context was only needed by code for processing coverage statements, which
has been removed.
2023-10-18 23:42:45 +11:00
Ben Kimock
33b0e4be06
Automatically enable cross-crate inlining for small functions
2023-10-17 19:53:51 -04:00
Michael Goulet
b2d2184ede
Format all the let chains in compiler
2023-10-13 08:59:36 +00:00
Camille GILLOT
e63d19c4dd
Remove mir::LocalDecl::internal.
2023-10-04 17:55:15 +00:00
ouz-a
6f0c5ee2d4
change is_subtype to relate_types
2023-10-02 23:39:45 +03:00
lcnr
3c52a3e280
subst -> instantiate
2023-09-26 09:37:55 +02:00
Ralf Jung
c94410c145
rename mir::Constant -> mir::ConstOperand, mir::ConstKind -> mir::Const
2023-09-21 08:12:30 +02:00
Deadbeef
a0a801cd38
treat host effect params as erased generics in codegen
...
This fixes the changes brought to codegen tests when effect params are
added to libcore, by not attempting to monomorphize functions that get
the host param by being `const fn`.
2023-09-14 07:34:35 +00:00
Tomasz Miąsko
fe3cd2d194
Fix inlining with -Zalways-encode-mir
...
Only inline functions that are considered eligible for inlining
by the reachability pass.
This constraint was previously indirectly enforced by only exporting MIR
of eligible functions, but that approach doesn't work with
-Zalways-encode-mir enabled.
2023-08-27 23:52:27 +02:00
Ralf Jung
4c53783f3c
when terminating during unwinding, show the reason why
2023-08-24 13:28:26 +02:00
Ralf Jung
818ec8e23a
give some unwind-related terminators a more clear name
2023-08-20 15:52:38 +02:00
Michael Goulet
3c9549b349
Explicitly don't inline user-written rust-call fns
2023-08-03 18:35:56 +00:00
Michael Goulet
0391af0e1f
Only unpack tupled args in inliner if we expect args to be unpacked
2023-08-03 18:35:56 +00:00
Camille GILLOT
b6cd7006e0
Reuse MIR validator for inliner.
2023-07-21 13:58:33 +00:00
Camille GILLOT
45ffe41d14
Substitute types before checking compatibility.
2023-07-19 12:38:15 +00:00
Mahdi Dibaiee
e55583c4b8
refactor(rustc_middle): Substs -> GenericArg
2023-07-14 13:27:35 +01:00
bors
b9ad9b78a2
Auto merge of #112693 - ericmarkmartin:use-more-placeref, r=spastorino
...
Use PlaceRef abstractions more often
Associated issue: https://github.com/rust-lang/rust/issues/80647
r? `@spastorino`
2023-06-27 00:34:49 +00:00
Eric Mark Martin
c07c10d1e4
use PlaceRef abstractions more consistently
2023-06-25 20:38:01 -04:00
Ziru Niu
8fb4c41f35
merge BorrowKind::Unique
into BorrowKind::Mut
2023-06-20 20:55:31 +08:00
Ben Kimock
c153f3a356
Ignore the always part of #[inline(always)] in MIR inlining
2023-06-16 11:36:02 -04:00
Camille GILLOT
9e683442a9
Only check inlining counter after recusing.
2023-06-03 12:29:49 +00:00
lcnr
08d149ca85
EarlyBinder::new -> EarlyBinder::bind
2023-05-29 13:46:10 +02:00
Kyle Matsuda
03534ac8b7
Replace EarlyBinder(x) with EarlyBinder::new(x)
2023-05-28 10:44:50 -06:00
Guillaume Gomez
ddb5424569
Rollup merge of #111952 - cjgillot:drop-replace, r=WaffleLapkin
...
Remove DesugaringKind::Replace.
A simple boolean flag is enough.
2023-05-27 13:38:31 +02:00
Camille GILLOT
844c1cc5fe
Remove DesugaringKind::Replace.
2023-05-25 17:40:46 +00:00
Camille GILLOT
0919ec3ecc
Remove ExpnKind::Inlined.
2023-05-25 16:43:14 +00:00
Jakob Degen
60cc72cf7b
Don't inline functions with unsized args
2023-05-20 17:45:50 -07:00
Dylan DPC
828caa80a9
Rollup merge of #110930 - b-naber:normalize-elaborate-drops, r=cjgillot
...
Don't expect normalization to succeed in elaborate_drops
Fixes https://github.com/rust-lang/rust/issues/110682
This was exposed through the changes in https://github.com/rust-lang/rust/pull/109247 , which causes more things to be inlined. Inlining can happen before monomorphization, so we can't expect normalization to succeed. In the elaborate_drops analysis we currently have [this call](033aa092ab/compiler/rustc_mir_dataflow/src/elaborate_drops.rs (L278)
) to `normalize_erasing_regions`, which ICEs when normalization fails. The types are used to infer [whether the type needs a drop](033aa092ab/compiler/rustc_mir_dataflow/src/elaborate_drops.rs (L374)
), where `needs_drop` itself [uses `try_normalize_erasing_regions`](033aa092ab/compiler/rustc_middle/src/ty/util.rs (L1121)
).
~[`instance_mir`](https://doc.rust-lang.org/stable/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.instance_mir ) isn't explicit about whether it expects the instances corresponding to the `InstanceDef`s to be monomorphized (though I think in all other contexts the function is used post-monomorphization), so the use of `instance_mir` in inlining doesn't necessarily seem wrong to me.~
2023-05-17 19:11:53 +05:30
b-naber
e7a2f52ba1
don't inline polymorphic adt instances whose fields contain projections
...
in DropGlue.
2023-05-10 16:03:52 +00:00
Kyle Matsuda
e4f6b8b43b
make subst_mir take EarlyBinder
2023-05-06 22:32:39 -06:00
Kyle Matsuda
82f57c16b7
use EarlyBinder in tcx.(try_)subst_mir_and_normalize_erasing_regions
2023-05-06 22:32:39 -06:00
Maybe Waffle
e496fbec92
Split {Idx, IndexVec, IndexSlice}
into their own modules
2023-04-24 13:53:35 +00:00