Commit Graph

144 Commits

Author SHA1 Message Date
Nilstrieb
0f4925e436 Make intrinsic fallback bodies cross-crate inlineable
This change was prompted by the stage1 compiler spending 4% of its time
when compiling the polymorphic-recursion MIR opt test in `unlikely`.

Intrinsic fallback bodies like `unlikely` should always be inlined, it's
very silly if they are not. To do this, we enable the fallback bodies to
be cross-crate inlineable. Not that this matters for our workloads since
the compiler never actually _uses_ the "fallback bodies", it just uses
whatever was cfg(bootstrap)ped, so I've also added `#[inline]` to those.
2024-02-19 19:25:20 +01:00
Michael Goulet
9789e88cfe Check that the ABI of the instance we are inlining is correct 2024-02-11 19:17:42 +00:00
zetanumbers
da4ec6ffa7 Deduplicate tcx.instance_mir(instance) calls in try_instance_mir 2024-02-08 15:47:14 +03:00
Michael Goulet
ca44416023 Fix drop shim for AsyncFnOnce closure, AsyncFnMut shim for AsyncFn closure 2024-02-06 02:22:58 +00:00
Michael Goulet
427896dd7e Construct body for by-move coroutine closure output 2024-02-06 02:22:58 +00:00
Michael Goulet
fc4fff4038 Build a shim to call async closures with different AsyncFn trait kinds 2024-02-06 02:22:58 +00:00
Martin Nordholts
16ba56c242 compiler: Lower fn call arg spans down to MIR
To enable improved accuracy of diagnostics in upcoming commits.
2024-01-15 19:07:11 +01:00
Camille GILLOT
4071572cb4 Merge dead bb pruning and unreachable bb deduplication. 2024-01-07 15:12:10 +00:00
surechen
40ae34194c remove redundant imports
detects redundant imports that can be eliminated.

for #117772 :

In order to facilitate review and modification, split the checking code and
removing redundant imports code into two PR.
2023-12-10 10:56:22 +08:00
Nilstrieb
21a870515b Fix clippy::needless_borrow in the compiler
`x clippy compiler -Aclippy::all -Wclippy::needless_borrow --fix`.

Then I had to remove a few unnecessary parens and muts that were exposed
now.
2023-11-21 20:13:40 +01:00
Tomasz Miąsko
1be1c2ebcf Fix insertion of statements to be executed along return edge in inlining
Inlining creates additional statements to be executed along the return
edge: an assignment to the destination, storage end for temporaries.

Previously those statements where inserted directly into a call target,
but this is incorrect when the target has other predecessors.

Avoid the issue by creating a new dedicated block for those statements.
When the block happens to be redundant it will be removed by CFG
simplification that follows inlining.

Fixes #117355
2023-11-20 09:27:30 +01:00
Tomasz Miąsko
525a64fa81 Use let else to reduce indentation 2023-11-19 22:55:31 +01:00
Ralf Jung
f7985afe4f explain why we don't inline when target features differ 2023-10-28 16:50:40 +02:00
Tomasz Miąsko
011b260cc8 Require target features to match exactly during inlining
In general it is not correct to inline a callee with a target features
that are subset of the callee. Require target features to match exactly
during inlining.

The exact match could be potentially relaxed, but this would require
identifying specific feature that are allowed to differ, those that need
to match, and those that can be present in caller but not in callee.

This resolves MIR part of #116573. For other concerns with respect to
the previous implementation also see areInlineCompatible in LLVM.
2023-10-24 22:49:15 +02:00
Camille GILLOT
3cb0c2e385 Make instance an option in CostChecker. 2023-10-21 06:58:00 +00:00
Camille GILLOT
8252ad02c4 Extract cost checker from inliner. 2023-10-21 06:57:49 +00:00
Oli Scherer
e96ce20b34 s/generator/coroutine/ 2023-10-20 21:14:01 +00:00
Oli Scherer
60956837cf s/Generator/Coroutine/ 2023-10-20 21:10:38 +00:00
Zalathar
7d38f4a611 Remove unused TyCtxt from remove_dead_blocks
This context was only needed by code for processing coverage statements, which
has been removed.
2023-10-18 23:42:45 +11:00
Ben Kimock
33b0e4be06 Automatically enable cross-crate inlining for small functions 2023-10-17 19:53:51 -04:00
Michael Goulet
b2d2184ede Format all the let chains in compiler 2023-10-13 08:59:36 +00:00
Camille GILLOT
e63d19c4dd Remove mir::LocalDecl::internal. 2023-10-04 17:55:15 +00:00
ouz-a
6f0c5ee2d4 change is_subtype to relate_types 2023-10-02 23:39:45 +03:00
lcnr
3c52a3e280 subst -> instantiate 2023-09-26 09:37:55 +02:00
Ralf Jung
c94410c145 rename mir::Constant -> mir::ConstOperand, mir::ConstKind -> mir::Const 2023-09-21 08:12:30 +02:00
Deadbeef
a0a801cd38 treat host effect params as erased generics in codegen
This fixes the changes brought to codegen tests when effect params are
added to libcore, by not attempting to monomorphize functions that get
the host param by being `const fn`.
2023-09-14 07:34:35 +00:00
Tomasz Miąsko
fe3cd2d194 Fix inlining with -Zalways-encode-mir
Only inline functions that are considered eligible for inlining
by the reachability pass.

This constraint was previously indirectly enforced by only exporting MIR
of eligible functions, but that approach doesn't work with
-Zalways-encode-mir enabled.
2023-08-27 23:52:27 +02:00
Ralf Jung
4c53783f3c when terminating during unwinding, show the reason why 2023-08-24 13:28:26 +02:00
Ralf Jung
818ec8e23a give some unwind-related terminators a more clear name 2023-08-20 15:52:38 +02:00
Michael Goulet
3c9549b349 Explicitly don't inline user-written rust-call fns 2023-08-03 18:35:56 +00:00
Michael Goulet
0391af0e1f Only unpack tupled args in inliner if we expect args to be unpacked 2023-08-03 18:35:56 +00:00
Camille GILLOT
b6cd7006e0 Reuse MIR validator for inliner. 2023-07-21 13:58:33 +00:00
Camille GILLOT
45ffe41d14 Substitute types before checking compatibility. 2023-07-19 12:38:15 +00:00
Mahdi Dibaiee
e55583c4b8 refactor(rustc_middle): Substs -> GenericArg 2023-07-14 13:27:35 +01:00
bors
b9ad9b78a2 Auto merge of #112693 - ericmarkmartin:use-more-placeref, r=spastorino
Use PlaceRef abstractions more often

Associated issue: https://github.com/rust-lang/rust/issues/80647

r? `@spastorino`
2023-06-27 00:34:49 +00:00
Eric Mark Martin
c07c10d1e4 use PlaceRef abstractions more consistently 2023-06-25 20:38:01 -04:00
Ziru Niu
8fb4c41f35 merge BorrowKind::Unique into BorrowKind::Mut 2023-06-20 20:55:31 +08:00
Ben Kimock
c153f3a356 Ignore the always part of #[inline(always)] in MIR inlining 2023-06-16 11:36:02 -04:00
Camille GILLOT
9e683442a9 Only check inlining counter after recusing. 2023-06-03 12:29:49 +00:00
lcnr
08d149ca85 EarlyBinder::new -> EarlyBinder::bind 2023-05-29 13:46:10 +02:00
Kyle Matsuda
03534ac8b7 Replace EarlyBinder(x) with EarlyBinder::new(x) 2023-05-28 10:44:50 -06:00
Guillaume Gomez
ddb5424569
Rollup merge of #111952 - cjgillot:drop-replace, r=WaffleLapkin
Remove DesugaringKind::Replace.

A simple boolean flag is enough.
2023-05-27 13:38:31 +02:00
Camille GILLOT
844c1cc5fe Remove DesugaringKind::Replace. 2023-05-25 17:40:46 +00:00
Camille GILLOT
0919ec3ecc Remove ExpnKind::Inlined. 2023-05-25 16:43:14 +00:00
Jakob Degen
60cc72cf7b Don't inline functions with unsized args 2023-05-20 17:45:50 -07:00
Dylan DPC
828caa80a9
Rollup merge of #110930 - b-naber:normalize-elaborate-drops, r=cjgillot
Don't expect normalization to succeed in elaborate_drops

Fixes https://github.com/rust-lang/rust/issues/110682

This was exposed through the changes in https://github.com/rust-lang/rust/pull/109247, which causes more things to be inlined. Inlining can happen before monomorphization, so we can't expect normalization to succeed. In the elaborate_drops analysis we currently have [this call](033aa092ab/compiler/rustc_mir_dataflow/src/elaborate_drops.rs (L278)) to `normalize_erasing_regions`, which ICEs when normalization fails. The types are used to infer [whether the type needs a drop](033aa092ab/compiler/rustc_mir_dataflow/src/elaborate_drops.rs (L374)), where `needs_drop` itself [uses `try_normalize_erasing_regions`](033aa092ab/compiler/rustc_middle/src/ty/util.rs (L1121)).

~[`instance_mir`](https://doc.rust-lang.org/stable/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.instance_mir) isn't explicit about whether it expects the instances corresponding to the `InstanceDef`s to be monomorphized (though I think in all other contexts the function is used post-monomorphization), so the use of `instance_mir` in inlining doesn't necessarily seem wrong to me.~
2023-05-17 19:11:53 +05:30
b-naber
e7a2f52ba1 don't inline polymorphic adt instances whose fields contain projections
in DropGlue.
2023-05-10 16:03:52 +00:00
Kyle Matsuda
e4f6b8b43b make subst_mir take EarlyBinder 2023-05-06 22:32:39 -06:00
Kyle Matsuda
82f57c16b7 use EarlyBinder in tcx.(try_)subst_mir_and_normalize_erasing_regions 2023-05-06 22:32:39 -06:00
Maybe Waffle
e496fbec92 Split {Idx, IndexVec, IndexSlice} into their own modules 2023-04-24 13:53:35 +00:00