rust/compiler
Aaron Hill cfe07cd42a
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse
During incremental ThinLTO compilation, we attempt to re-use the
optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do
so.

Up until now, 'safe' has meant that the set of modules that our current
modules imports from/exports to is unchanged from the previous
compilation session. See PR #67020 and PR #71131 for more details.

However, this turns out be insufficient to guarantee that it's safe
to reuse the post-LTO module (i.e. that optimizing the pre-LTO module
would produce the same result). When LLVM optimizes a module during
ThinLTO, it may look at other information from the 'module index', such
as whether a (non-imported!) global variable is used. If this
information changes between compilation runs, we may end up re-using an
optimized module that (for example) had dead-code elimination run on a
function that is now used by another module.

Fortunately, LLVM implements its own ThinLTO module cache, which is used
when ThinLTO is performed by a linker plugin (e.g. when clang is used to
compile a C proect). Using this cache directly would require extensive
refactoring of our code - but fortunately for us, LLVM provides a
function that does exactly what we need.

The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash
from all data that might influence the result of ThinLTO on a module.
In addition to the module imports/exports that we manually track, it
also hashes information about global variables (e.g. their liveness)
which might be used during optimization. By using this function, we
shouldn't have to worry about new LLVM passes breaking our module re-use
behavior.

In LLVM, the output of this function forms part of the filename used to
store the post-ThinLTO module. To keep our current filename structure
intact, this PR just writes out the mapping 'CGU name -> Hash' to a
file. To determine if a post-LTO module should be reused, we compare
hashes from the previous session.

This should unblock PR #75199 - by sheer chance, it seems to have hit
this issue due to the particular CGU partitioning and optimization
decisions that end up getting made.
2020-09-17 22:04:13 -04:00
..
rustc just max_level_info 2020-09-11 09:37:51 -07:00
rustc_apfloat mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_arena mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_ast Auto merge of #76541 - matthiaskrgr:unstable_sort, r=davidtwco 2020-09-14 21:43:17 +00:00
rustc_ast_lowering Auto merge of #76541 - matthiaskrgr:unstable_sort, r=davidtwco 2020-09-14 21:43:17 +00:00
rustc_ast_passes Stabilize doc_alias feature 2020-09-14 11:03:47 +02:00
rustc_ast_pretty Fully integrate token collection for additional AST structs 2020-09-10 17:58:14 -04:00
rustc_attr mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_builtin_macros Attach tokens to ast::Stmt 2020-09-10 17:33:06 -04:00
rustc_codegen_llvm Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse 2020-09-17 22:04:13 -04:00
rustc_codegen_ssa use push(char) instead of push_str(&str) to add single chars to strings 2020-09-10 13:58:41 +02:00
rustc_data_structures update the version of itertools and parking_lot 2020-09-12 08:26:53 +02:00
rustc_driver just max_level_info 2020-09-11 09:37:51 -07:00
rustc_error_codes Adjust spelling 2020-09-15 18:10:41 +02:00
rustc_errors replacing sub's that can wrap by saturating_sub's 2020-09-11 11:11:11 +02:00
rustc_expand Rollup merge of #75146 - tmiasko:range-overflow, r=Mark-Simulacrum 2020-09-16 01:30:30 +02:00
rustc_feature Stabilize doc_alias feature 2020-09-14 11:03:47 +02:00
rustc_fs_util mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_graphviz Make graphviz font configurable 2020-09-16 08:10:06 -07:00
rustc_hir review, improve note span 2020-09-13 22:53:51 +02:00
rustc_hir_pretty mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_incremental mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_index Improve BitSet APIs 2020-08-30 11:13:18 -07:00
rustc_infer Rollup merge of #76756 - matthiaskrgr:cl123ppy, r=Dylan-DPC 2020-09-16 12:24:17 -07:00
rustc_interface Attach tokens to ast::Stmt 2020-09-10 17:33:06 -04:00
rustc_lexer lexer: Tiny improvement to shebang detection 2020-09-02 00:40:19 +03:00
rustc_lint don't lazily evaulate some trivial values for Option::None replacements (clippy::unnecessary_lazy_evaluations) 2020-09-15 23:17:51 +02:00
rustc_llvm Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse 2020-09-17 22:04:13 -04:00
rustc_macros Fix non-determinism in generated format string. 2020-09-09 21:23:25 +10:00
rustc_metadata don't lazily evaulate some trivial values for Option::None replacements (clippy::unnecessary_lazy_evaluations) 2020-09-15 23:17:51 +02:00
rustc_middle Rollup merge of #76756 - matthiaskrgr:cl123ppy, r=Dylan-DPC 2020-09-16 12:24:17 -07:00
rustc_mir Auto merge of #76634 - RalfJung:miri-guaranteed-eq-ne, r=oli-obk 2020-09-17 12:15:57 +00:00
rustc_mir_build Note when a a move/borrow error is caused by a deref coercion 2020-09-10 20:56:20 -04:00
rustc_parse improve diagnostics for lifetime after &mut 2020-09-15 10:36:06 -04:00
rustc_parse_format mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_passes allow concrete self types in consts 2020-09-13 22:53:51 +02:00
rustc_plugin_impl mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_privacy Change ty.kind to a method 2020-09-04 17:47:51 +02:00
rustc_query_system update the version of itertools and parking_lot 2020-09-12 08:26:53 +02:00
rustc_resolve Rollup merge of #76756 - matthiaskrgr:cl123ppy, r=Dylan-DPC 2020-09-16 12:24:17 -07:00
rustc_save_analysis use push(char) instead of push_str(&str) to add single chars to strings 2020-09-10 13:58:41 +02:00
rustc_serialize mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_session Rollup merge of #76794 - richkadel:graphviz-font, r=ecstatic-morse 2020-09-16 12:24:30 -07:00
rustc_span Rollup merge of #75304 - Aaron1011:feature/diag-deref-move-out, r=estebank 2020-09-16 01:30:32 +02:00
rustc_symbol_mangling Change ty.kind to a method 2020-09-04 17:47:51 +02:00
rustc_target librustc_target: Address comments 2020-09-14 17:43:06 -07:00
rustc_trait_selection Rollup merge of #76756 - matthiaskrgr:cl123ppy, r=Dylan-DPC 2020-09-16 12:24:17 -07:00
rustc_traits Rollup merge of #76641 - nox:pointee-random-stuff, r=eddyb 2020-09-16 08:25:00 +02:00
rustc_ty Remove def_id field from ParamEnv 2020-09-09 10:14:31 +02:00
rustc_typeck Auto merge of #76028 - aticu:improve_e0118, r=estebank,jyn514,GuillaumeGomez 2020-09-17 03:56:38 +00:00