rust/compiler
bors c71248b708 Auto merge of #76859 - Aaron1011:fix/llvm-cgu-reuse, r=davidtwco,nikic
Use llvm::computeLTOCacheKey to determine post-ThinLTO CGU reuse

During incremental ThinLTO compilation, we attempt to re-use the
optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do
so.

Up until now, 'safe' has meant that the set of modules that our current
modules imports from/exports to is unchanged from the previous
compilation session. See PR #67020 and PR #71131 for more details.

However, this turns out be insufficient to guarantee that it's safe
to reuse the post-LTO module (i.e. that optimizing the pre-LTO module
would produce the same result). When LLVM optimizes a module during
ThinLTO, it may look at other information from the 'module index', such
as whether a (non-imported!) global variable is used. If this
information changes between compilation runs, we may end up re-using an
optimized module that (for example) had dead-code elimination run on a
function that is now used by another module.

Fortunately, LLVM implements its own ThinLTO module cache, which is used
when ThinLTO is performed by a linker plugin (e.g. when clang is used to
compile a C proect). Using this cache directly would require extensive
refactoring of our code - but fortunately for us, LLVM provides a
function that does exactly what we need.

The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash
from all data that might influence the result of ThinLTO on a module.
In addition to the module imports/exports that we manually track, it
also hashes information about global variables (e.g. their liveness)
which might be used during optimization. By using this function, we
shouldn't have to worry about new LLVM passes breaking our module re-use
behavior.

In LLVM, the output of this function forms part of the filename used to
store the post-ThinLTO module. To keep our current filename structure
intact, this PR just writes out the mapping 'CGU name -> Hash' to a
file. To determine if a post-LTO module should be reused, we compare
hashes from the previous session.

This should unblock PR #75199 - by sheer chance, it seems to have hit
this issue due to the particular CGU partitioning and optimization
decisions that end up getting made.
2020-10-11 20:50:02 +00:00
..
rustc just max_level_info 2020-09-11 09:37:51 -07:00
rustc_apfloat Backport another LLVM commit to rustc_apfloat 2020-10-04 02:02:25 +02:00
rustc_arena Remove unused #[allow(...)] statements from compiler/ 2020-09-26 01:25:55 +02:00
rustc_ast address review comments 2020-10-09 22:00:48 -07:00
rustc_ast_lowering Separate bounds and predicates for associated/opaque types 2020-10-06 11:19:29 +01:00
rustc_ast_passes use matches!() macro for simple if let conditions 2020-09-18 20:28:35 +02:00
rustc_ast_pretty Fully integrate token collection for additional AST structs 2020-09-10 17:58:14 -04:00
rustc_attr Implement the instruction_set attribute 2020-10-08 23:32:20 +01:00
rustc_builtin_macros Auto merge of #77595 - petrochenkov:asmident, r=oli-obk 2020-10-07 11:51:51 +00:00
rustc_codegen_llvm Auto merge of #76859 - Aaron1011:fix/llvm-cgu-reuse, r=davidtwco,nikic 2020-10-11 20:50:02 +00:00
rustc_codegen_ssa Let backends access span information 2020-10-06 15:39:12 +02:00
rustc_data_structures Auto merge of #77080 - richkadel:llvm-coverage-counters-2, r=tmandry 2020-10-05 19:34:44 +00:00
rustc_driver Auto merge of #77649 - dash2507:replace_run_compiler, r=matthewjasper 2020-10-11 01:26:06 +00:00
rustc_error_codes Fix typo in error code description 2020-10-10 18:02:53 +09:00
rustc_errors /nightly/nightly-rustc 2020-09-23 21:51:56 +02:00
rustc_expand Detect blocks that could be struct expr bodies 2020-10-07 13:40:52 -07:00
rustc_feature Implement the instruction_set attribute 2020-10-08 23:32:20 +01:00
rustc_fs_util mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_graphviz /nightly/nightly-rustc 2020-09-23 21:51:56 +02:00
rustc_hir Rollup merge of #77514 - scottmcm:less-once-chain-once, r=estebank 2020-10-05 02:29:42 +02:00
rustc_hir_pretty mv compiler to compiler/ 2020-08-30 18:45:07 +03:00
rustc_incremental /nightly/nightly-rustc 2020-09-23 21:51:56 +02:00
rustc_index cache types during normalization 2020-09-19 17:27:13 +02:00
rustc_infer Remove unused part of return value from replace_bound_vars_with_placeholders 2020-10-06 11:19:30 +01:00
rustc_interface Auto merge of #77565 - khyperia:codegen-backend-dep, r=ecstatic-morse 2020-10-11 06:03:23 +00:00
rustc_lexer Noticed a potential bug in eat_while(): it doesn't account for number of UTF8 bytes. 2020-10-09 11:12:54 +02:00
rustc_lint Auto merge of #77119 - GuillaumeGomez:unclosed-html-tag-lint, r=jyn514 2020-10-07 09:56:51 +00:00
rustc_llvm Auto merge of #76859 - Aaron1011:fix/llvm-cgu-reuse, r=davidtwco,nikic 2020-10-11 20:50:02 +00:00
rustc_macros Remove unused #[allow(...)] statements from compiler/ 2020-09-26 01:25:55 +02:00
rustc_metadata Revert calculate_dtor signature change 2020-10-09 17:18:57 +02:00
rustc_middle Auto merge of #76934 - camelid:rustdoc-allow-generic-params, r=jyn514 2020-10-10 21:19:50 +00:00
rustc_mir Auto merge of #77793 - tmiasko:no-op-discriminant, r=ecstatic-morse 2020-10-11 16:33:47 +00:00
rustc_mir_build Rollup merge of #77560 - rschoon:fix-litkind-rc-bytebuf, r=lcnr 2020-10-06 16:26:11 +09:00
rustc_parse rustc_parse: More precise spans for tuple.0.0 2020-10-11 02:33:49 +03:00
rustc_parse_format /nightly/nightly-rustc 2020-09-23 21:51:56 +02:00
rustc_passes Rollup merge of #76329 - GuillaumeGomez:doc-alias-crate-level, r=matthewjasper 2020-10-05 02:29:27 +02:00
rustc_plugin_impl /nightly/nightly-rustc 2020-09-23 21:51:56 +02:00
rustc_privacy Separate bounds and predicates for associated/opaque types 2020-10-06 11:19:29 +01:00
rustc_query_system update the version of itertools and parking_lot 2020-09-12 08:26:53 +02:00
rustc_resolve address review comments 2020-10-09 22:00:48 -07:00
rustc_save_analysis /nightly/nightly-rustc 2020-09-23 21:51:56 +02:00
rustc_serialize Deduplicate and generalize some (de/)serializer impls 2020-09-26 14:55:42 +02:00
rustc_session Auto merge of #77119 - GuillaumeGomez:unclosed-html-tag-lint, r=jyn514 2020-10-07 09:56:51 +00:00
rustc_span Auto merge of #77687 - est31:hash_shorter_path, r=davidtwco 2020-10-09 08:09:32 +00:00
rustc_symbol_mangling Rollup merge of #77605 - da-x:fix-rustc-def-path, r=petrochenkov 2020-10-07 00:16:12 +02:00
rustc_target Auto merge of #77337 - lzutao:asm-mips64, r=Amanieu 2020-10-10 17:32:28 +00:00
rustc_trait_selection Add find_map_relevant_impl 2020-10-09 16:22:49 +02:00
rustc_traits Update chalk to 0.32.0 2020-10-08 13:17:01 +02:00
rustc_ty Move item_bounds to typeck::collect 2020-10-06 11:18:45 +01:00
rustc_typeck Revert calculate_dtor signature change 2020-10-09 17:18:57 +02:00