We've been relying on a custom patch to add `MCSubtargetInfo::getCPUTable`
for `rustc --print target-cpus`, and just printing that it's not supported
on external LLVM builds. LLVM `main` now has `getAllProcessorDescriptions`
that can replace ours, so now we try to use that. In addition, the fallback
path can at least print the native and default cpu options.
There were also some mismatches in the function signatures here between
`LLVM_RUSTLLVM` and otherwise; this is now mitigated by sharing these
functions and only using cpp to adjust the function bodies.
Fix printing native CPU on cross-compiled compiler.
If `rustc` is cross-compiled from a different host, then the "native" entry in `rustc --print=target-cpus` would not appear. There is a check in the printing code that will avoid printing the "native" entry if the user has passed `--target`. However, that check was comparing the `--target` value with the `LLVM_TARGET_TRIPLE` which is the triple of the host that `rustc` was built on (the "build" target in Rust lingo), not the target it was being built for (the "host" in Rust lingo). This fixes it to use the target that LLVM was built for (which I'm pretty sure this is the correct function to determine that).
This fixes the cpu listing for aarch64-apple-darwin which is built on CI using the x86_64-apple-darwin host.
Initial support for loongarch64-unknown-linux-gnu
Hi, We hope to add a new port in rust for LoongArch.
LoongArch intro
LoongArch is a RISC style ISA which is independently designed by Loongson
Technology in China. It is divided into two versions, the 32-bit version (LA32)
and the 64-bit version (LA64). LA64 applications have application-level
backward binary compatibility with LA32 applications. LoongArch is composed of
a basic part (Loongson Base) and an expanded part. The expansion part includes
Loongson Binary Translation (LBT), Loongson VirtualiZation (LVZ), Loongson SIMD
EXtension (LSX) and Loongson Advanced SIMD EXtension(LASX).
Currently the LA464 processor core supports LoongArch ISA and the Loongson
3A5000 processor integrates 4 64-bit LA464 cores. LA464 is a four-issue 64-bit
high-performance processor core. It can be used as a single core for high-end
embedded and desktop applications, or as a basic processor core to form an
on-chip multi-core system for server and high-performance machine applications.
Documentations:
ISA:
https://loongson.github.io/LoongArch-Documentation/LoongArch-Vol1-EN.html
ABI:
https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html
More docs can be found at:
https://loongson.github.io/LoongArch-Documentation/README-EN.html
Since last year, we have locally adapted two versions of rust, rust1.41 and rust1.57, and completed the test locally.
I'm not sure if I'm submitting all the patches at once, so I split up the patches and here's one of the commits
LLVM 15 added `Optional::has_value`, and LLVM `main` (16) has deprecated
`hasValue`. However, its `explicit operator bool` does the same thing,
and was added long ago, so we can use that across our full LLVM range of
compatibility.
No functional changes intended.
LLVM commit 633f5663c3 removed `createWriteThinLTOBitcodePass`.
This adapts PassWrapper similarly to the example mentioned upstream: 633f5663c3.
Update the minimum external LLVM to 13
With this change, we'll have stable support for LLVM 13 through 15 (pending release).
For reference, the previous increase to LLVM 12 was #90175.
r? `@nagisa`
Allow to disable thinLTO buffer to support lto-embed-bitcode lld feature
Hello
This change is to fix issue (https://github.com/rust-lang/rust/issues/84395) in which passing "-lto-embed-bitcode=optimized" to lld when linking rust code via linker-plugin-lto doesn't produce the expected result.
Instead of emitting a single unified module into a llvmbc section of the linked elf, it emits multiple submodules.
This is caused because rustc emits the BC modules after running llvm `createWriteThinLTOBitcodePass` pass.
Which in turn triggers a thinLTO linkage and causes the said issue.
This patch allows via compiler flag (-Cemit-thin-lto=<bool>) to select between running `createWriteThinLTOBitcodePass` and `createBitcodeWriterPass`.
Note this pattern of selecting between those 2 passes is common inside of LLVM code.
The default is to match the old behavior.
Revert "Work around invalid DWARF bugs for fat LTO"
Since September, the toolchain has not been generating reliable DWARF
information for static variables when LTO is on. This has affected
projects in the embedded space where the use of LTO is typical. In our
case, it has kept us from bumping past the 2021-09-22 nightly toolchain
lest our debugger break. This has been a pretty dramatic regression for
people using debuggers and static variables. See #90357 for more info
and a repro case.
This commit is a mechanical revert of
d5de680e20 from PR #89041, which caused
the issue. (Note on that PR that the commit's author has requested it be
reverted.)
I have locally verified that this fixes#90357 by restoring the
functionality of both the repro case I posted on that bug, and debugger
behavior on real programs. There do not appear to be test cases for this
in the toolchain; if I've missed them, point me at 'em and I'll update
them.
Adding the option to control from rustc CLI
if the resulted ".o" bitcode module files are with
thinLTO info or regular LTO info.
Allows using "-lto-embed-bitcode=optimized" during linkage
correctly.
Signed-off-by: Ziv Dunkelman <ziv.dunkelman@nextsilicon.com>
Since September, the toolchain has not been generating reliable DWARF
information for static variables when LTO is on. This has affected
projects in the embedded space where the use of LTO is typical. In our
case, it has kept us from bumping past the 2021-09-22 nightly toolchain
lest our debugger break. This has been a pretty dramatic regression for
people using debuggers and static variables. See #90357 for more info
and a repro case.
This commit is a mechanical revert of
d5de680e20 from PR #89041, which caused
the issue. (Note on that PR that the commit's author has requested it be
reverted.)
I have locally verified that this fixes#90357 by restoring the
functionality of both the repro case I posted on that bug, and debugger
behavior on real programs. There do not appear to be test cases for this
in the toolchain; if I've missed them, point me at 'em and I'll update
them.
This was originally introduced in #10916 as a way to remove all landing
pads when performing LTO. However this is no longer necessary today
since rustc properly marks all functions and call-sites as nounwind
where appropriate.
In fact this is incorrect in the presence of `extern "C-unwind"` which
must create a landing pad when compiled with `-C panic=abort` so that
foreign exceptions are caught and properly turned into aborts.
In https://reviews.llvm.org/D71059 LLVM 11, the time trace profiler was
extended to support multiple threads.
`timeTraceProfilerInitialize` creates a thread local profiler instance.
When a thread finishes `timeTraceProfilerFinishThread` moves a thread
local instance into a global collection of instances. Finally when all
codegen work is complete `timeTraceProfilerWrite` writes data from the
current thread local instance and the instances in global collection
of instances.
Previously, the profiler was intialized on a single thread only. Since
this thread performs no code generation on its own, the resulting
profile was empty.
Update LLVM codegen to initialize & finish time trace profiler on each
code generation thread.
Add -Z no-unique-section-names to reduce ELF header bloat.
This change adds a new compiler flag that can help reduce the size of ELF binaries that contain many functions.
By default, when enabling function sections (which is the default for most targets), the LLVM backend will generate different section names for each function. For example, a function `func` would generate a section called `.text.func`. Normally this is fine because the linker will merge all those sections into a single one in the binary. However, starting with [LLVM 12](https://github.com/llvm/llvm-project/commit/ee5d1a04), the backend will also generate unique section names for exception handling, resulting in thousands of `.gcc_except_table.*` sections ending up in the final binary because some linkers like LLD don't currently merge or strip these EH sections (see discussion [here](https://reviews.llvm.org/D83655)). This can bloat the ELF headers and string table significantly in binaries that contain many functions.
The new option is analogous to Clang's `-fno-unique-section-names`, and instructs LLVM to generate the same `.text` and `.gcc_except_table` section for each function, resulting in a smaller final binary.
The motivation to add this new option was because we have a binary that ended up with so many ELF sections (over 65,000) that it broke some existing ELF tools, which couldn't handle so many sections.
Here's our old binary:
```
$ readelf --sections old.elf | head -1
There are 71746 section headers, starting at offset 0x2a246508:
$ readelf --sections old.elf | grep shstrtab
[71742] .shstrtab STRTAB 0000000000000000 2977204c ad44bb 00 0 0 1
```
That's an 11MB+ string table. Here's the new binary using this option:
```
$ readelf --sections new.elf | head -1
There are 43 section headers, starting at offset 0x29143ca8:
$ readelf --sections new.elf | grep shstrtab
[40] .shstrtab STRTAB 0000000000000000 29143acc 0001db 00 0 0 1
```
The whole binary size went down by over 20MB, which is quite significant.
This change adds a new compiler flag that can help reduce the size of
ELF binaries that contain many functions.
By default, when enabling function sections (which is the default for most
targets), the LLVM backend will generate different section names for each
function. For example, a function "func" would generate a section called
".text.func". Normally this is fine because the linker will merge all those
sections into a single one in the binary. However, starting with LLVM 12
(llvm/llvm-project@ee5d1a0), the backend will
also generate unique section names for exception handling, resulting in
thousands of ".gcc_except_table.*" sections ending up in the final binary
because some linkers don't currently merge or strip these EH sections.
This can bloat the ELF headers and string table significantly in
binaries that contain many functions.
The new option is analogous to Clang's -fno-unique-section-names, and
instructs LLVM to generate the same ".text" and ".gcc_except_table"
section for each function, resulting in smaller object files and
potentially a smaller final binary.
This largely involves implementing the options debug-info-for-profiling
and profile-sample-use and forwarding them on to LLVM.
AutoFDO can be used on x86-64 Linux like this:
rustc -O -Cdebug-info-for-profiling main.rs -o main
perf record -b ./main
create_llvm_prof --binary=main --out=code.prof
rustc -O -Cprofile-sample-use=code.prof main.rs -o main2
Now `main2` will have feedback directed optimization applied to it.
The create_llvm_prof tool can be obtained from this github repository:
https://github.com/google/autofdoFixes#64892.
thinLTOResolvePrevailingInModule became thinLTOFinalizeInModule and
gained the ability to propagate noRecurse and noUnwind function
attributes. I ran codegen tests with it both on and off, as the upstream
patch uses it in both modes, and the tests pass both ways. Given that,
it seemed reasonable to go ahead and let the propagation be enabled in
rustc, and see what happens. See https://reviews.llvm.org/D36850 for
more examples of how the new version of the function gets used.
Change ab41eef9aca3 in LLVM split MemorySanitizerPass into
MemorySanitizerPass for functions and ModuleMemorySanitizerPass for
modules. There's a related change for ThreadSanitizerPass, and in here
since we're using a ModulePassManager I only add the module flavor of
the pass on LLVM 14.
r? @nikic cc @nagisa
These API changes appear to have all taken place in
https://reviews.llvm.org/D105007, which moved HWAddressSanitizerPass and
AddressSanitizerPass to only accept their options type as a ctor
argument instead of the sequence of bools etc. This required a couple of
parameter additions, which I made match the default prior to the
mentioned upstream LLVM change.
This patch restores rustc to building (though not quite passing all
tests, I've mailed other patches for those issues) against LLVM HEAD.
PassWrapper: update for LLVM change D102093
In https://reviews.llvm.org/D102093 lots of things stopped taking the
DebugLogging boolean parameter. Mercifully we appear to always set
DebugPassManager to false, so I don't think we're losing anything by not
passing this parameter.
In https://reviews.llvm.org/D102093 lots of things stopped taking the
DebugLogging boolean parameter. Mercifully we appear to always set
DebugPassManager to false, so I don't think we're losing anything by not
passing this parameter.
This changed in 54fb3ca96e261f7107cb1b5778c34cb0e0808be6 - I'm not
entirely sure it's correct that we're leaving config empty, but the one
case in LLVM that looked similar did that.
This works around a design defect in the LLVM 12 pass builder
implementation. In LLVM 13, the PreLink ThinLTO pipeline properly
respects the OptimizerLastEPCallbacks.
This commit modifies the FFI bindings to LLVM required for Split DWARF
support in rustc. In particular:
- `addPassesToEmitFile`'s wrapper, `LLVMRustWriteOutputFile` now takes
a `DwoPath` `const char*`. When disabled, `nullptr` should be provided
which will preserve existing behaviour. When enabled, the path to the
`.dwo` file should be provided.
- `createCompileUnit`'s wrapper, `LLVMRustDIBuilderCreateCompileUnit`
now has two additional arguments, for the `DWOId` and to enable
`SplitDebugInlining`. `DWOId` should always be zero.
- `createTargetMachine`'s wrapper, `LLVMRustCreateTargetMachine` has an
additional argument which should be provided the path to the `.dwo`
when enabled.
Signed-off-by: David Wood <david@davidtw.co>
This commit grepped for LLVM_VERSION_GE, LLVM_VERSION_LT, get_major_version and
min-llvm-version and statically evaluated every expression possible
(and sensible) assuming that the LLVM version is >=9 now
During incremental ThinLTO compilation, we attempt to re-use the
optimized (post-ThinLTO) bitcode file for a module if it is 'safe' to do
so.
Up until now, 'safe' has meant that the set of modules that our current
modules imports from/exports to is unchanged from the previous
compilation session. See PR #67020 and PR #71131 for more details.
However, this turns out be insufficient to guarantee that it's safe
to reuse the post-LTO module (i.e. that optimizing the pre-LTO module
would produce the same result). When LLVM optimizes a module during
ThinLTO, it may look at other information from the 'module index', such
as whether a (non-imported!) global variable is used. If this
information changes between compilation runs, we may end up re-using an
optimized module that (for example) had dead-code elimination run on a
function that is now used by another module.
Fortunately, LLVM implements its own ThinLTO module cache, which is used
when ThinLTO is performed by a linker plugin (e.g. when clang is used to
compile a C proect). Using this cache directly would require extensive
refactoring of our code - but fortunately for us, LLVM provides a
function that does exactly what we need.
The function `llvm::computeLTOCacheKey` is used to compute a SHA-1 hash
from all data that might influence the result of ThinLTO on a module.
In addition to the module imports/exports that we manually track, it
also hashes information about global variables (e.g. their liveness)
which might be used during optimization. By using this function, we
shouldn't have to worry about new LLVM passes breaking our module re-use
behavior.
In LLVM, the output of this function forms part of the filename used to
store the post-ThinLTO module. To keep our current filename structure
intact, this PR just writes out the mapping 'CGU name -> Hash' to a
file. To determine if a post-LTO module should be reused, we compare
hashes from the previous session.
This should unblock PR #75199 - by sheer chance, it seems to have hit
this issue due to the particular CGU partitioning and optimization
decisions that end up getting made.
