Empowering everyone to build reliable and efficient software.
Go to file
Jubilee e1ceadcdfe
Rollup merge of #117458 - kjetilkjeka:embedded-linker, r=petrochenkov
LLVM Bitcode Linker: A self contained linker for nvptx and other targets

This PR introduces a new linker named `llvm-bitcode-linker`. It is a `self-contained` linker that can be used to link programs in `llbc` before optimizing and compiling to native code. It will first be used internally in the Rust compiler to enable tests for the `nvptx64-nvidia-cuda` target as the original `rust-ptx-linker` is deprecated. It will then be provided to users of the `nvptx64-nvidia-cuda` target with the purpose of linking ptx. More targets than nvptx will also be supported eventually.

The PR introduces a new unstable `LinkerFlavor` for the compiler. The compiler will also not be shipped with rustc but most likely instead be shipped in it's own unstable component (a follow up PR will be opened for this). This means that merging this PR should not add any stability guarantees.

When more details of `self-contained` is implemented it will only be possible to use the linker when `-Clink-self-contained=+linker` is passed.

<details>
  <summary>Original Description</summary>

**When this PR was created it was focused a bit differently. The original text is preserved here in case there's some interests in it**

I have experimenting with approaches to replace the ptx-linker and enable the nvptx target tests again. I think it's time to get some feedback on the approach.

### The problem
The only useful linker for the nvptx target is [this crate](https://github.com/denzp/rust-ptx-linker). Since this linker performs linking on llvm bitcode it needs to track the llvm version of rustc and use the same format. It has not been maintained for 3+ years and must be considered abandoned. Over the years rust have upgraded LLVM while the linker has been left to bitrot. It is no longer in a usable state.

Due to the difficulty of keeping the ptx-linker up to date outside of tree the nvptx tests was [disabled a long time ago](f8f9a2869c). It was [previously discussed](https://github.com/rust-lang/rust/pull/96842#issuecomment-1146470177) if adding the ptx-linker to the rust repo would be a possibility. My efforts in doing this stopped at getting an answered if the license would prohibit it from inclusion in the [Rust repo](https://github.com/rust-lang/rust/pull/96842#issuecomment-1148397554). I therefore concluded that a re-write would be necessary.

### The possible solution presented here
The llvm tools know perfectly well how to link and optimize llvm bitcode. Each of them only perform a single task, and are therefore a bit cumbersome to call with the current linker approach rustc takes.

This PR adds a simple tool (current name `embedded-linker`) which can link self contained (often embedded) programs in llvm bitcode before compiling to the target format. Optimization will also be performed if lto is enabled. The rust compiler will make a single invocation to this tool, while the tool will orchestrate the many calls to the llvm tools.

### The questions
 - Is having control over the nvptx linking and therefore also tests worth it to add such tool? or should the tool live outside the rust repo?
 - Is the approach of calling llvm tools acceptable? Or would we want to keep the ptx-linker approach of using the llvm library? The tools seems to provide more simplicity and stability, but more intermediate files are being written. Perhaps there also are some performance penalty for the calling tools approach.
 - What is the process for adding such tool? MCP?
 - Does adding `llvm-link` to the llvm-tool component require any process?
 - Does it require some sort of FCP to remove ptx-linker as the default linker for ptx? Or is it sufficient that using the upstream ptx-linker is broken in its current state. it is possible to use a somewhat patched version of ptx-linker.
</details>
2024-03-11 09:29:32 -07:00
.github ci: add a runner for vanilla LLVM 18 2024-03-07 18:30:04 -08:00
.reuse update license reference 2024-02-16 12:10:49 +01:00
compiler Rollup merge of #117458 - kjetilkjeka:embedded-linker, r=petrochenkov 2024-03-11 09:29:32 -07:00
library Auto merge of #122331 - jhpratt:rollup-cbl8xsy, r=jhpratt 2024-03-11 10:22:10 +00:00
LICENSES Add missing CC-BY-SA-4.0. 2023-11-27 11:03:53 +00:00
src Bootstrap: Add argument for building llvm bitcode linker 2024-03-11 13:35:55 +01:00
tests NVPTX: Enable previously disabled tests 2024-03-11 13:35:58 +01:00
.editorconfig Only use max_line_length = 100 for *.rs 2023-07-10 15:18:36 -07:00
.git-blame-ignore-revs Ignore compiletest test directive migration commits 2024-02-22 18:55:02 +00:00
.gitattributes Rename config.toml.example to config.example.toml 2023-03-11 14:10:00 -08:00
.gitignore don't globally ignore rustc-ice files 2023-09-16 09:44:44 +02:00
.gitmodules Update to LLVM 18 2024-02-13 10:33:40 +01:00
.mailmap correct my mailmap entry 2024-01-21 20:47:26 -05:00
Cargo.lock LLVM Bitcode Linker: Added crate 2024-03-11 13:35:35 +01:00
Cargo.toml LLVM Bitcode Linker: Added crate 2024-03-11 13:35:35 +01:00
CODE_OF_CONDUCT.md Remove the code of conduct; instead link https://www.rust-lang.org/conduct.html 2019-10-05 22:55:19 +02:00
config.example.toml Bootstrap: Add argument for building llvm bitcode linker 2024-03-11 13:35:55 +01:00
configure Ensure ./configure works when configure.py path contains spaces 2024-02-16 18:57:22 +00:00
CONTRIBUTING.md fix: Update CONTRIBUTING.md recommend -> recommended 2023-11-16 23:57:09 +05:30
COPYRIGHT Update COPYRIGHT file 2022-10-30 10:23:14 -04:00
INSTALL.md Revert back to Git-for-Windows for MinGW CI builds 2024-03-07 03:09:29 +00:00
LICENSE-APACHE Remove appendix from LICENCE-APACHE 2019-12-30 14:25:53 +00:00
LICENSE-MIT LICENSE-MIT: Remove inaccurate (misattributed) copyright notice 2017-07-26 16:51:58 -07:00
README.md Update Readme 2024-01-15 13:57:29 -08:00
RELEASES.md Remove duplicate release note 2024-02-09 12:31:32 -05:00
rust-bors.toml Add integration for new bors 2023-09-28 10:43:24 +02:00
rustfmt.toml Remove exception for deleted file from rustfmt.toml 2024-03-08 20:46:33 +00:00
triagebot.toml Rollup merge of #122264 - fee1-dead-contrib:add, r=fee1-dead 2024-03-10 10:58:19 +01:00
x Make x capable of resolving symlinks 2023-10-14 17:53:33 +03:00
x.ps1 use & instead of start-process in x.ps1 2023-12-09 09:46:16 -05:00
x.py Fix recent python linting errors 2023-08-02 04:40:28 -04:00

The Rust Programming Language

Rust Community

This is the main source code repository for Rust. It contains the compiler, standard library, and documentation.

Note: this README is for users rather than contributors. If you wish to contribute to the compiler, you should read CONTRIBUTING.md instead.

Table of Contents

Quick Start

Read "Installation" from The Book.

Installing from Source

If you really want to install from source (though this is not recommended), see INSTALL.md.

Getting Help

See https://www.rust-lang.org/community for a list of chat platforms and forums.

Contributing

See CONTRIBUTING.md.

License

Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.

Trademark

The Rust Foundation owns and protects the Rust and Cargo trademarks and logos (the "Rust Trademarks").

If you want to use these names or brands, please read the media guide.

Third-party logos may be subject to third-party copyrights and trademarks. See Licenses for details.