trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
//! A helper class for dealing with static archives
|
|
|
|
|
2024-07-25 20:17:59 +00:00
|
|
|
use std::ffi::{CStr, CString, c_char, c_void};
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
use std::path::{Path, PathBuf};
|
2024-07-25 20:17:59 +00:00
|
|
|
use std::{io, mem, ptr, str};
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
|
2022-05-28 10:43:51 +00:00
|
|
|
use rustc_codegen_ssa::back::archive::{
|
2024-08-16 19:32:22 +00:00
|
|
|
ArArchiveBuilder, ArchiveBuildFailure, ArchiveBuilder, ArchiveBuilderBuilder,
|
|
|
|
DEFAULT_OBJECT_READER, ObjectReader, UnknownArchiveKind, try_extract_macho_fat_archive,
|
2022-05-28 10:43:51 +00:00
|
|
|
};
|
2020-03-11 11:49:08 +00:00
|
|
|
use rustc_session::Session;
|
2024-07-28 22:13:50 +00:00
|
|
|
|
2019-02-17 18:58:58 +00:00
|
|
|
use crate::llvm::archive_ro::{ArchiveRO, Child};
|
2024-08-16 19:32:22 +00:00
|
|
|
use crate::llvm::{self, ArchiveKind};
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
|
2017-10-08 08:41:12 +00:00
|
|
|
/// Helper for adding many files to an archive.
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
#[must_use = "must call build() to finish building the archive"]
|
2022-05-28 10:43:51 +00:00
|
|
|
pub(crate) struct LlvmArchiveBuilder<'a> {
|
2022-06-14 14:33:48 +00:00
|
|
|
sess: &'a Session,
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
additions: Vec<Addition>,
|
|
|
|
}
|
|
|
|
|
|
|
|
enum Addition {
|
|
|
|
File { path: PathBuf, name_in_archive: String },
|
2018-07-11 10:49:11 +00:00
|
|
|
Archive { path: PathBuf, archive: ArchiveRO, skip: Box<dyn FnMut(&str) -> bool> },
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
}
|
|
|
|
|
2019-07-02 00:28:10 +00:00
|
|
|
impl Addition {
|
|
|
|
fn path(&self) -> &Path {
|
|
|
|
match self {
|
|
|
|
Addition::File { path, .. } | Addition::Archive { path, .. } => path,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-02-25 07:40:18 +00:00
|
|
|
fn is_relevant_child(c: &Child<'_>) -> bool {
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
match c.name() {
|
|
|
|
Some(name) => !name.contains("SYMDEF"),
|
|
|
|
None => false,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2024-02-26 14:37:04 +00:00
|
|
|
impl<'a> ArchiveBuilder for LlvmArchiveBuilder<'a> {
|
2022-07-28 09:07:49 +00:00
|
|
|
fn add_archive(
|
|
|
|
&mut self,
|
|
|
|
archive: &Path,
|
|
|
|
skip: Box<dyn FnMut(&str) -> bool + 'static>,
|
|
|
|
) -> io::Result<()> {
|
2023-01-27 11:46:27 +00:00
|
|
|
let mut archive = archive.to_path_buf();
|
|
|
|
if self.sess.target.llvm_target.contains("-apple-macosx") {
|
2023-11-21 19:07:32 +00:00
|
|
|
if let Some(new_archive) = try_extract_macho_fat_archive(self.sess, &archive)? {
|
2023-01-27 11:46:27 +00:00
|
|
|
archive = new_archive
|
|
|
|
}
|
|
|
|
}
|
|
|
|
let archive_ro = match ArchiveRO::open(&archive) {
|
2021-09-01 12:38:37 +00:00
|
|
|
Ok(ar) => ar,
|
|
|
|
Err(e) => return Err(io::Error::new(io::ErrorKind::Other, e)),
|
|
|
|
};
|
|
|
|
if self.additions.iter().any(|ar| ar.path() == archive) {
|
|
|
|
return Ok(());
|
|
|
|
}
|
|
|
|
self.additions.push(Addition::Archive {
|
2023-01-27 11:46:27 +00:00
|
|
|
path: archive,
|
2021-09-01 12:38:37 +00:00
|
|
|
archive: archive_ro,
|
|
|
|
skip: Box::new(skip),
|
2015-09-25 04:46:33 +00:00
|
|
|
});
|
2021-09-01 12:38:37 +00:00
|
|
|
Ok(())
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/// Adds an arbitrary file to this archive
|
2019-03-30 11:59:40 +00:00
|
|
|
fn add_file(&mut self, file: &Path) {
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
let name = file.file_name().unwrap().to_str().unwrap();
|
|
|
|
self.additions
|
2018-10-06 09:50:00 +00:00
|
|
|
.push(Addition::File { path: file.to_path_buf(), name_in_archive: name.to_owned() });
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
/// Combine the provided files, rlibs, and native libraries into a single
|
|
|
|
/// `Archive`.
|
2022-07-28 09:07:49 +00:00
|
|
|
fn build(mut self: Box<Self>, output: &Path) -> bool {
|
2022-07-28 08:39:19 +00:00
|
|
|
match self.build_with_llvm(output) {
|
2022-06-18 17:55:24 +00:00
|
|
|
Ok(any_members) => any_members,
|
2024-07-16 20:43:40 +00:00
|
|
|
Err(error) => {
|
|
|
|
self.sess.dcx().emit_fatal(ArchiveBuildFailure { path: output.to_owned(), error })
|
|
|
|
}
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
}
|
|
|
|
}
|
2022-07-28 09:07:49 +00:00
|
|
|
}
|
|
|
|
|
2024-07-06 12:26:42 +00:00
|
|
|
pub(crate) struct LlvmArchiveBuilderBuilder;
|
2022-07-28 09:07:49 +00:00
|
|
|
|
|
|
|
impl ArchiveBuilderBuilder for LlvmArchiveBuilderBuilder {
|
2024-02-26 14:37:04 +00:00
|
|
|
fn new_archive_builder<'a>(&self, sess: &'a Session) -> Box<dyn ArchiveBuilder + 'a> {
|
2024-08-10 18:49:36 +00:00
|
|
|
// Keeping LlvmArchiveBuilder around in case of a regression caused by using
|
|
|
|
// ArArchiveBuilder.
|
2024-08-11 10:29:32 +00:00
|
|
|
// FIXME(#128955) remove a couple of months after #128936 gets merged in case
|
|
|
|
// no regression is found.
|
2024-08-10 17:45:39 +00:00
|
|
|
if false {
|
2022-05-28 10:43:51 +00:00
|
|
|
Box::new(LlvmArchiveBuilder { sess, additions: Vec::new() })
|
|
|
|
} else {
|
2024-04-16 18:31:43 +00:00
|
|
|
Box::new(ArArchiveBuilder::new(sess, &LLVM_OBJECT_READER))
|
2022-05-28 10:43:51 +00:00
|
|
|
}
|
2022-07-28 09:07:49 +00:00
|
|
|
}
|
2019-03-30 11:59:40 +00:00
|
|
|
}
|
|
|
|
|
2022-11-26 13:55:18 +00:00
|
|
|
// The object crate doesn't know how to get symbols for LLVM bitcode and COFF bigobj files.
|
|
|
|
// As such we need to use LLVM for them.
|
2024-04-16 18:31:43 +00:00
|
|
|
|
|
|
|
static LLVM_OBJECT_READER: ObjectReader = ObjectReader {
|
|
|
|
get_symbols: get_llvm_object_symbols,
|
|
|
|
is_64_bit_object_file: llvm_is_64_bit_object_file,
|
|
|
|
is_ec_object_file: llvm_is_ec_object_file,
|
|
|
|
get_xcoff_member_alignment: DEFAULT_OBJECT_READER.get_xcoff_member_alignment,
|
|
|
|
};
|
|
|
|
|
2022-05-28 10:43:51 +00:00
|
|
|
#[deny(unsafe_op_in_unsafe_fn)]
|
|
|
|
fn get_llvm_object_symbols(
|
|
|
|
buf: &[u8],
|
|
|
|
f: &mut dyn FnMut(&[u8]) -> io::Result<()>,
|
|
|
|
) -> io::Result<bool> {
|
2024-04-16 18:31:43 +00:00
|
|
|
let mut state = Box::new(f);
|
2022-11-26 13:55:18 +00:00
|
|
|
|
2024-04-16 18:31:43 +00:00
|
|
|
let err = unsafe {
|
|
|
|
llvm::LLVMRustGetSymbols(
|
|
|
|
buf.as_ptr(),
|
|
|
|
buf.len(),
|
|
|
|
std::ptr::addr_of_mut!(*state) as *mut c_void,
|
|
|
|
callback,
|
|
|
|
error_callback,
|
|
|
|
)
|
|
|
|
};
|
2022-05-28 10:43:51 +00:00
|
|
|
|
2024-04-16 18:31:43 +00:00
|
|
|
if err.is_null() {
|
|
|
|
return Ok(true);
|
|
|
|
} else {
|
|
|
|
return Err(unsafe { *Box::from_raw(err as *mut io::Error) });
|
|
|
|
}
|
2022-05-28 10:43:51 +00:00
|
|
|
|
2024-04-16 18:31:43 +00:00
|
|
|
unsafe extern "C" fn callback(state: *mut c_void, symbol_name: *const c_char) -> *mut c_void {
|
|
|
|
let f = unsafe { &mut *(state as *mut &mut dyn FnMut(&[u8]) -> io::Result<()>) };
|
|
|
|
match f(unsafe { CStr::from_ptr(symbol_name) }.to_bytes()) {
|
|
|
|
Ok(()) => std::ptr::null_mut(),
|
|
|
|
Err(err) => Box::into_raw(Box::new(err)) as *mut c_void,
|
2022-05-28 10:43:51 +00:00
|
|
|
}
|
2024-04-16 18:31:43 +00:00
|
|
|
}
|
2022-05-28 10:43:51 +00:00
|
|
|
|
2024-04-16 18:31:43 +00:00
|
|
|
unsafe extern "C" fn error_callback(error: *const c_char) -> *mut c_void {
|
|
|
|
let error = unsafe { CStr::from_ptr(error) };
|
|
|
|
Box::into_raw(Box::new(io::Error::new(
|
|
|
|
io::ErrorKind::Other,
|
|
|
|
format!("LLVM error: {}", error.to_string_lossy()),
|
|
|
|
))) as *mut c_void
|
2022-05-28 10:43:51 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2024-04-16 18:31:43 +00:00
|
|
|
fn llvm_is_64_bit_object_file(buf: &[u8]) -> bool {
|
|
|
|
unsafe { llvm::LLVMRustIs64BitSymbolicFile(buf.as_ptr(), buf.len()) }
|
|
|
|
}
|
|
|
|
|
|
|
|
fn llvm_is_ec_object_file(buf: &[u8]) -> bool {
|
|
|
|
unsafe { llvm::LLVMRustIsECObject(buf.as_ptr(), buf.len()) }
|
|
|
|
}
|
|
|
|
|
2019-03-30 11:59:40 +00:00
|
|
|
impl<'a> LlvmArchiveBuilder<'a> {
|
2022-07-28 08:39:19 +00:00
|
|
|
fn build_with_llvm(&mut self, output: &Path) -> io::Result<bool> {
|
2022-06-14 14:33:48 +00:00
|
|
|
let kind = &*self.sess.target.archive_format;
|
2022-08-26 17:42:29 +00:00
|
|
|
let kind = kind
|
|
|
|
.parse::<ArchiveKind>()
|
|
|
|
.map_err(|_| kind)
|
2023-12-18 11:21:37 +00:00
|
|
|
.unwrap_or_else(|kind| self.sess.dcx().emit_fatal(UnknownArchiveKind { kind }));
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
|
2019-06-30 18:30:01 +00:00
|
|
|
let mut additions = mem::take(&mut self.additions);
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
let mut strings = Vec::new();
|
|
|
|
let mut members = Vec::new();
|
2018-07-17 13:00:10 +00:00
|
|
|
|
2022-07-28 08:39:19 +00:00
|
|
|
let dst = CString::new(output.to_str().unwrap())?;
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
|
|
|
|
unsafe {
|
2018-07-17 13:00:10 +00:00
|
|
|
for addition in &mut additions {
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
match addition {
|
|
|
|
Addition::File { path, name_in_archive } => {
|
2016-03-23 03:01:37 +00:00
|
|
|
let path = CString::new(path.to_str().unwrap())?;
|
2023-09-01 02:22:22 +00:00
|
|
|
let name = CString::new(name_in_archive.as_bytes())?;
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
members.push(llvm::LLVMRustArchiveMemberNew(
|
|
|
|
path.as_ptr(),
|
|
|
|
name.as_ptr(),
|
2018-06-27 10:12:47 +00:00
|
|
|
None,
|
|
|
|
));
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
strings.push(path);
|
|
|
|
strings.push(name);
|
|
|
|
}
|
2019-07-02 00:28:10 +00:00
|
|
|
Addition::Archive { archive, skip, .. } => {
|
2015-10-23 05:07:19 +00:00
|
|
|
for child in archive.iter() {
|
2016-03-23 03:01:37 +00:00
|
|
|
let child = child.map_err(string_to_io_error)?;
|
2015-10-23 05:07:19 +00:00
|
|
|
if !is_relevant_child(&child) {
|
|
|
|
continue;
|
|
|
|
}
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
let child_name = child.name().unwrap();
|
2015-10-23 05:07:19 +00:00
|
|
|
if skip(child_name) {
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
// It appears that LLVM's archive writer is a little
|
|
|
|
// buggy if the name we pass down isn't just the
|
|
|
|
// filename component, so chop that off here and
|
|
|
|
// pass it in.
|
|
|
|
//
|
|
|
|
// See LLVM bug 25877 for more info.
|
|
|
|
let child_name =
|
|
|
|
Path::new(child_name).file_name().unwrap().to_str().unwrap();
|
2016-03-23 03:01:37 +00:00
|
|
|
let name = CString::new(child_name)?;
|
2015-09-03 06:49:50 +00:00
|
|
|
let m = llvm::LLVMRustArchiveMemberNew(
|
|
|
|
ptr::null(),
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
name.as_ptr(),
|
2018-07-17 12:02:11 +00:00
|
|
|
Some(child.raw),
|
|
|
|
);
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
members.push(m);
|
|
|
|
strings.push(name);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
let r = llvm::LLVMRustWriteArchive(
|
|
|
|
dst.as_ptr(),
|
|
|
|
members.len() as libc::size_t,
|
2018-07-17 13:00:10 +00:00
|
|
|
members.as_ptr() as *const &_,
|
2022-02-10 17:27:18 +00:00
|
|
|
true,
|
2015-07-16 07:11:09 +00:00
|
|
|
kind,
|
2024-03-27 17:49:21 +00:00
|
|
|
self.sess.target.arch == "arm64ec",
|
2015-07-16 07:11:09 +00:00
|
|
|
);
|
2016-08-01 21:16:16 +00:00
|
|
|
let ret = if r.into_result().is_err() {
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
let err = llvm::LLVMRustGetLastError();
|
|
|
|
let msg = if err.is_null() {
|
2018-10-06 09:42:14 +00:00
|
|
|
"failed to write archive".into()
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
} else {
|
|
|
|
String::from_utf8_lossy(CStr::from_ptr(err).to_bytes())
|
|
|
|
};
|
|
|
|
Err(io::Error::new(io::ErrorKind::Other, msg))
|
|
|
|
} else {
|
2022-06-18 17:55:24 +00:00
|
|
|
Ok(!members.is_empty())
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
};
|
|
|
|
for member in members {
|
|
|
|
llvm::LLVMRustArchiveMemberFree(member);
|
|
|
|
}
|
2018-10-06 09:46:46 +00:00
|
|
|
ret
|
trans: Use LLVM's writeArchive to modify archives
We have previously always relied upon an external tool, `ar`, to modify archives
that the compiler produces (staticlibs, rlibs, etc). This approach, however, has
a number of downsides:
* Spawning a process is relatively expensive for small compilations
* Encoding arguments across process boundaries often incurs unnecessary overhead
or lossiness. For example `ar` has a tough time dealing with files that have
the same name in archives, and the compiler copies many files around to ensure
they can be passed to `ar` in a reasonable fashion.
* Most `ar` programs found do **not** have the ability to target arbitrary
platforms, so this is an extra tool which needs to be found/specified when
cross compiling.
The LLVM project has had a tool called `llvm-ar` for quite some time now, but it
wasn't available in the standard LLVM libraries (it was just a standalone
program). Recently, however, in LLVM 3.7, this functionality has been moved to a
library and is now accessible by consumers of LLVM via the `writeArchive`
function.
This commit migrates our archive bindings to no longer invoke `ar` by default
but instead make a library call to LLVM to do various operations. This solves
all of the downsides listed above:
* Archive management is now much faster, for example creating a "hello world"
staticlib is now 6x faster (50ms => 8ms). Linking dynamic libraries also
recently started requiring modification of rlibs, and linking a hello world
dynamic library is now 2x faster.
* The compiler is now one step closer to "hassle free" cross compilation because
no external tool is needed for managing archives, LLVM does the right thing!
This commit does not remove support for calling a system `ar` utility currently.
We will continue to maintain compatibility with LLVM 3.5 and 3.6 looking forward
(so the system LLVM can be used wherever possible), and in these cases we must
shell out to a system utility. All nightly builds of Rust, however, will stop
needing a system `ar`.
2015-07-09 07:14:20 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2015-10-23 05:07:19 +00:00
|
|
|
|
2015-10-24 01:18:44 +00:00
|
|
|
fn string_to_io_error(s: String) -> io::Error {
|
2023-07-25 21:04:01 +00:00
|
|
|
io::Error::new(io::ErrorKind::Other, format!("bad archive: {s}"))
|
2015-10-23 05:07:19 +00:00
|
|
|
}
|