2017-08-26 03:16:51 +00:00
|
|
|
//! A thin wrapper around `Command` in the standard library which allows us to
|
|
|
|
//! read the arguments that are built up.
|
|
|
|
|
|
|
|
use std::ffi::{OsStr, OsString};
|
2017-10-30 17:42:21 +00:00
|
|
|
use std::process::{self, Output};
|
|
|
|
use std::{fmt, io, mem};
|
2017-08-26 03:16:51 +00:00
|
|
|
|
2017-12-08 19:18:21 +00:00
|
|
|
use rustc_target::spec::LldFlavor;
|
rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
2017-08-27 01:30:12 +00:00
|
|
|
|
2018-01-16 23:30:57 +00:00
|
|
|
#[derive(Clone)]
|
2017-08-26 03:16:51 +00:00
|
|
|
pub struct Command {
|
2018-01-16 23:30:57 +00:00
|
|
|
program: Program,
|
2017-08-26 03:16:51 +00:00
|
|
|
args: Vec<OsString>,
|
|
|
|
env: Vec<(OsString, OsString)>,
|
2019-09-12 10:47:17 +00:00
|
|
|
env_remove: Vec<OsString>,
|
2017-08-26 03:16:51 +00:00
|
|
|
}
|
|
|
|
|
2018-01-16 23:30:57 +00:00
|
|
|
#[derive(Clone)]
|
|
|
|
enum Program {
|
|
|
|
Normal(OsString),
|
|
|
|
CmdBatScript(OsString),
|
rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
2017-08-27 01:30:12 +00:00
|
|
|
Lld(OsString, LldFlavor),
|
2018-01-16 23:30:57 +00:00
|
|
|
}
|
|
|
|
|
2017-08-26 03:16:51 +00:00
|
|
|
impl Command {
|
|
|
|
pub fn new<P: AsRef<OsStr>>(program: P) -> Command {
|
2018-01-16 23:30:57 +00:00
|
|
|
Command::_new(Program::Normal(program.as_ref().to_owned()))
|
|
|
|
}
|
|
|
|
|
|
|
|
pub fn bat_script<P: AsRef<OsStr>>(program: P) -> Command {
|
|
|
|
Command::_new(Program::CmdBatScript(program.as_ref().to_owned()))
|
2017-08-26 03:16:51 +00:00
|
|
|
}
|
|
|
|
|
rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
2017-08-27 01:30:12 +00:00
|
|
|
pub fn lld<P: AsRef<OsStr>>(program: P, flavor: LldFlavor) -> Command {
|
|
|
|
Command::_new(Program::Lld(program.as_ref().to_owned(), flavor))
|
|
|
|
}
|
|
|
|
|
2018-01-16 23:30:57 +00:00
|
|
|
fn _new(program: Program) -> Command {
|
2019-09-12 10:47:17 +00:00
|
|
|
Command { program, args: Vec::new(), env: Vec::new(), env_remove: Vec::new() }
|
2017-08-26 03:16:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
pub fn arg<P: AsRef<OsStr>>(&mut self, arg: P) -> &mut Command {
|
|
|
|
self._arg(arg.as_ref());
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
|
|
|
pub fn args<I>(&mut self, args: I) -> &mut Command
|
2019-07-31 19:00:35 +00:00
|
|
|
where
|
|
|
|
I: IntoIterator<Item: AsRef<OsStr>>,
|
2017-08-26 03:16:51 +00:00
|
|
|
{
|
|
|
|
for arg in args {
|
|
|
|
self._arg(arg.as_ref());
|
|
|
|
}
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
|
|
|
fn _arg(&mut self, arg: &OsStr) {
|
|
|
|
self.args.push(arg.to_owned());
|
|
|
|
}
|
|
|
|
|
|
|
|
pub fn env<K, V>(&mut self, key: K, value: V) -> &mut Command
|
|
|
|
where
|
|
|
|
K: AsRef<OsStr>,
|
|
|
|
V: AsRef<OsStr>,
|
|
|
|
{
|
|
|
|
self._env(key.as_ref(), value.as_ref());
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
|
|
|
fn _env(&mut self, key: &OsStr, value: &OsStr) {
|
|
|
|
self.env.push((key.to_owned(), value.to_owned()));
|
|
|
|
}
|
|
|
|
|
2019-09-12 10:47:17 +00:00
|
|
|
pub fn env_remove<K>(&mut self, key: K) -> &mut Command
|
|
|
|
where
|
|
|
|
K: AsRef<OsStr>,
|
|
|
|
{
|
|
|
|
self._env_remove(key.as_ref());
|
|
|
|
self
|
|
|
|
}
|
|
|
|
|
|
|
|
fn _env_remove(&mut self, key: &OsStr) {
|
|
|
|
self.env_remove.push(key.to_owned());
|
|
|
|
}
|
|
|
|
|
2017-08-26 03:16:51 +00:00
|
|
|
pub fn output(&mut self) -> io::Result<Output> {
|
|
|
|
self.command().output()
|
|
|
|
}
|
|
|
|
|
|
|
|
pub fn command(&self) -> process::Command {
|
2018-01-16 23:30:57 +00:00
|
|
|
let mut ret = match self.program {
|
|
|
|
Program::Normal(ref p) => process::Command::new(p),
|
|
|
|
Program::CmdBatScript(ref p) => {
|
|
|
|
let mut c = process::Command::new("cmd");
|
|
|
|
c.arg("/c").arg(p);
|
|
|
|
c
|
|
|
|
}
|
rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
2017-08-27 01:30:12 +00:00
|
|
|
Program::Lld(ref p, flavor) => {
|
|
|
|
let mut c = process::Command::new(p);
|
2022-05-24 20:29:15 +00:00
|
|
|
c.arg("-flavor").arg(flavor.as_str());
|
rust: Import LLD for linking wasm objects
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
2017-08-27 01:30:12 +00:00
|
|
|
c
|
|
|
|
}
|
2018-01-16 23:30:57 +00:00
|
|
|
};
|
2017-08-26 03:16:51 +00:00
|
|
|
ret.args(&self.args);
|
|
|
|
ret.envs(self.env.clone());
|
2019-09-12 10:47:17 +00:00
|
|
|
for k in &self.env_remove {
|
|
|
|
ret.env_remove(k);
|
|
|
|
}
|
2020-03-20 14:03:11 +00:00
|
|
|
ret
|
2017-08-26 03:16:51 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
// extensions
|
|
|
|
|
2018-02-11 15:50:18 +00:00
|
|
|
pub fn get_args(&self) -> &[OsString] {
|
|
|
|
&self.args
|
|
|
|
}
|
|
|
|
|
2018-01-16 23:30:57 +00:00
|
|
|
pub fn take_args(&mut self) -> Vec<OsString> {
|
2019-06-30 18:30:01 +00:00
|
|
|
mem::take(&mut self.args)
|
2017-08-26 03:16:51 +00:00
|
|
|
}
|
|
|
|
|
2018-01-16 23:30:57 +00:00
|
|
|
/// Returns a `true` if we're pretty sure that this'll blow OS spawn limits,
|
|
|
|
/// or `false` if we should attempt to spawn and see what the OS says.
|
|
|
|
pub fn very_likely_to_exceed_some_spawn_limit(&self) -> bool {
|
|
|
|
// We mostly only care about Windows in this method, on Unix the limits
|
|
|
|
// can be gargantuan anyway so we're pretty unlikely to hit them
|
|
|
|
if cfg!(unix) {
|
|
|
|
return false;
|
|
|
|
}
|
2017-08-26 03:16:51 +00:00
|
|
|
|
2018-03-13 14:54:03 +00:00
|
|
|
// Right now LLD doesn't support the `@` syntax of passing an argument
|
|
|
|
// through files, so regardless of the platform we try to go to the OS
|
|
|
|
// on this one.
|
|
|
|
if let Program::Lld(..) = self.program {
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2018-01-16 23:30:57 +00:00
|
|
|
// Ok so on Windows to spawn a process is 32,768 characters in its
|
|
|
|
// command line [1]. Unfortunately we don't actually have access to that
|
|
|
|
// as it's calculated just before spawning. Instead we perform a
|
|
|
|
// poor-man's guess as to how long our command line will be. We're
|
|
|
|
// assuming here that we don't have to escape every character...
|
|
|
|
//
|
|
|
|
// Turns out though that `cmd.exe` has even smaller limits, 8192
|
|
|
|
// characters [2]. Linkers can often be batch scripts (for example
|
|
|
|
// Emscripten, Gecko's current build system) which means that we're
|
|
|
|
// running through batch scripts. These linkers often just forward
|
|
|
|
// arguments elsewhere (and maybe tack on more), so if we blow 8192
|
|
|
|
// bytes we'll typically cause them to blow as well.
|
|
|
|
//
|
|
|
|
// Basically as a result just perform an inflated estimate of what our
|
|
|
|
// command line will look like and test if it's > 8192 (we actually
|
|
|
|
// test against 6k to artificially inflate our estimate). If all else
|
|
|
|
// fails we'll fall back to the normal unix logic of testing the OS
|
|
|
|
// error code if we fail to spawn and automatically re-spawning the
|
|
|
|
// linker with smaller arguments.
|
|
|
|
//
|
2019-12-25 15:35:54 +00:00
|
|
|
// [1]: https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa
|
|
|
|
// [2]: https://devblogs.microsoft.com/oldnewthing/?p=41553
|
2018-01-16 23:30:57 +00:00
|
|
|
|
|
|
|
let estimated_command_line_len = self.args.iter().map(|a| a.len()).sum::<usize>();
|
|
|
|
estimated_command_line_len > 1024 * 6
|
2017-08-26 03:16:51 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
impl fmt::Debug for Command {
|
2019-02-25 07:52:46 +00:00
|
|
|
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
2017-08-26 03:16:51 +00:00
|
|
|
self.command().fmt(f)
|
|
|
|
}
|
|
|
|
}
|