rust/src/librustc_codegen_llvm/lib.rs

390 lines
11 KiB
Rust
Raw Normal View History

// Copyright 2012-2013 The Rust Project Developers. See the COPYRIGHT
// file at the top-level directory of this distribution and at
// http://rust-lang.org/COPYRIGHT.
//
// Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or
// http://www.apache.org/licenses/LICENSE-2.0> or the MIT license
// <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your
// option. This file may not be copied, modified, or distributed
// except according to those terms.
//! The Rust compiler.
//!
//! # Note
//!
//! This API is completely unstable and subject to change.
#![doc(html_logo_url = "https://www.rust-lang.org/logos/rust-logo-128x128-blk-v2.png",
html_favicon_url = "https://doc.rust-lang.org/favicon.ico",
html_root_url = "https://doc.rust-lang.org/nightly/")]
#![feature(box_patterns)]
#![feature(box_syntax)]
2015-09-27 20:20:49 +00:00
#![feature(custom_attribute)]
#![feature(fs_read_write)]
2015-09-27 20:20:49 +00:00
#![allow(unused_attributes)]
#![feature(libc)]
#![feature(quote)]
#![feature(range_contains)]
#![feature(rustc_diagnostic_macros)]
#![feature(slice_sort_by_cached_key)]
#![feature(optin_builtin_traits)]
use rustc::dep_graph::WorkProduct;
use syntax_pos::symbol::Symbol;
#[macro_use] extern crate bitflags;
extern crate flate2;
extern crate libc;
#[macro_use] extern crate rustc;
std: Add a new wasm32-unknown-unknown target This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld. Notable features of this target include: * There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set. * There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything. * Very small wasm modules can be generated directly from Rust code using this target. * Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc). * Naturally, any `#[no_std]` crate should be 100% compatible with this new target. This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker. This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready". --- Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is still getting LLVM bugs fixed to get that working and will take some time. Relatively simple programs all seem to work though! --- It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is: cargo install --git https://github.com/alexcrichton/wasm-gc wasm-gc foo.wasm bar.wasm And then `bar.wasm` should be the smallest we can get it! --- In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
2017-10-23 03:01:00 +00:00
extern crate jobserver;
extern crate num_cpus;
2017-10-25 13:39:54 +00:00
extern crate rustc_mir;
extern crate rustc_allocator;
extern crate rustc_apfloat;
extern crate rustc_target;
2018-03-03 05:17:06 +00:00
#[macro_use] extern crate rustc_data_structures;
std: Add a new wasm32-unknown-unknown target This commit adds a new target to the compiler: wasm32-unknown-unknown. This target is a reimagining of what it looks like to generate WebAssembly code from Rust. Instead of using Emscripten which can bring with it a weighty runtime this instead is a target which uses only the LLVM backend for WebAssembly and a "custom linker" for now which will hopefully one day be direct calls to lld. Notable features of this target include: * There is zero runtime footprint. The target assumes nothing exists other than the wasm32 instruction set. * There is zero toolchain footprint beyond adding the target. No custom linker is needed, rustc contains everything. * Very small wasm modules can be generated directly from Rust code using this target. * Most of the standard library is stubbed out to return an error, but anything related to allocation works (aka `HashMap`, `Vec`, etc). * Naturally, any `#[no_std]` crate should be 100% compatible with this new target. This target is currently somewhat janky due to how linking works. The "linking" is currently unconditional whole program LTO (aka LLVM is being used as a linker). Naturally that means compiling programs is pretty slow! Eventually though this target should have a linker. This target is also intended to be quite experimental. I'm hoping that this can act as a catalyst for further experimentation in Rust with WebAssembly. Breaking changes are very likely to land to this target, so it's not recommended to rely on it in any critical capacity yet. We'll let you know when it's "production ready". --- Currently testing-wise this target is looking pretty good but isn't complete. I've got almost the entire `run-pass` test suite working with this target (lots of tests ignored, but many passing as well). The `core` test suite is still getting LLVM bugs fixed to get that working and will take some time. Relatively simple programs all seem to work though! --- It's worth nothing that you may not immediately see the "smallest possible wasm module" for the input you feed to rustc. For various reasons it's very difficult to get rid of the final "bloat" in vanilla rustc (again, a real linker should fix all this). For now what you'll have to do is: cargo install --git https://github.com/alexcrichton/wasm-gc wasm-gc foo.wasm bar.wasm And then `bar.wasm` should be the smallest we can get it! --- In any case for now I'd love feedback on this, particularly on the various integration points if you've got better ideas of how to approach them!
2017-10-23 03:01:00 +00:00
extern crate rustc_demangle;
extern crate rustc_incremental;
2017-08-19 00:09:55 +00:00
extern crate rustc_llvm as llvm;
extern crate rustc_platform_intrinsics as intrinsics;
2018-05-08 13:10:16 +00:00
extern crate rustc_codegen_utils;
#[macro_use] extern crate log;
#[macro_use] extern crate syntax;
extern crate syntax_pos;
extern crate rustc_errors as errors;
extern crate serialize;
extern crate cc; // Used to locate MSVC
extern crate tempfile;
use back::bytecode::RLIB_BYTECODE_EXTENSION;
pub use llvm_util::target_features;
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
use std::any::Any;
use std::path::PathBuf;
use std::sync::mpsc;
2018-02-27 16:11:14 +00:00
use rustc_data_structures::sync::Lrc;
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
use rustc::dep_graph::DepGraph;
use rustc::hir::def_id::CrateNum;
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
use rustc::middle::cstore::MetadataLoader;
use rustc::middle::cstore::{NativeLibrary, CrateSource, LibSource};
use rustc::middle::lang_items::LangItem;
use rustc::session::{Session, CompileIncomplete};
use rustc::session::config::{OutputFilenames, OutputType, PrintRequest};
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
use rustc::ty::{self, TyCtxt};
2018-06-23 11:52:52 +00:00
use rustc::util::time_graph;
use rustc::util::nodemap::{FxHashSet, FxHashMap};
use rustc_mir::monomorphize;
2018-05-08 13:10:16 +00:00
use rustc_codegen_utils::codegen_backend::CodegenBackend;
2017-10-19 09:01:31 +00:00
mod diagnostics;
2017-12-27 18:03:48 +00:00
mod back {
2018-05-08 13:10:16 +00:00
pub use rustc_codegen_utils::symbol_names;
mod archive;
pub mod bytecode;
mod command;
2017-12-27 18:03:48 +00:00
pub mod linker;
pub mod link;
mod lto;
2017-12-27 18:03:48 +00:00
pub mod symbol_export;
pub mod write;
2017-08-19 00:09:55 +00:00
mod rpath;
Upgrade to LLVM's master branch (LLVM 7) This commit upgrades the main LLVM submodule to LLVM's current master branch. The LLD submodule is updated in tandem as well as compiler-builtins. Along the way support was also added for LLVM 7's new features. This primarily includes the support for custom section concatenation natively in LLD so we now add wasm custom sections in LLVM IR rather than having custom support in rustc itself for doing so. Some other miscellaneous changes are: * We now pass `--gc-sections` to `wasm-ld` * The optimization level is now passed to `wasm-ld` * A `--stack-first` option is passed to LLD to have stack overflow always cause a trap instead of corrupting static data * The wasm target for LLVM switched to `wasm32-unknown-unknown`. * The syntax for aligned pointers has changed in LLVM IR and tests are updated to reflect this. * The `thumbv6m-none-eabi` target is disabled due to an [LLVM bug][llbug] Nowadays we've been mostly only upgrading whenever there's a major release of LLVM but enough changes have been happening on the wasm target that there's been growing motivation for quite some time now to upgrade out version of LLD. To upgrade LLD, however, we need to upgrade LLVM to avoid needing to build yet another version of LLVM on the builders. The revision of LLVM in use here is arbitrarily chosen. We will likely need to continue to update it over time if and when we discover bugs. Once LLVM 7 is fully released we can switch to that channel as well. [llbug]: https://bugs.llvm.org/show_bug.cgi?id=37382
2018-06-01 17:20:00 +00:00
pub mod wasm;
}
mod abi;
mod allocator;
mod asm;
mod attributes;
mod base;
mod builder;
mod callee;
mod common;
mod consts;
mod context;
mod debuginfo;
mod declare;
mod glue;
mod intrinsic;
mod llvm_util;
mod metadata;
mod meth;
mod mir;
2018-05-08 13:10:16 +00:00
mod mono_item;
mod type_;
mod type_of;
mod value;
2018-05-08 13:10:16 +00:00
pub struct LlvmCodegenBackend(());
2017-09-16 15:27:29 +00:00
2018-05-08 13:10:16 +00:00
impl !Send for LlvmCodegenBackend {} // Llvm is on a per-thread basis
impl !Sync for LlvmCodegenBackend {}
2018-05-08 13:10:16 +00:00
impl LlvmCodegenBackend {
pub fn new() -> Box<dyn CodegenBackend> {
2018-05-08 13:10:16 +00:00
box LlvmCodegenBackend(())
2017-09-16 15:27:29 +00:00
}
}
2018-05-08 13:10:16 +00:00
impl CodegenBackend for LlvmCodegenBackend {
fn init(&self, sess: &Session) {
llvm_util::init(sess); // Make sure llvm is inited
}
fn print(&self, req: PrintRequest, sess: &Session) {
match req {
PrintRequest::RelocationModels => {
println!("Available relocation models:");
for &(name, _) in back::write::RELOC_MODEL_ARGS.iter() {
println!(" {}", name);
}
println!("");
}
PrintRequest::CodeModels => {
println!("Available code models:");
for &(name, _) in back::write::CODE_GEN_MODEL_ARGS.iter(){
println!(" {}", name);
}
println!("");
}
PrintRequest::TlsModels => {
println!("Available TLS models:");
for &(name, _) in back::write::TLS_MODEL_ARGS.iter(){
println!(" {}", name);
}
println!("");
}
req => llvm_util::print(req, sess),
}
}
2017-09-16 15:27:29 +00:00
fn print_passes(&self) {
llvm_util::print_passes();
}
fn print_version(&self) {
llvm_util::print_version();
}
fn diagnostics(&self) -> &[(&'static str, &'static str)] {
&DIAGNOSTICS
}
fn target_features(&self, sess: &Session) -> Vec<Symbol> {
target_features(sess)
}
fn metadata_loader(&self) -> Box<dyn MetadataLoader + Sync> {
2017-09-16 15:27:29 +00:00
box metadata::LlvmMetadataLoader
}
2018-06-13 13:44:43 +00:00
fn provide(&self, providers: &mut ty::query::Providers) {
back::symbol_names::provide(providers);
back::symbol_export::provide(providers);
base::provide(providers);
attributes::provide(providers);
2017-09-23 12:46:15 +00:00
}
2018-06-13 13:44:43 +00:00
fn provide_extern(&self, providers: &mut ty::query::Providers) {
back::symbol_export::provide_extern(providers);
rustc: Add a `#[wasm_import_module]` attribute This commit adds a new attribute to the Rust compiler specific to the wasm target (and no other targets). The `#[wasm_import_module]` attribute is used to specify the module that a name is imported from, and is used like so: #[wasm_import_module = "./foo.js"] extern { fn some_js_function(); } Here the import of the symbol `some_js_function` is tagged with the `./foo.js` module in the wasm output file. Wasm-the-format includes two fields on all imports, a module and a field. The field is the symbol name (`some_js_function` above) and the module has historically unconditionally been `"env"`. I'm not sure if this `"env"` convention has asm.js or LLVM roots, but regardless we'd like the ability to configure it! The proposed ES module integration with wasm (aka a wasm module is "just another ES module") requires that the import module of wasm imports is interpreted as an ES module import, meaning that you'll need to encode paths, NPM packages, etc. As a result, we'll need this to be something other than `"env"`! Unfortunately neither our version of LLVM nor LLD supports custom import modules (aka anything not `"env"`). My hope is that by the time LLVM 7 is released both will have support, but in the meantime this commit adds some primitive encoding/decoding of wasm files to the compiler. This way rustc postprocesses the wasm module that LLVM emits to ensure it's got all the imports we'd like to have in it. Eventually I'd ideally like to unconditionally require this attribute to be placed on all `extern { ... }` blocks. For now though it seemed prudent to add it as an unstable attribute, so for now it's not required (as that'd force usage of a feature gate). Hopefully it doesn't take too long to "stabilize" this! cc rust-lang-nursery/rust-wasm#29
2018-02-10 22:28:17 +00:00
base::provide_extern(providers);
attributes::provide_extern(providers);
2017-09-16 15:27:29 +00:00
}
2018-05-08 13:10:16 +00:00
fn codegen_crate<'a, 'tcx>(
&self,
2017-09-16 15:27:29 +00:00
tcx: TyCtxt<'a, 'tcx, 'tcx>,
rx: mpsc::Receiver<Box<dyn Any + Send>>
) -> Box<dyn Any> {
2018-05-08 13:10:16 +00:00
box base::codegen_crate(tcx, rx)
2017-09-16 15:27:29 +00:00
}
2018-05-08 13:10:16 +00:00
fn join_codegen_and_link(
&self,
ongoing_codegen: Box<dyn Any>,
2017-09-16 15:27:29 +00:00
sess: &Session,
dep_graph: &DepGraph,
outputs: &OutputFilenames,
) -> Result<(), CompileIncomplete>{
use rustc::util::common::time;
2018-05-08 13:10:16 +00:00
let (ongoing_codegen, work_products) =
ongoing_codegen.downcast::<::back::write::OngoingCodegen>()
.expect("Expected LlvmCodegenBackend's OngoingCodegen, found Box<Any>")
.join(sess);
if sess.opts.debugging_opts.incremental_info {
2018-05-08 13:10:16 +00:00
back::write::dump_incremental_data(&ongoing_codegen);
}
2017-09-16 15:27:29 +00:00
2017-12-03 13:21:23 +00:00
time(sess,
"serialize work products",
2018-05-09 15:16:08 +00:00
move || rustc_incremental::save_work_product_index(sess, &dep_graph, work_products));
sess.compile_status()?;
if !sess.opts.output_types.keys().any(|&i| i == OutputType::Exe ||
i == OutputType::Metadata) {
return Ok(());
}
// Run the linker on any artifacts that resulted from the LLVM run.
// This should produce either a finished executable or library.
2017-12-03 13:21:23 +00:00
time(sess, "linking", || {
2018-05-08 13:10:16 +00:00
back::link::link_binary(sess, &ongoing_codegen,
outputs, &ongoing_codegen.crate_name.as_str());
});
// Now that we won't touch anything in the incremental compilation directory
// any more, we can finalize it (which involves renaming it)
2018-05-08 13:10:16 +00:00
rustc_incremental::finalize_session_directory(sess, ongoing_codegen.link.crate_hash);
2017-09-16 15:27:29 +00:00
Ok(())
2017-09-16 15:27:29 +00:00
}
}
2018-05-08 13:10:16 +00:00
/// This is the entrypoint for a hot plugged rustc_codegen_llvm
2018-01-04 11:40:11 +00:00
#[no_mangle]
pub fn __rustc_codegen_backend() -> Box<dyn CodegenBackend> {
2018-05-08 13:10:16 +00:00
LlvmCodegenBackend::new()
2018-01-04 11:40:11 +00:00
}
2018-01-01 11:17:07 +00:00
2018-05-08 13:10:16 +00:00
struct ModuleCodegen {
/// The name of the module. When the crate may be saved between
/// compilations, incremental compilation requires that name be
/// unique amongst **all** crates. Therefore, it should contain
/// something unique to this crate (e.g., a module path) as well
/// as the crate name and disambiguator.
2017-08-19 00:09:55 +00:00
name: String,
llmod_id: String,
2017-12-27 18:03:48 +00:00
source: ModuleSource,
kind: ModuleKind,
}
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
#[derive(Copy, Clone, Debug, PartialEq)]
2017-12-28 09:11:26 +00:00
enum ModuleKind {
Regular,
Metadata,
Allocator,
}
2018-05-08 13:10:16 +00:00
impl ModuleCodegen {
2017-12-27 18:03:48 +00:00
fn llvm(&self) -> Option<&ModuleLlvm> {
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
match self.source {
2018-05-08 13:10:16 +00:00
ModuleSource::Codegened(ref llvm) => Some(llvm),
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
ModuleSource::Preexisting(_) => None,
}
}
2017-12-27 18:03:48 +00:00
fn into_compiled_module(self,
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
emit_obj: bool,
emit_bc: bool,
emit_bc_compressed: bool,
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
outputs: &OutputFilenames) -> CompiledModule {
let pre_existing = match self.source {
ModuleSource::Preexisting(_) => true,
2018-05-08 13:10:16 +00:00
ModuleSource::Codegened(_) => false,
};
let object = if emit_obj {
Some(outputs.temp_path(OutputType::Object, Some(&self.name)))
} else {
None
};
let bytecode = if emit_bc {
Some(outputs.temp_path(OutputType::Bitcode, Some(&self.name)))
} else {
None
};
let bytecode_compressed = if emit_bc_compressed {
Some(outputs.temp_path(OutputType::Bitcode, Some(&self.name))
.with_extension(RLIB_BYTECODE_EXTENSION))
} else {
None
};
CompiledModule {
llmod_id: self.llmod_id,
name: self.name.clone(),
kind: self.kind,
pre_existing,
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
object,
bytecode,
bytecode_compressed,
}
}
}
#[derive(Debug)]
2017-12-28 09:11:26 +00:00
struct CompiledModule {
2017-12-27 18:03:48 +00:00
name: String,
llmod_id: String,
2017-12-27 18:03:48 +00:00
kind: ModuleKind,
pre_existing: bool,
object: Option<PathBuf>,
bytecode: Option<PathBuf>,
bytecode_compressed: Option<PathBuf>,
}
2017-12-27 18:03:48 +00:00
enum ModuleSource {
/// Copy the `.o` files or whatever from the incr. comp. directory.
Preexisting(WorkProduct),
/// Rebuild from this LLVM module.
2018-05-08 13:10:16 +00:00
Codegened(ModuleLlvm),
}
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
#[derive(Debug)]
2017-12-28 09:11:26 +00:00
struct ModuleLlvm {
2017-08-19 00:09:55 +00:00
llcx: llvm::ContextRef,
2017-12-27 18:03:48 +00:00
llmod: llvm::ModuleRef,
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
tm: llvm::TargetMachineRef,
}
rustc: Enable LTO and multiple codegen units This commit is a refactoring of the LTO backend in Rust to support compilations with multiple codegen units. The immediate result of this PR is to remove the artificial error emitted by rustc about `-C lto -C codegen-units-8`, but longer term this is intended to lay the groundwork for LTO with incremental compilation and ultimately be the underpinning of ThinLTO support. The problem here that needed solving is that when rustc is producing multiple codegen units in one compilation LTO needs to merge them all together. Previously only upstream dependencies were merged and it was inherently relied on that there was only one local codegen unit. Supporting this involved refactoring the optimization backend architecture for rustc, namely splitting the `optimize_and_codegen` function into `optimize` and `codegen`. After an LLVM module has been optimized it may be blocked and queued up for LTO, and only after LTO are modules code generated. Non-LTO compilations should look the same as they do today backend-wise, we'll spin up a thread for each codegen unit and optimize/codegen in that thread. LTO compilations will, however, send the LLVM module back to the coordinator thread once optimizations have finished. When all LLVM modules have finished optimizing the coordinator will invoke the LTO backend, producing a further list of LLVM modules. Currently this is always a list of one LLVM module. The coordinator then spawns further work to run LTO and code generation passes over each module. In the course of this refactoring a number of other pieces were refactored: * Management of the bytecode encoding in rlibs was centralized into one module instead of being scattered across LTO and linking. * Some internal refactorings on the link stage of the compiler was done to work directly from `CompiledModule` structures instead of lists of paths. * The trans time-graph output was tweaked a little to include a name on each bar and inflate the size of the bars a little
2017-07-23 15:14:38 +00:00
unsafe impl Send for ModuleLlvm { }
unsafe impl Sync for ModuleLlvm { }
impl Drop for ModuleLlvm {
fn drop(&mut self) {
unsafe {
llvm::LLVMDisposeModule(self.llmod);
llvm::LLVMContextDispose(self.llcx);
llvm::LLVMRustDisposeTargetMachine(self.tm);
}
}
}
2018-05-08 13:10:16 +00:00
struct CodegenResults {
2017-12-27 18:03:48 +00:00
crate_name: Symbol,
modules: Vec<CompiledModule>,
2017-08-19 00:09:55 +00:00
allocator_module: Option<CompiledModule>,
metadata_module: CompiledModule,
2017-12-27 18:03:48 +00:00
link: rustc::middle::cstore::LinkMeta,
metadata: rustc::middle::cstore::EncodedMetadata,
2017-08-19 00:09:55 +00:00
windows_subsystem: Option<String>,
linker_info: back::linker::LinkerInfo,
crate_info: CrateInfo,
}
/// Misc info we load from metadata to persist beyond the tcx
2017-12-28 09:11:26 +00:00
struct CrateInfo {
panic_runtime: Option<CrateNum>,
compiler_builtins: Option<CrateNum>,
profiler_runtime: Option<CrateNum>,
sanitizer_runtime: Option<CrateNum>,
is_no_builtins: FxHashSet<CrateNum>,
2018-02-27 16:11:14 +00:00
native_libraries: FxHashMap<CrateNum, Lrc<Vec<NativeLibrary>>>,
crate_name: FxHashMap<CrateNum, String>,
2018-02-27 16:11:14 +00:00
used_libraries: Lrc<Vec<NativeLibrary>>,
link_args: Lrc<Vec<String>>,
used_crate_source: FxHashMap<CrateNum, Lrc<CrateSource>>,
used_crates_static: Vec<(CrateNum, LibSource)>,
used_crates_dynamic: Vec<(CrateNum, LibSource)>,
rustc: Add a `#[wasm_import_module]` attribute This commit adds a new attribute to the Rust compiler specific to the wasm target (and no other targets). The `#[wasm_import_module]` attribute is used to specify the module that a name is imported from, and is used like so: #[wasm_import_module = "./foo.js"] extern { fn some_js_function(); } Here the import of the symbol `some_js_function` is tagged with the `./foo.js` module in the wasm output file. Wasm-the-format includes two fields on all imports, a module and a field. The field is the symbol name (`some_js_function` above) and the module has historically unconditionally been `"env"`. I'm not sure if this `"env"` convention has asm.js or LLVM roots, but regardless we'd like the ability to configure it! The proposed ES module integration with wasm (aka a wasm module is "just another ES module") requires that the import module of wasm imports is interpreted as an ES module import, meaning that you'll need to encode paths, NPM packages, etc. As a result, we'll need this to be something other than `"env"`! Unfortunately neither our version of LLVM nor LLD supports custom import modules (aka anything not `"env"`). My hope is that by the time LLVM 7 is released both will have support, but in the meantime this commit adds some primitive encoding/decoding of wasm files to the compiler. This way rustc postprocesses the wasm module that LLVM emits to ensure it's got all the imports we'd like to have in it. Eventually I'd ideally like to unconditionally require this attribute to be placed on all `extern { ... }` blocks. For now though it seemed prudent to add it as an unstable attribute, so for now it's not required (as that'd force usage of a feature gate). Hopefully it doesn't take too long to "stabilize" this! cc rust-lang-nursery/rust-wasm#29
2018-02-10 22:28:17 +00:00
wasm_imports: FxHashMap<String, String>,
lang_item_to_crate: FxHashMap<LangItem, CrateNum>,
missing_lang_items: FxHashMap<CrateNum, Vec<LangItem>>,
}
2018-05-08 13:10:16 +00:00
__build_diagnostic_array! { librustc_codegen_llvm, DIAGNOSTICS }