This commit removes all jemalloc related submodules, configuration, etc,
from the bootstrap, from the standard library, and from the compiler.
This will be followed up with a change to use jemalloc specifically as
part of rustc on blessed platforms.
This optionally adds lldb (and clang, which it needs) to the build.
Because rust uses LLVM 7, and because clang 7 is not yet released, a
recent git master version of clang is used.
The lldb that is used includes the Rust plugin.
lldb is only built when asked for, or when doing a nightly build on
macOS. Only macOS is done for now due to difficulties with the Python
dependency.
If you are using a hard-linked file as your config.toml, this change will affect the way other instances of the file is modified.
The original version would modify all other instances whereas the new version will leave others unchanged, reducing the ref count by one.
Signed-off-by: NODA, Kai <nodakai@gmail.com>
This ensures that the working directory of rustbuild has no effect on
it's run; since tests will run with a different cwd this is required for
consistent behavior.
This option was introduced in 72cb109bec, but it uses two different
spellings (fast-submodule vs fast-submodules) and isn't handled by
Rust bootstrap which means that any attempt to set this flag fails.
Faster submodule updating
For the common case when there are no submodules which need updating, this takes 0.48 seconds instead of 47 seconds.
r? @alexcrichton
- The bootstrap crate currently passes -v to Cargo if itself invoked
with -vv. But Cargo supports -vv (to show build script output), so make
bootstrap pass that if itself invoked with -vvv. (More specifically,
pass N '-v's to Cargo if invoked with N+1 of them.)
- bootstrap.py currently tries to pass on up to two '-v's to cargo when
building bootstrap, but incorrectly ('-v' is marked as 'store_true', so
argparse stores either False or True, ignoring multiple '-v's). Fix
this, allow passing any number of '-v's, and make it consistent with
bootstrap's invocation of Cargo (i.e. subtract one from the number of
'-v's).
- Also improve bootstrap.py's config.toml 'parsing' to support arbitrary
verbosity levels, + allow command line to override it.
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
Restore the download of rust-mingw
The build might otherwise break due to mixing MinGW object files from rust-std and the local MinGW which might be newer/older than the version used to build rust-std.
Fixes#48272
r? @alexcrichton
Improve --help performance for x.py
Since compiling the bootstrap command doesn't require any submodules,
we can skip updating submodules when a --help command is passed in.
On my machine, this saves 1 minute if the submodules are already
downloaded, and 10 minutes if run on a clean repo.
This commit also adds a message before compiling/downloading anything
when a --help command is passed in, to tell the user WHY --help
takes so long to complete. It also points the user to the bootstrap
README.md for faster help.
Finally, this fixes one warning message that still referenced using
make instead of x.py, even though x.py is now the standard way of
building rust.
Closes#37305
It was an existing solution to tell the user why a --help command
takes a long time to process. However, it would only print if the
stage0 rust compiler needed to be downloaded, it came after
update_submodules (which took a long time), and it was immediately
followed by download messages and loading bars, meaning users could
easily gloss over the message.
This commit also moves the help message out of main(), and instead
puts it at the top of bootstrap(). main() is intended to be minimal,
only handling error messages.
The build might otherwise break due to mixing MinGW object files from
rust-std and the local MinGW which might be newer/older than the version
used to build rust-std.
Fixes#48272
Since compiling the bootstrap command doesn't require any submodules,
we can skip updating submodules when a --help command is passed in.
On my machine, this saves 1 minute if the submodules are already
downloaded, and 10 minutes if run on a clean repo.
This commit also adds a message before compiling/downloading anything
when a --help command is passed in, to tell the user WHY --help
takes so long to complete. It also points the user to the bootstrap
README.md for faster help.
Finally, this fixes one warning message that still referenced using
make instead of x.py, even though x.py is now the standard way of
building rust.
This commit introduces a separately compiled backend for Emscripten, avoiding
compiling the `JSBackend` target in the main LLVM codegen backend. This builds
on the foundation provided by #47671 to create a new codegen backend dedicated
solely to Emscripten, removing the `JSBackend` of the main codegen backend in
the process.
A new field was added to each target for this commit which specifies the backend
to use for translation, the default being `llvm` which is the main backend that
we use. The Emscripten targets specify an `emscripten` backend instead of the
main `llvm` one.
There's a whole bunch of consequences of this change, but I'll try to enumerate
them here:
* A *second* LLVM submodule was added in this commit. The main LLVM submodule
will soon start to drift from the Emscripten submodule, but currently they're
both at the same revision.
* Logic was added to rustbuild to *not* build the Emscripten backend by default.
This is gated behind a `--enable-emscripten` flag to the configure script. By
default users should neither check out the emscripten submodule nor compile
it.
* The `init_repo.sh` script was updated to fetch the Emscripten submodule from
GitHub the same way we do the main LLVM submodule (a tarball fetch).
* The Emscripten backend, turned off by default, is still turned on for a number
of targets on CI. We'll only be shipping an Emscripten backend with Tier 1
platforms, though. All cross-compiled platforms will not be receiving an
Emscripten backend yet.
This commit means that when you download the `rustc` package in Rustup for Tier
1 platforms you'll be receiving two trans backends, one for Emscripten and one
that's the general LLVM backend. If you never compile for Emscripten you'll
never use the Emscripten backend, so we may update this one day to only download
the Emscripten backend when you add the Emscripten target. For now though it's
just an extra 10MB gzip'd.
Closes#46819
This currently only supports a limited subset of the full compilation,
but is likely 90% of what people will want and is possible without
building a full compiler (i.e., running LLVM). In theory, this means
that contributors who don't want to build LLVM now have an easy way to
compile locally, though running tests won't work.
If config.toml doesn't exist, then an IOError will be raised
on the `with open(...)` line. Prior to e788fa7, this was
caught because the `except` clause didn't specify what
exceptions it caught, so both IOError and OSError were
caught
This commit rewrites our ancient `./configure` script from shell into Python.
The impetus for this change is to remove `config.mk` which is just a vestige of
the old makefile build system at this point. Instead all configuration is now
solely done through `config.toml`.
The python script allows us to more flexibly program (aka we can use loops
easily) and create a `config.toml` which is based off `config.toml.example`.
This way we can preserve comments and munge various values as we see fit.
It is intended that the configure script here is a drop-in replacement for the
previous configure script, no functional change is intended. Also note that the
rationale for this is also because our build system requires Python, so having a
python script a bit earlier shouldn't cause too many problems.
Closes#40730
This commit includes the following:
* Fix syntax errors in Python 3
* Include more docstrings in classes, methods, and functions
* Include unit tests using `unittest`
* Merge implementation of `{rustc,cargo}_out_of_date`
* Merge implementation of `RustBuild.{cargo,rustc}`
* Remove unnecessary source code
* Move all the attributes defined outside of `__init__`
* Remove remaining `%s` from print function
* Remove `WindowsError` reference on non-windows systems
* Rename some variables to be more explicit avoid their meaning
* Run bootstrap tests in the CI process
* Remove non-pythonic getters
* Remove duplicate code in `download_stage0` method
* Reduce the number of branches in `build_bootstrap` method
* Re-raise exception when we cannot execute uname in non-windows systems
* Avoid long lines
This commit migrates the in-tree `libcompiler_builtins` to the upstream version
at https://github.com/rust-lang-nursery/compiler-builtins. The upstream version
has a number of intrinsics written in Rust and serves as an in-progress rewrite
of compiler-rt into Rust. Additionally it also contains all the existing
intrinsics defined in `libcompiler_builtins` for 128-bit integers.
It's been the intention since the beginning to make this transition but
previously it just lacked the manpower to get done. As this PR likely shows it
wasn't a trivial integration! Some highlight changes are:
* The PR rust-lang-nursery/compiler-builtins#166 contains a number of fixes
across platforms and also some refactorings to make the intrinsics easier to
read. The additional testing added there also fixed a number of integration
issues when pulling the repository into this tree.
* LTO with the compiler-builtins crate was fixed to link in the entire crate
after the LTO process as these intrinsics are excluded from LTO.
* Treatment of hidden symbols was updated as previously the
`#![compiler_builtins]` crate would mark all symbol *imports* as hidden
whereas it was only intended to mark *exports* as hidden.
When bootstrap is executed with python not in `$PATH`, (e. g.
`c:\Python27\python.exe x.py test`) bootstrap cannot find python
and crashes.
This commit passes path to python in `BOOTSTRAP_PYTHON` env var.
