- Suggest `x.py setup` if config.toml doesn't exist yet (twice, once
before and once after the build)
- Prompt for a profile if not given on the command line
- Print the configuration file that will be used
- Print helpful starting commands after setup
- Link to the dev-guide after finishing
- Note that distro maintainers will see the changelog warning
The performance difference is negligible, but it makes me feel better.
Note that this does not remove some clones in `config`, because it would
require changing the logic around (and performance doesn't matter
for bootstrap).
Fix generating rustc docs with non-default lib directory.
If `libdir` is set in `config.toml`, then the tool to generate the rustc docs was unable to run `rustc` because it could not find the shared libraries. The solution is to set the dylib search path to include the libdir.
I changed the API of `add_rustc_lib_path` to take `Command` instead of `Cargo` to try to share the code in several places. This is how it worked before https://github.com/rust-lang/rust/pull/64316, and I think this still retains the spirit of that change.
Fixes#76702
Make the default stage for x.py configurable
This also allows configuring each sub-command individually.
Possibly #76617 should land before this? I don't feel strongly either way, I don't mind waiting.
Closes https://github.com/rust-lang/rust/issues/76165.
r? `@Mark-Simulacrum`
This allows configuring the default stage for each sub-command individually.
- Normalize the stage as early as possible, so there's no confusion
about which to use.
- Don't add an explicit `stage` option in config.toml
This offers no more flexibility than `*_stage` and makes it confusing
which takes precedence.
- Always give `--stage N` precedence over config.toml
- Fix bootstrap tests
This changes the tests to go through `Config::parse` so that they test
the actual defaults, not the dummy ones provided by `default_opts`. To
make this workable (and independent of the environment), it does not
read `config.toml` for tests.
Add host triples to CI builders
This is a follow-up to #76415, which changed how x.py interprets cross-compilation target/host flags. This should fix the known cases, but I'm still working through CI logs before/after that PR to identify if anything else is missing.
It is really painful to inspect differences in what was built in CI if things
are appearing and disappearing randomly as they hover around the 100ms mark. No
matter what we choose there's always going to be quite a bit of variability on
CI in timing, so we're going to see things appear and vanish.
rustc is a natively cross-compiling compiler, and generally none of our steps
should care whether they are using a compiler built of triple A or B, just the
--target directive being passed to the running compiler. e.g., when building for
some target C, you don't generally want to build two stds: one with a host A
compiler and the other with a host B compiler. Just one std is sufficient.
rustbuild: Build tests with LLD if `use-lld = true` was passed
Addresses https://github.com/rust-lang/rust/pull/76127#discussion_r479932392.
Our test suite is generally ready to run with an explicitly specified linker (https://github.com/rust-lang/rust/pull/45191),
so LLD specified with `use-lld = true` works as well.
Only 4 tests fail (on `x86_64-pc-windows-msvc`):
```
ui/panic-runtime/lto-unwind.rs
run-make-fulldeps/debug-assertions
run-make-fulldeps/foreign-exceptions
run-make-fulldeps/test-harness
```
All of them are legitimate issues with LLD (or at least with combination Rust+LLD) and manifest in segfaults on access to TLS (https://github.com/rust-lang/rust/pull/76127#issuecomment-683473325). UPD: These issues are caused by https://github.com/rust-lang/rust/issues/72145 and appear because I had `-Ctarget-cpu=native` set.
UPD: Further commits build tests with LLD for non-MSVC targets and propagate LLD to more places when `use-lld` is enabled.
This currently includes libLLVM, llvm-config, and FileCheck, but will perhaps
expand to more tooling overtime. It should be considered entirely unstable and
may change at any time.
Often when modifying compiler code you'll miss that you've changed an API used
by unit tests, since x.py check didn't previously catch that.
It's also useful to have this for editing with rust-analyzer and similar tooling
where editing tests previously didn't notify you of errors in test files.
Add option to use the new symbol mangling in rustc/std
I don't know if this causes problems in some cases -- maybe it should be on by default for at least rustc. I've never encountered problems with it other than tools not supporting it, though.
cc @nnethercote
r? @eddyb
- Use stage 2 for makefile
- Move assert to builder
- Don't add an assert for --help
- Allow --stage 0 if passed explicitly
- Don't assert defaults during tests
Otherwise it's impossible to test the defaults!
### x.py build/test: stage 1
I've seen very few people who actually use full stage 2 builds on purpose. These compile rustc and libstd twice and don't give you much more information than a stage 1 build (except in rare cases like https://github.com/rust-lang/rust/pull/68692#discussion_r376392145). For new contributors, this makes the build process even more daunting than it already is. As long as CI is changed to use `--stage 2` I see no downside here.
### x.py bench/dist/install: stage 2
These commands have to do with a finished, optimized version of rustc. It seems very rare to want to use these with a stage 1 build.
### x.py doc: stage 0
Normally when you document things you're just fixing a typo. In this case there is no need to build the whole rust compiler, since the documentation will usually be the same when generated with the beta compiler or with stage 1.
Note that for this release cycle only there will be a significant different between stage0 and stage1 docs: https://github.com/rust-lang/rust/pull/73101. However most of the time this will not be the case.
Stabilize control-flow-guard codegen option
This is the stabilization PR discussed in #68793. It converts the `-Z control-flow-guard` debugging option into a codegen option (`-C control-flow-guard`), and changes the associated tests.
Generating the coverage map
@tmandry @wesleywiser
rustc now generates the coverage map and can support (limited)
coverage report generation, at the function level.
Example commands to generate a coverage report:
```shell
$ BUILD=$HOME/rust/build/x86_64-unknown-linux-gnu
$ $BUILD/stage1/bin/rustc -Zinstrument-coverage \
$HOME/rust/src/test/run-make-fulldeps/instrument-coverage/main.rs
$ LLVM_PROFILE_FILE="main.profraw" ./main
called
$ $BUILD/llvm/bin/llvm-profdata merge -sparse main.profraw -o main.profdata
$ $BUILD/llvm/bin/llvm-cov show --instr-profile=main.profdata main
```
r? @wesleywiser
Rust compiler MCP rust-lang/compiler-team#278
Relevant issue: #34701 - Implement support for LLVMs code coverage instrumentation
Teach bootstrap about target files vs target triples
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
`rustc` allows passing in predefined target triples as well as JSON
target specification files. This change allows bootstrap to have the
first inkling about those differences. This allows building a
cross-compiler for an out-of-tree architecture (even though that
compiler won't work for other reasons).
Even if no one ever uses this functionality, I think the newtype
around the `Interned<String>` improves the readability of the code.
The current plan is that submodule tracks the `release` branch of
rust-analyzer, which is updated once a week.
rust-analyzer is a workspace (with a virtual manifest), the actual
binary is provide by `crates/rust-analyzer` package.
Note that we intentionally don't add rust-analyzer to `Kind::Test`,
for two reasons.
*First*, at the moment rust-analyzer's test suite does a couple of
things which might not work in the context of rust repository. For
example, it shells out directly to `rustup` and `rustfmt`. So, making
this work requires non-trivial efforts.
*Second*, it seems unlikely that running tests in rust-lang/rust repo
would provide any additional guarantees. rust-analyzer builds with
stable and does not depend on the specifics of the compiler, so
changes to compiler can't break ra, unless they break stability
guarantee. Additionally, rust-analyzer itself is gated on bors, so we
are pretty confident that test suite passes.
This commit moves the compiler-builtins-specific build logic from
`src/bootstrap/bin/rustc.rs` into the workspace `Cargo.toml`'s
`[profile]` configuration. Now that rust-lang/cargo#7253 is fixed we can
ensure that Cargo knows about debug assertions settings, and it can also
be configured to specifically disable debug assertions unconditionally
for compiler-builtins. This should improve rebuild logic when
debug-assertions settings change and also improve build-std integration
where Cargo externally now has an avenue to learn how to build
compiler-builtins as well.
Speed up bootstrap a little.
The bootstrap script was calling `cargo metadata` 3 times (or 6 with `-v`). This is a very expensive operation, and this attempts to avoid the extra calls. On my system, a simple command like `./x.py test -h -v` goes from about 3 seconds to 0.4.
An overview of the changes:
- Call `cargo metadata` only once with `--no-deps`. Optional dependencies are filtered in `in_tree_crates` (handling `profiler_builtins` and `rustc_codegen_llvm` which are driven by the config).
- Remove a duplicate call to `metadata::build` when using `-v`. I'm not sure why it was there, it looks like a mistake or vestigial from previous behavior.
- Remove check for `_shim`, I believe all the `_shim` crates are now gone.
- Remove check for `rustc_` and `*san` for `test::Crate::should_run`, these are no longer dependencies in the `test` tree.
- Use relative paths in `./x.py test -h -v` output.
- Some code cleanup (remove unnecessary `find_compiler_crates`, etc.).
- Show suite paths (`src/test/ui/...`) in `./x.py test -h -v` output.
- Some doc comments.
Remove vestigial CI job msvc-aux.
This CI job isn't really doing anything, so it seems prudent to remove it.
For some history:
* This was introduced in #48809 when the msvc job was split in two to keep it under 2 hours (oh the good old days). At the time, this check-aux job did a bunch of things:
* tidy
* src/test/pretty
* src/test/run-pass/pretty
* src/test/run-fail/pretty
* src/test/run-pass-valgrind/pretty
* src/test/run-pass-fulldeps/pretty
* src/test/run-fail-fulldeps/pretty
* Tidy was removed in #60777.
* run-pass and run-pass-fulldeps moved to UI in #63029
* src/test/pretty removed in #58140
* src/test/run-fail moved to UI in #71185
* run-fail-fulldeps removed in #51285
Over time through attrition, the job was left with one lonely thing: `src/test/run-pass-valgrind/pretty`. And of course, this wasn't actually running the "pretty" tests. The normal `run-pass-valgrind` tests ran, and then when it tried to run in "pretty" mode, all the tests were ignored because compiletest thought nothing had changed (apparently compiletest isn't fingerprinting the mode? Needs more investigation…). `run-pass-valgrind` is already being run as part of `x86_64-msvc-1`, so there's no need to run it here.
I've taken the liberty of removing `src/test/run-pass-valgrind/pretty` as a distinct test. I'm guessing from the other PR's that the pretty tests should now live in `src/test/pretty`, and that the team has moved away from doing pretty tests on other parts of the `src/test` tree.
This should run much faster.
There are also some drive-by cleanups here to try to simplify things.
Also, the paths for in-tree crates are now displayed as relative
in `x.py test -h -v`.
Provide separate option for std debug asserts
On local one-off benchmarking of libcore metadata-only, debug asserts in std are a significant hit (15s to 20s). Provide an option for compiler developers to disable them. A build with a nightly compiler is around 10s, for reference.
bootstrap: remove lldb dist packaging
The lldb-preview rustup package is missing on every single target, and has never been shipped beyond x86_64-apple-darwin. It was removed in #62592 which landed around a year ago, and there's not been demand that we re-enable it since, so we're now removing support entirely to cleanup the code a bit.
The hope is that this will also kill the useless "lldb-preview" row on https://rust-lang.github.io/rustup-components-history/.
Enable Control Flow Guard in rustbuild
Now that Rust supports Control Flow Guard (#68180), add a config.toml option to build the standard library with CFG enabled.
r? @nagisa
Add an option to use LLD to link the compiler on Windows platforms
Based on https://github.com/rust-lang/rust/pull/68609.
Using LLD is good way to improve compile times on Windows since `link.exe` is quite slow. The time for `x.py build --stage 1 src/libtest` goes from 0:12:00 to 0:08:29. Compile time for `rustc_driver` goes from 226.34s to 18.5s. `rustc_macros` goes from 28.69s to 7.7s. The size of `rustc_driver` is also reduced from 83.3 MB to 78.7 MB.
r? @Mark-Simulacrum
Clear out target directory if compiler has changed
Previously, we relied fully on Cargo to detect that the compiler had changed and
it needed to rebuild the standard library (or later "components"). This used to
not quite be the case prior to moving to LLVM be a separate cargo invocation;
subsequent compiles would recompile std and friends if LLVM had changed
(#67077 is the PR that changes things here).
This PR moves us to clearing out libstd when it is being compiled if the rustc
we're using has changed. We fairly harshly limit the cases in which we do this
(e.g., ignoring dry run mode, and so forth, as well as rustdoc invocations).
This is primarily because when we're not using the compiler directly, so
clearing out in other cases is likely to lead to bugs, particularly as our
deletion scheme is pretty blunt today (basically removing more than is needed,
i.e., not just the rustc artifacts).
In practice, this targeted fix does fix the known bug, though it may not fully
resolve the problem here. It's also not clear that there is a full fix hiding
here that doesn't involve a more major change (like -Zbinary-dep-depinfo was).
As a drive-by fix, don't delete the compiler before calling Build::copy, as that
also deletes the compiler.
Previously, we relied fully on Cargo to detect that the compiler had changed and
it needed to rebuild the standard library (or later "components"). This used to
not quite be the case prior to moving to LLVM be a separate cargo invocation;
subsequent compiles would recompile std and friends if LLVM had changed
(#67077 is the PR that changes things here).
This PR moves us to clearing out libstd when it is being compiled if the rustc
we're using has changed. We fairly harshly limit the cases in which we do this
(e.g., ignoring dry run mode, and so forth, as well as rustdoc invocations).
This is primarily because when we're not using the compiler directly, so
clearing out in other cases is likely to lead to bugs, particularly as our
deletion scheme is pretty blunt today (basically removing more than is needed,
i.e., not just the rustc artifacts).
In practice, this targeted fix does fix the known bug, though it may not fully
resolve the problem here. It's also not clear that there is a full fix hiding
here that doesn't involve a more major change (like -Zbinary-dep-depinfo was).
As a drive-by fix, don't delete the compiler before calling Build::copy, as that
also deletes the compiler.
Bootstrap: change logic for choosing linker and rpath
This is a follow-up from #66957 and #67023. Apparently there was one more location with a hard-coded list of targets to influence linking.
I've filed #67171 to track this madness.
r? @alexcrichton
This commit builds on #65501 continue to simplify the build system and
compiler now that we no longer have multiple LLVM backends to ship by
default. Here this switches the compiler back to what it once was long
long ago, which is linking LLVM directly to the compiler rather than
dynamically loading it at runtime. The `codegen-backends` directory of
the sysroot no longer exists and all relevant support in the build
system is removed. Note that `rustc` still supports a dynamically loaded
codegen backend as it did previously, it just no longer supports
dynamically loaded codegen backends in its own sysroot.
Additionally as part of this the `librustc_codegen_llvm` crate now once
again explicitly depends on all of its crates instead of implicitly
loading them through the sysroot. This involved filling out its
`Cargo.toml` and deleting all the now-unnecessary `extern crate`
annotations in the header of the crate. (this in turn required adding a
number of imports for names of macros too).
The end results of this change are:
* Rustbuild's build process for the compiler as all the "oh don't forget
the codegen backend" checks can be easily removed.
* Building `rustc_codegen_llvm` is much simpler since it's simply
another compiler crate.
* Managing the dependencies of `rustc_codegen_llvm` is much simpler since
it's "just another `Cargo.toml` to edit"
* The build process should be a smidge faster because there's more
parallelism in the main rustc build step rather than splitting
`librustc_codegen_llvm` out to its own step.
* The compiler is expected to be slightly faster by default because the
codegen backend does not need to be dynamically loaded.
* Disabling LLVM as part of rustbuild is still supported, supporting
multiple codegen backends is still supported, and dynamic loading of a
codegen backend is still supported.
This is not yet actually used by CI, but implements the logic for
checking that tools are properly building on beta/stable and during beta
cutoff week.
This attempts to mirror the checking functionality in
src/ci/docker/x86_64-gnu-tools/checktools.sh, and called scripts. It
does not attempt to run the relevant steps (that functionality was
originally desired to be moved into bootstrap as well, but doing so
proved more difficult than expected).
This is intended as a way to centralize and make clearer the logic
involved in toolstate checking. In particular, the previous logic was
spread across numerous python and shell scripts in such a way that made
interpretation quite difficult.
Use a relative bindir for rustdoc to find rustc
In bootstrap, we set `RUSTC_INSTALL_BINDIR` to `config.bindir`, so
rustdoc can find rustc relative to the toolchain sysroot. However, if a
distro script like Fedora's `%configure` sets an absolute path, then
rustdoc's `sysroot.join(bin_path)` ignores that sysroot altogether.
That would be OK once the toolchain is actually installed, but it breaks
the in-tree doc tests during the build, since `/usr/bin/rustc` is still
the old version. So now we try to make `RUSTC_INSTALL_BINDIR` relative
to the sysroot prefix in the first place.
r? @Mark-Simulacrum
In bootstrap, we set `RUSTC_INSTALL_BINDIR` to `config.bindir`, so
rustdoc can find rustc relative to the toolchain sysroot. However, if a
distro script like Fedora's `%configure` sets an absolute path, then
rustdoc's `sysroot.join(bin_path)` ignores that sysroot altogether.
That would be OK once the toolchain is actually installed, but it breaks
the in-tree doc tests during the build, since `/usr/bin/rustc` is still
the old version. So now we try to make `RUSTC_INSTALL_BINDIR` relative
to the sysroot prefix in the first place.
Split the rustc target libraries into separate rustc-dev component
This is re-applies a squashed version of #64823 as well as including #65337 to fix bugs noted after merging the first PR.
The second PR is confirmed as fixing windows-gnu, and presumably also fixes other platforms, such as musl (i.e. #65335 should be fixed); `RUSTUP_DIST_SERVER=https://dev-static.rust-lang.org rustup toolchain install nightly-2019-10-16` can be installed to confirm that this is indeed the case.
- Compatible with Emscripten 1.38.46-upstream or later upstream.
- Refactors the Emscripten target spec to share code with other wasm
targets.
- Replaces the old incorrect wasm32 C call ABI with the correct one,
preserving the old one as wasm32_bindgen_compat for wasm-bindgen
compatibility.
- Updates the varargs ABI used by Emscripten and deletes the old one.
- Removes the obsolete wasm32-experimental-emscripten target.
- Uses EMCC_CFLAGS on CI to avoid the timeout problems with #63649.
This splits out a rustc-dev component with the compiler crates, and
keeps the status quo of default installed files on nightly. The default
changing to not install compiler libraries by default is left for a
future pull request.
However, on stable and beta, this does remove the compiler libraries
from the set of libraries installed by default, as they are never needed
there (per our stability story, they "cannot" be used).
minimize the rust-std component
This changes the `rust-std` dist component to only include the artifacts of compiling the `libstd` step, as listed in `.libstd.stamp`. This does include `test` and `proc-macro` as well. The remaining _unstable_ libraries that are built as part of `rustc` are packaged into a new `rustc-dev` component, intended for use in the development of closely related tools (clippy, miri, rls).
Here are the component sizes from the [try build](https://dev-static.rust-lang.org/dist/2019-10-07/index.html):
| Name | Size
| --- | ---
| rust-std-nightly-x86_64-unknown-linux-gnu.tar.gz | 23.94 MiB
| rust-std-nightly-x86_64-unknown-linux-gnu.tar.xz | 17.4 MiB
| rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.gz | 182.03 MiB
| rustc-dev-nightly-x86_64-unknown-linux-gnu.tar.xz | 157.91 MiB
Fixes#61978Fixes#62486
This is needed to ensure that the crates during a normal build are
shared with the crates during testing, otherwise they'll end up hasing
differently and we'll recompile crates like `core` during tests.
This commit changes the return type of `Builder::cargo` to return a
builder that allows dynamically adding more `RUSTFLAGS` values
after-the-fact. While not used yet, this will later be used to delete
more of `rustc.rs`
Most of `bootstrap/bin/rustc.rs` doesn't need to exist with the advent
of `RUSTFLAGS` (yes this is super old) so this starts by refactoring a
bit to make it easier locally in the `Builder::cargo` method to append
to `RUSTFLAGS` that gets down to rustc.
This ensures that the failure cases for finding the codegen backend and
for finding the rustc binary are essentially the same, and since we
almost always will load the codegen backend, this is essentially meaning
that the rustc change is not a regression.
Since its inception rustbuild has always worked in three stages: one for
libstd, one for libtest, and one for rustc. These three stages were
architected around crates.io dependencies, where rustc wants to depend
on crates.io crates but said crates don't explicitly depend on libstd,
requiring a sysroot assembly step in the middle. This same logic was
applied for libtest where libtest wants to depend on crates.io crates
(`getopts`) but `getopts` didn't say that it depended on std, so it
needed `std` built ahead of time.
Lots of time has passed since the inception of rustbuild, however,
and we've since gotten to the point where even `std` itself is depending
on crates.io crates (albeit with some wonky configuration). This
commit applies the same logic to the two dependencies that the `test`
crate pulls in from crates.io, `getopts` and `unicode-width`. Over the
many years since rustbuild's inception `unicode-width` was the only
dependency picked up by the `test` crate, so the extra configuration
necessary to get crates building in this crate graph is unlikely to be
too much of a burden on developers.
After this patch it means that there are now only two build phasese of
rustbuild, one for libstd and one for rustc. The libtest/libproc_macro
build phase is all lumped into one now with `std`.
This was originally motivated by rust-lang/cargo#7216 where Cargo was
having to deal with synthesizing dependency edges but this commit makes
them explicit in this repository.
This patch supports less behavior than before, since specifiying stage 1
vs stage 2 is no longer possible, but that is presumably a somewhat rare
use case anyway, so not supporting it seems acceptable (and it can be
readded easily if desired).
Utilize -Zbinary-dep-depinfo in rustbuild
We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.
Both of these are fixable in the longer term but this existing patch gives us the following benefits:
* We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes#54712.
* Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes#54008, fixes#50481.
* We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes#53284, and fixes#52719.
* Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes#49979 (but no reproduction steps in that issue). Fixes#59105.
cc #63012
ci: Attempt to skip a full rustc compile on dist*
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
This extends a test in the previous commit to assert that we don't build
extra rustc compilers even when the "extended" option is set to true.
This involved some internal refactoring to have more judicious usage of
`compiler_for`, added in the previous commit, as well.
Various `dist::*` targets were refactored to be parameterized with a
`Compiler` instead of a `stage`/`host`, and then the various parameters
within the `Extended` target were tweaked to ensure that we don't ever
accidentally ask for a stage2 build compiler when we're distributing
something.
This commit furthers the previous one to ensure that we don't build an
extra stage of the compiler in CI. A test has been added to rustbuild to
ensure that this doesn't regress, and then in debugging this test it was
hunted down that the `dist::Std` target was the one erroneously pulling
in the wrong compiler.
The `dist::Std` step was updated to instead account for the "full
bootstrap" or not flag, ensuring that the correct compiler for compiling
the final standard library was used. This was another use of the
`force_use_stage1` function which was in theory supposed to be pretty
central, so existing users were all evaluated and a new function,
`Builder::compiler_for`, was introduced. All existing users of
`force_use_stage1` have been updated to use `compiler_for`, where the
semantics of `compiler_for` are similar to that of `compiler` except
that it doesn't guarantee the presence of a sysroot for the arguments
passed (as they may be modified).
Perhaps one day we can unify `compiler` and `compiler_for`, but the
usage of `Builder::compiler` is so ubiquitous it would take quite some
time to evaluate whether each one needs the sysroot or not, so it's
hoped that can be done in parallel.
Rework how bootstrap tools are built
This makes bootstrap tools buildable and testable in stage 0 with the downloaded bootstrap compiler, futhermore, it makes it such that they cannot be built in any other stage.
Notably, this will also mean that compiletest may need to wait a cycle before it can use changes to `libtest`, as it no longer depends on the in-tree libtest.
Introduce assembly tests suite
The change introduces a new test suite - **Assembly** tests. The motivation behind this is an ability to perform end-to-end codegen testing with LLVM backend. Turned out, NVPTX backend sometimes missing common Rust features (`i128` and libcalls in the past, and still full atomics support) due to different reasons.
Prior to this change, basic NVPTX assembly tests were implemented within `run-make` suite. Now, it's easier to write additional and maintain existing tests for the target.
cc @gnzlbg @peterhj
cc @eddyb I adjusted mangling scheme expectation, so there is no need to change the tests for #57967
Fixes rustdoc in stage 0, stage 1
When a request for rustdoc is passed for stage 0, x.py build --stage 0
src/tools/rustdoc or ensure(tool::Rustdoc { .. }) with top_stage = 0, we
return the rustdoc for that compiler (i.e., the beta rustdoc).
This fixes stage 0 of https://github.com/rust-lang/rust/issues/52186 as well as being part of general workflow improvements (making stage 0 testing for std work) for rustbuild.
The stage 1 fix (second commit) completely resolves the problem, so this fixes https://github.com/rust-lang/rust/issues/52186.
The RUSTDOC_LIBDIR should be rustc_libdir, not sysroot_libdir; rustdoc
is like the compiler and should link against rustc's libdir.
Some people currently (i.e., in general, may not be on master) have doc
tests working, but no attempt to determine why has been attempted.
Using CC, CFLAGS, CXX, CXXFLAGS, AR and RANLIB breaks cross compilation
because host is built first and has correct values. The same
values are incorrect for the target however.
This commit works around the newly-introduced LLVM shared library.
This is needed such that llvm-config run from
librustc_llvm's build script can correctly locate it's own LLVM, not the
one in stage0/lib. The LLVM build system uses the DT_RUNPATH/RUNPATH
header within the llvm-config binary, which we want to use, but because
Cargo always adds the host compiler's "libdir" (stage0/lib in our
case) to the dynamic linker's search path, we weren't properly finding
the freshly-built LLVM in llvm/lib. By restoring the environment
variable setting the search path to what bootstrap sees, the problem is
resolved and librustc_llvm correctly links and finds the appropriate
LLVM.
Several run-make-fulldeps tests are also updated with similar handling.
Use `clear_if_dirty` on std for backend changes, just as we do for
changes to rustc itself, so new codegen is correctly applied to all
later compiler stages.
Fixes#48298.
The (currently) single unit test of the compiletest tool was never
executed on CI. At least I couldn't find any references of it in the
logs. This adds a test suite for compiletest so that our tester is
tested, too.
The compiletest tests can then also be executed with:
./x.py test src/tools/compiletest
Add index page argument
@Mark-Simulacrum: I might need some help from you: in bootstrap, I want to add an argument (a new flag added into `rustdoc`) in order to generate the current index directly when `rustdoc` is documenting the `std` lib. However, my change in `bootstrap` didn't do it and I assume it must be moved inside the `Std` struct. But there, I don't see how to pass it to `rustdoc` through `cargo`. Did I miss anything?
r? @QuietMisdreavus
* Make it influence the behavior of the compiled rustc, rather than
just the rustc build system. That is, if verify_llvm_ir=true,
even manual invocations of the built rustc will verify LLVM IR.
* Enable verification of LLVM IR in CI, for non-deploy and
deploy-alt builds. This is similar to how LLVM assertions are
handled.
Clean up dependency tracking in Rustbuild [2/2]
Make `clear_if_dirty` calls in `Builder::cargo` with stamp dependencies for the given Mode.
Continuation of #50904
Ref issue #50509
r? @Mark-Simulacrum
This commit updates the debuginfo that is encoded in all of our released
artifacts by default. Currently it has paths like `/checkout/src/...` but these
are a little inconsistent and have changed over time. This commit instead
attempts to actually define the file paths in our debuginfo to be consistent
between releases.
All debuginfo paths are now intended to be `/rustc/$sha` where `$sha` is the git
sha of the released compiler. Sub-paths are all paths into the git repo at that
`$sha`.
Options:
-h, --help Display this message
--cfg SPEC Configure the compilation environment
-L [KIND=]PATH Add a directory to the library search path. The
optional KIND can be one of dependency, crate, native,
framework or all (the default).
-l [KIND=]NAME Link the generated crate(s) to the specified native
library NAME. The optional KIND can be one of static,
dylib, or framework. If omitted, dylib is assumed.
--crate-type [bin|lib|rlib|dylib|cdylib|staticlib|proc-macro]
Comma separated list of types of crates for the
compiler to emit
--crate-name NAME
Specify the name of the crate being built
--emit [asm|llvm-bc|llvm-ir|obj|metadata|link|dep-info|mir]
Comma separated list of types of output for the
compiler to emit
--print [crate-name|file-names|sysroot|cfg|target-list|target-cpus|target-features|relocation-models|code-models|tls-models|target-spec-json|native-static-libs]
Comma separated list of compiler information to print
on stdout
-g Equivalent to -C debuginfo=2
-O Equivalent to -C opt-level=2
-o FILENAME Write output to <filename>
--out-dir DIR Write output to compiler-chosen filename in <dir>
--explain OPT Provide a detailed explanation of an error message
--test Build a test harness
--target TARGET Target triple for which the code is compiled
-W, --warn OPT Set lint warnings
-A, --allow OPT Set lint allowed
-D, --deny OPT Set lint denied
-F, --forbid OPT Set lint forbidden
--cap-lints LEVEL
Set the most restrictive lint level. More restrictive
lints are capped at this level
-C, --codegen OPT[=VALUE]
Set a codegen option
-V, --version Print version info and exit
-v, --verbose Use verbose output
Additional help:
-C help Print codegen options
-W help Print 'lint' options and default settings
--help -v Print the full set of options rustc accepts command.
This optionally adds lldb (and clang, which it needs) to the build.
Because rust uses LLVM 7, and because clang 7 is not yet released, a
recent git master version of clang is used.
The lldb that is used includes the Rust plugin.
lldb is only built when asked for, or when doing a nightly build on
macOS. Only macOS is done for now due to difficulties with the Python
dependency.
If we detect a local rebuild (e.g. bootstrap compiler is the same version as target compiler), we set stage to 1.
When trying to build e.g. UnstableBook, we use Mode::ToolBootstrap and stage is 1.
Just allow Mode::ToolBootstrap and stagge != 0 if we are in a local_rebuild
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Allow clippy to be installed with make install
After #51122 clippy is available as a component but doesn't install when building from source.
This PR allows to install clippy with extended tools.
Previously we'd only do so for stage 0 but with keep-stage
improvements it seems likely that we'll see more developers working in
the stage 1, so we should allow enabling incremental for them.
Ideally, the check we probably want is to only enable incremental for
the last compiler build scheduled, but there's no good way to do so
today. Just enabling incremental in all stages should be sufficient;
we may be doing extra work that's needles -- compiling incrementally
something that will never be recompiled in-place -- but that should be
sufficiently unlikely (i.e., users either don't care or won't be
compiling the compiler twice).
Disable LLVM verification by default
Currently -Z no-verify only controls IR verification prior to LLVM codegen, while verification is performed unconditionally both before and after linking with (Thin)LTO.
Also wondering what the sentiment is on disabling verification by default (and e.g. only enabling it on ALT builds with assertions). This does not seem terribly useful outside of rustc development and it does seem to show up in profiles (at something like 3%).
**EDIT:** A table showing the various configurations and what is enabled when.
| Configuration | Dynamic verification performed | LLVM static assertions compiled in |
| --- | --- | --- |
| alt builds | | yes |
| nightly builds | | no |
| stable builds | | no |
| CI builds | | |
| dev builds in a checkout | | |
This commit updates the stage0 build of tools to use the libraries of the stage0
compiler instead of the compiled libraries by the stage0 compiler. This should
enable us to avoid any stage0 hacks (like missing SIMD).
ship LLVM tools with the toolchain
this PR adds llvm-{nm,objcopy,objdump,size} to the rustc sysroot (right next to LLD)
this slightly increases the size of the rustc component. I measured these numbers on x86_64 Linux:
- rustc-1.27.0-dev-x86_64-unknown-linux-gnu.tar.gz 180M -> 193M (+7%)
- rustc-1.27.0-dev-x86_64-unknown-linux-gnu.tar.xz 129M -> 137M (+6%)
r? @alexcrichton
cc #49584
Since they are unlikely to fail and are almost never going to fail
except with bootstrap changes (which would be tested locally anyway) it
makes sense to run these tests close to last.
Switch to bootstrapping from 1.27
It's possible the Float trait could be removed from core, but I couldn't tell whether it was intended to be removed or not. @SimonSapin may be able to comment more here; we can presumably also do that in a follow up PR as this one is already quite large.
Issue #50786 shows a case with local rebuild where the libraries built
by stage0 had the same suffix as stage0's own, and were accidentally
loaded by that stage0 rustc when compiling `librustc_trans`.
Now we set `__CARGO_DEFAULT_LIB_METADATA` to "bootstrap" during stage0,
rather than the release channel like usual, so the library suffix will
always be completely distinct from the stage0 compiler.
Use the correct crt*.o files when linking musl targets.
This is supposed to support optionally using the system copy of musl
libc instead of the included one if supported. This currently only
affects the start files, which is enough to allow building rustc on musl
targets.
Most of the changes are analogous to crt-static.
Excluding the start files is something musl based distributions usually patch into their copy of rustc:
- https://github.com/alpinelinux/aports/blob/eb064c8/community/rust/musl-fix-linux_musl_base.patch
- https://github.com/voidlinux/void-packages/blob/77400fc/srcpkgs/rust/patches/link-musl-dynamically.patch
For third-party distributions that not yet carry those patches it would be nice if it was supported without the need to patch upstream sources.
## Reasons
### What breaks?
Some start files were missed when originally writing the logic to swap in musl start files (gcc comes with its own start files, which are suppressed by -nostdlib, but not manually included later on). This caused #36710, which also affects rustc with the internal llvm copy or any other system libraries that need crtbegin/crtend.
### How is it fixed?
The system linker already has all the logic to decide which start files to include, so we can just defer to it (except of course if it doesn't target musl).
### Why is it optional?
In #40113 it was first tried to remove the start files, which broke compiling musl-targeting static binaries with a glibc-targeting compiler. This is why it eventually landed without removing the start files. Being an option side-steps the issue.
### Why are the start files still installed?
This has the nice side-effect, that the produced rust-std-* binaries can still be used by on a glibc-targeting system with a rustc built against glibc.
## Does it work?
With the following build script (using [musl-cross-make](https://github.com/richfelker/musl-cross-make)): https://shadowice.org/~mixi/rust-musl/build.sh, I was able to cross-compile a musl-host musl-targeting rustc on a glibc-based system. The resulting binaries are at https://shadowice.org/~mixi/rust-musl/binaries/. This also requires #50103 and #50104 (which are also applied to the branch the build script uses).
./x.py test should be able to run individual tests
Allows user to be able to run individual tests by specifying filename i.e `./x.py test src/test/run-pass/foo.rs`
Fixes#48483
Currently on CI we predominately compile LLVM with the default system compiler
which means gcc on Linux, some version of Clang on OSX, MSVC on Windows, and
gcc on MinGW. This commit switches Linux, OSX, and Windows to all use Clang
6.0.0 to build LLVM (aka the C/C++ compiler as part of the bootstrap). This
looks to generate faster code according to #49879 which translates to a faster
rustc (as LLVM internally is faster)
The major changes here were to the containers that build Linux releases,
basically adding a new step that uses the previous gcc 4.8 compiler to compile
the next Clang 6.0.0 compiler. Otherwise the OSX and Windows scripts have been
updated to download precompiled versions of Clang 6 and configure the build to
use them.
Note that `cc` was updated here to fix using `clang-cl` with `cc-rs` on MSVC, as
well as an update to `sccache` on Windows which was needed to correctly work
with `clang-cl`. Finally the MinGW compiler is entirely left out here
intentionally as it's currently thought that Clang can't generate C++ code for
MinGW and we need to use gcc, but this should be verified eventually.
trim and pass relative test paths as test-args
use collect for getting test_args
move suite_path to ShouldRun and make Option
append existing args to test_args
use enum for PathSet
handle Suites differently from Paths
Error out if part of test suite but not file
refactor, make requested changes
rustbuild: Allow quick testing of libstd and libcore at stage0
This PR implemented two features:
1. Added a `--no-doc` flag to allow testing a crate *without* doc tests. In this mode, we don't need to build rustdoc, and thus we can skip building the stage2 compiler. (Ideally stage0 test should use the bootstrap rustdoc, but I don't want to mess up the core builder logic here)
2. Moved all libcore tests externally and added a tidy test to ensure we don't accidentally add `#[test]` into libcore.
After this PR, one could run `./x.py test --stage 0 --no-doc src/libstd` to test `libstd` without building the compiler, thus enables us to quickly test new library features.
Add "the Rustc book"
This PR introduces a new book into the documentation, "The rustc book". We already have books for Cargo, and for Rustdoc, rustc should have some too. This book is focused on *users* of rustc, and provides a nice place to write documentation for users.
I haven't put content here, but plan on scaffolding it out very soon, and wanted this PR open for a few discussions first. One of those is "what exactly should said TOC be?" I plan on having a proposed one up tomorrow, but figured I'd let people know to start thinking about it now.
The big one is that we also will want to put https://github.com/rust-lang-nursery/rustc-guide in-tree as well, and the naming is... tough. I'm proposing:
* doc.rust-lang.org/rustc is "The Rustc book", to mirror the other tools' books.
* doc.rust-lang.org/rustc-contribution is "The Rustc contribution guide", and contains that book
@nikomatsakis et al, any thoughts on this? I'm not attached to it in particular, but had to put something together to get this discussion going. I think mirroring the other tools is a good idea for this work, but am not sure where exactly that leaves yours.
Fixes https://github.com/rust-docs/team/issues/11
Add rustc_trans to x.py check
r? @Mark-Simulacrum
I looked at `bootstrap/compile.rs` and `bootstrap/check.rs` to try to work out which steps were appropriate, but I'm sure I've overlooked some details here, so it's worth checking carefully I've got all the steps right (e.g. I wasn't sure whether we want to build LLVM if necessary with `x.py check`, though I thought it was probably better to than to not).
From a quick test, it seems to be working, though.
Only emit save-analysis data for `cargo build` tasks
Previously, we were emittinng analysis data for all tasks, including `doc`. That meant we got two sets of save-analysis data, one from the normal build and one from the docs. That means indexing with the RLS took twice as long and made downloads larger and build times longer.
cc https://github.com/rust-lang-nursery/rls/issues/826
r? @Mark-Simulacrum
Previously, we were emittinng analysis data for all tasks, including `doc`. That meant we got two sets of save-analysis data, one from the normal build and one from the docs. That means indexing with the RLS took twice as long and made downloads larger and build times longer.
cc https://github.com/rust-lang-nursery/rls/issues/826
Debugging information for the extended tools is currently disabled for
concerns about the size. This patch adds `--enable-debuginfo-tools` to
let one opt into having that debuginfo.
This is useful for debugging the tools in distro packages. We always
strip debuginfo into separate packages anyway, so the extra size is not
a concern in regular use.
This ensures that each build will support the testing design of "dry
running" builds. It's also checked that a dry run build is equivalent
step-wise to a "wet" run build; the graphs we generate when running are
directly compared node/node and edge/edge, both for order and contents.
In order to run tests, previous commits have cfg'd out various parts of
rustbuild. Generally speaking, these are filesystem-related operations
and process-spawning related parts. Then, rustbuild is run "as normal"
and the various steps that where run are retrieved from the cache and
checked against the expected results.
Note that this means that the current implementation primarily tests
"what" we build, but doesn't actually test that what we build *will*
build. In other words, it doesn't do any form of dependency verification
for any crate. This is possible to implement, but is considered future
work.
This implementation strives to cfg out as little code as possible; it
also does not currently test anywhere near all of rustbuild. The current
tests are also not checked for "correctness," rather, they simply
represent what we do as of this commit, which may be wrong.
Test cases are drawn from the old implementation of rustbuild, though
the expected results may vary.
When the RUSTFLAGS_STAGE_{1,2} is not set, including a space means
the string will always be non-empty and RUSTFLAGS will be always be
reset which breaks other ways of setting these such as through config
in CARGO_HOME.
This commit is an implementation of adding custom sections to wasm artifacts in
rustc. The intention here is to expose the ability of the wasm binary format to
contain custom sections with arbitrary user-defined data. Currently neither our
version of LLVM nor LLD supports this so the implementation is currently custom
to rustc itself.
The implementation here is to attach a `#[wasm_custom_section = "foo"]`
attribute to any `const` which has a type like `[u8; N]`. Other types of
constants aren't supported yet but may be added one day! This should hopefully
be enough to get off the ground with *some* custom section support.
The current semantics are that any constant tagged with `#[wasm_custom_section]`
section will be *appended* to the corresponding section in the final output wasm
artifact (and this affects dependencies linked in as well, not just the final
crate). This means that whatever is interpreting the contents must be able to
interpret binary-concatenated sections (or each constant needs to be in its own
custom section).
To test this change the existing `run-make` test suite was moved to a
`run-make-fulldeps` folder and a new `run-make` test suite was added which
applies to all targets by default. This test suite currently only has one test
which only runs for the wasm target (using a node.js script to use `WebAssembly`
in JS to parse the wasm output).
This commit updates CI configuration to inform rustbuild that it should print
out how long each step takes on CI. This'll hopefully allow us to track the
duration of steps over time and follow regressions a bit more closesly (as well
as have closer analysis of differences between two builds).
cc #48829
Support extra-verbose builds
- The bootstrap crate currently passes -v to Cargo if itself invoked with -vv. But Cargo supports -vv (to show build script output), so make bootstrap pass that if itself invoked with -vvv. (More specifically, pass N '-v's to Cargo if invoked with N+1 of them.)
- bootstrap.py currently tries to pass on up to two '-v's to cargo when building bootstrap, but incorrectly ('-v' is marked as 'store_true', so argparse stores either False or True, ignoring multiple '-v's). Fix this, allow passing any number of '-v's, and make it consistent with bootstrap's invocation of Cargo (i.e. subtract one from the number of '-v's).
- Also improve bootstrap.py's config.toml 'parsing' to support arbitrary verbosity levels, + allow command line to override it.
- The bootstrap crate currently passes -v to Cargo if itself invoked
with -vv. But Cargo supports -vv (to show build script output), so make
bootstrap pass that if itself invoked with -vvv. (More specifically,
pass N '-v's to Cargo if invoked with N+1 of them.)
- bootstrap.py currently tries to pass on up to two '-v's to cargo when
building bootstrap, but incorrectly ('-v' is marked as 'store_true', so
argparse stores either False or True, ignoring multiple '-v's). Fix
this, allow passing any number of '-v's, and make it consistent with
bootstrap's invocation of Cargo (i.e. subtract one from the number of
'-v's).
- Also improve bootstrap.py's config.toml 'parsing' to support arbitrary
verbosity levels, + allow command line to override it.
rustbuild: Remove ThinLTO-related configuration
This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.
This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
Required moving all fulldeps tests depending on `rand` to different locations as
now there's multiple `rand` crates that can't be implicitly linked against.
This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.
This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
rustbuild: Pass `ccache` to build scripts
This is a re-attempt at #48192 hopefully this time with 100% less randomly
[blocking builds for 20 minutes][block]. To work around #48192 the sccache
server is started in the `run.sh` script very early on in the compilation
process.
[block]: https://github.com/rust-lang/rust/issues/48192
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
This is a re-attempt at #48192 hopefully this time with 100% less randomly
[blocking builds for 20 minutes][block]. To work around #48192 the sccache
server is started in the `run.sh` script very early on in the compilation
process.
[block]: https://github.com/rust-lang/rust/issues/48192
This re-introduces a `Config.libdir_relative` field, now derived from
`libdir` and made relative to `prefix` if necessary.
This fixes a regression from #46592 when `--libdir` is given an absolute
path. `Builder::sysroot_libdir` should always use a relative path so
its callers don't clobber system locations, and `librustc` also asserts
that `CFG_LIBDIR_RELATIVE` is really relative.
We'd previously assumed that these paths would be relative to the src
dir, and that for example our various CI scripts would, when calling
x.py, use `../x.py build ../src/tools/...` but this isn't the case --
they use `../x.py` without using the relevant source-relative path.
We eventually may want to make this (actually somewhat logical) change,
but this is not that time.
Primarily for CI purposes; this is intended to avoid cases where we
update rustbuild and unintentionally make CI stop running some builds to
the arguments being passed no longer applying for some reason.
Implement excluding a build-step via --exclude
First step to fixing https://github.com/rust-lang/rust/issues/47911. This doesn't change any CI configuration, but implements what I believe necessary to make that feasible in rustbuild.
In theory this should be sufficient to allow someone to open a PR against .travis.yml and appveyor.yml which splits the Windows 32-bit tests and maybe the OS X tests into multiple builders (depending on what our cost-concerns are) to reduce runtimes.
r? @alexcrichton
cc @kennytm
Some Steps are by-default run but don't have any paths associated with
them. We need to have at least one PathSet per each Step, though, so we
add an empty one on calls to `never()`.
Previously, a Step would be able to tell on its own when it was invoked
"by-default" (that is, `./x.py test` was called instead of `./x.py test
some/path`). This commit replaces that functionality, invoking each Step
with each of the paths it has specified as "should be invoked by."
For example, if a step calls `path("src/tools/cargo")` and
`path("src/doc/cargo")` then it's make_run will be called twice, with
"src/tools/cargo" and "src/doc/cargo." This makes it so that default
handling logic is in builder, instead of spread across various Steps.
However, this meant that some Step specifications needed to be updated,
since for example `rustdoc` can be built by `./x.py build
src/librustdoc` or `./x.py build src/tools/rustdoc`. A `PathSet`
abstraction is added that handles this: now, each Step can not only list
`path(...)` but also `paths(&[a, b, ...])` which will make it so that we
don't invoke it with each of the individual paths, instead invoking it
with the first path in the list (though this shouldn't be depended on).
Future work likely consists of implementing a better/easier way for a
given Step to work with "any" crate in-tree, especially those that want
to run tests, build, or check crates in the std, test, or rustc crate
trees. Currently this is rather painful to do as most of the logic is
duplicated across should_run and make_run. It seems likely this can be
abstracted away into builder somehow.
rustbuild: Pass `ccache` to build scripts
Right now the ccache setting is only used for LLVM, but this tweaks it to also
be used for build scripts so C++ builds like `librustc_llvm` can be a bit
speedier.
Right now the ccache setting is only used for LLVM, but this tweaks it to also
be used for build scripts so C++ builds like `librustc_llvm` can be a bit
speedier.
Dist builds should always be as fast as we can make them, and since
those run on CI we don't care quite as much for the build being somewhat
slower. As such, we don't automatically enable ThinLTO on builds for the
dist builders.
Right now this directory is located under:
$sysroot/lib/rustlib/$target/lib/codegen-backends
but after seeing what we do in a few other places it seems that a more
appropriate location would be:
$sysroot/lib/rustlib/$target/codegen-backends
so this commit moves it!
This currently only supports a limited subset of the full compilation,
but is likely 90% of what people will want and is possible without
building a full compiler (i.e., running LLVM). In theory, this means
that contributors who don't want to build LLVM now have an easy way to
compile locally, though running tests won't work.
Test rustdoc js
Add tests for the rustdoc search. It was heavily required because of all the recent breaking changes that happened while I went through improvements in doc search (add search in/for generic search for example).
