ci: Attempt to skip a full rustc compile on dist*
Currently when we're preparing cross-compiled compilers it can take
quite some time because we have to build the compiler itself three
different times. The first is the normal bootstrap, the second is a
second build for the build platform, and the third is the actual target
architecture compiler. The second compiler was historically built
exclusively for procedural macros, and long ago we didn't actually need
it.
This commit tries out avoiding that second compiled compiler, meaning we
only compile rustc for the build platform only once. Some local testing
shows that this is promising, but bors is of course the ultimate test!
This extends a test in the previous commit to assert that we don't build
extra rustc compilers even when the "extended" option is set to true.
This involved some internal refactoring to have more judicious usage of
`compiler_for`, added in the previous commit, as well.
Various `dist::*` targets were refactored to be parameterized with a
`Compiler` instead of a `stage`/`host`, and then the various parameters
within the `Extended` target were tweaked to ensure that we don't ever
accidentally ask for a stage2 build compiler when we're distributing
something.
This commit furthers the previous one to ensure that we don't build an
extra stage of the compiler in CI. A test has been added to rustbuild to
ensure that this doesn't regress, and then in debugging this test it was
hunted down that the `dist::Std` target was the one erroneously pulling
in the wrong compiler.
The `dist::Std` step was updated to instead account for the "full
bootstrap" or not flag, ensuring that the correct compiler for compiling
the final standard library was used. This was another use of the
`force_use_stage1` function which was in theory supposed to be pretty
central, so existing users were all evaluated and a new function,
`Builder::compiler_for`, was introduced. All existing users of
`force_use_stage1` have been updated to use `compiler_for`, where the
semantics of `compiler_for` are similar to that of `compiler` except
that it doesn't guarantee the presence of a sysroot for the arguments
passed (as they may be modified).
Perhaps one day we can unify `compiler` and `compiler_for`, but the
usage of `Builder::compiler` is so ubiquitous it would take quite some
time to evaluate whether each one needs the sysroot or not, so it's
hoped that can be done in parallel.
Rework how bootstrap tools are built
This makes bootstrap tools buildable and testable in stage 0 with the downloaded bootstrap compiler, futhermore, it makes it such that they cannot be built in any other stage.
Notably, this will also mean that compiletest may need to wait a cycle before it can use changes to `libtest`, as it no longer depends on the in-tree libtest.
Introduce assembly tests suite
The change introduces a new test suite - **Assembly** tests. The motivation behind this is an ability to perform end-to-end codegen testing with LLVM backend. Turned out, NVPTX backend sometimes missing common Rust features (`i128` and libcalls in the past, and still full atomics support) due to different reasons.
Prior to this change, basic NVPTX assembly tests were implemented within `run-make` suite. Now, it's easier to write additional and maintain existing tests for the target.
cc @gnzlbg @peterhj
cc @eddyb I adjusted mangling scheme expectation, so there is no need to change the tests for #57967
Fixes rustdoc in stage 0, stage 1
When a request for rustdoc is passed for stage 0, x.py build --stage 0
src/tools/rustdoc or ensure(tool::Rustdoc { .. }) with top_stage = 0, we
return the rustdoc for that compiler (i.e., the beta rustdoc).
This fixes stage 0 of https://github.com/rust-lang/rust/issues/52186 as well as being part of general workflow improvements (making stage 0 testing for std work) for rustbuild.
The stage 1 fix (second commit) completely resolves the problem, so this fixes https://github.com/rust-lang/rust/issues/52186.
The RUSTDOC_LIBDIR should be rustc_libdir, not sysroot_libdir; rustdoc
is like the compiler and should link against rustc's libdir.
Some people currently (i.e., in general, may not be on master) have doc
tests working, but no attempt to determine why has been attempted.
Using CC, CFLAGS, CXX, CXXFLAGS, AR and RANLIB breaks cross compilation
because host is built first and has correct values. The same
values are incorrect for the target however.
This commit works around the newly-introduced LLVM shared library.
This is needed such that llvm-config run from
librustc_llvm's build script can correctly locate it's own LLVM, not the
one in stage0/lib. The LLVM build system uses the DT_RUNPATH/RUNPATH
header within the llvm-config binary, which we want to use, but because
Cargo always adds the host compiler's "libdir" (stage0/lib in our
case) to the dynamic linker's search path, we weren't properly finding
the freshly-built LLVM in llvm/lib. By restoring the environment
variable setting the search path to what bootstrap sees, the problem is
resolved and librustc_llvm correctly links and finds the appropriate
LLVM.
Several run-make-fulldeps tests are also updated with similar handling.
Use `clear_if_dirty` on std for backend changes, just as we do for
changes to rustc itself, so new codegen is correctly applied to all
later compiler stages.
Fixes#48298.
The (currently) single unit test of the compiletest tool was never
executed on CI. At least I couldn't find any references of it in the
logs. This adds a test suite for compiletest so that our tester is
tested, too.
The compiletest tests can then also be executed with:
./x.py test src/tools/compiletest
Add index page argument
@Mark-Simulacrum: I might need some help from you: in bootstrap, I want to add an argument (a new flag added into `rustdoc`) in order to generate the current index directly when `rustdoc` is documenting the `std` lib. However, my change in `bootstrap` didn't do it and I assume it must be moved inside the `Std` struct. But there, I don't see how to pass it to `rustdoc` through `cargo`. Did I miss anything?
r? @QuietMisdreavus
* Make it influence the behavior of the compiled rustc, rather than
just the rustc build system. That is, if verify_llvm_ir=true,
even manual invocations of the built rustc will verify LLVM IR.
* Enable verification of LLVM IR in CI, for non-deploy and
deploy-alt builds. This is similar to how LLVM assertions are
handled.
Clean up dependency tracking in Rustbuild [2/2]
Make `clear_if_dirty` calls in `Builder::cargo` with stamp dependencies for the given Mode.
Continuation of #50904
Ref issue #50509
r? @Mark-Simulacrum
This commit updates the debuginfo that is encoded in all of our released
artifacts by default. Currently it has paths like `/checkout/src/...` but these
are a little inconsistent and have changed over time. This commit instead
attempts to actually define the file paths in our debuginfo to be consistent
between releases.
All debuginfo paths are now intended to be `/rustc/$sha` where `$sha` is the git
sha of the released compiler. Sub-paths are all paths into the git repo at that
`$sha`.
Options:
-h, --help Display this message
--cfg SPEC Configure the compilation environment
-L [KIND=]PATH Add a directory to the library search path. The
optional KIND can be one of dependency, crate, native,
framework or all (the default).
-l [KIND=]NAME Link the generated crate(s) to the specified native
library NAME. The optional KIND can be one of static,
dylib, or framework. If omitted, dylib is assumed.
--crate-type [bin|lib|rlib|dylib|cdylib|staticlib|proc-macro]
Comma separated list of types of crates for the
compiler to emit
--crate-name NAME
Specify the name of the crate being built
--emit [asm|llvm-bc|llvm-ir|obj|metadata|link|dep-info|mir]
Comma separated list of types of output for the
compiler to emit
--print [crate-name|file-names|sysroot|cfg|target-list|target-cpus|target-features|relocation-models|code-models|tls-models|target-spec-json|native-static-libs]
Comma separated list of compiler information to print
on stdout
-g Equivalent to -C debuginfo=2
-O Equivalent to -C opt-level=2
-o FILENAME Write output to <filename>
--out-dir DIR Write output to compiler-chosen filename in <dir>
--explain OPT Provide a detailed explanation of an error message
--test Build a test harness
--target TARGET Target triple for which the code is compiled
-W, --warn OPT Set lint warnings
-A, --allow OPT Set lint allowed
-D, --deny OPT Set lint denied
-F, --forbid OPT Set lint forbidden
--cap-lints LEVEL
Set the most restrictive lint level. More restrictive
lints are capped at this level
-C, --codegen OPT[=VALUE]
Set a codegen option
-V, --version Print version info and exit
-v, --verbose Use verbose output
Additional help:
-C help Print codegen options
-W help Print 'lint' options and default settings
--help -v Print the full set of options rustc accepts command.
This optionally adds lldb (and clang, which it needs) to the build.
Because rust uses LLVM 7, and because clang 7 is not yet released, a
recent git master version of clang is used.
The lldb that is used includes the Rust plugin.
lldb is only built when asked for, or when doing a nightly build on
macOS. Only macOS is done for now due to difficulties with the Python
dependency.
If we detect a local rebuild (e.g. bootstrap compiler is the same version as target compiler), we set stage to 1.
When trying to build e.g. UnstableBook, we use Mode::ToolBootstrap and stage is 1.
Just allow Mode::ToolBootstrap and stagge != 0 if we are in a local_rebuild
Signed-off-by: Marc-Antoine Perennou <Marc-Antoine@Perennou.com>
Allow clippy to be installed with make install
After #51122 clippy is available as a component but doesn't install when building from source.
This PR allows to install clippy with extended tools.
Previously we'd only do so for stage 0 but with keep-stage
improvements it seems likely that we'll see more developers working in
the stage 1, so we should allow enabling incremental for them.
Ideally, the check we probably want is to only enable incremental for
the last compiler build scheduled, but there's no good way to do so
today. Just enabling incremental in all stages should be sufficient;
we may be doing extra work that's needles -- compiling incrementally
something that will never be recompiled in-place -- but that should be
sufficiently unlikely (i.e., users either don't care or won't be
compiling the compiler twice).
Disable LLVM verification by default
Currently -Z no-verify only controls IR verification prior to LLVM codegen, while verification is performed unconditionally both before and after linking with (Thin)LTO.
Also wondering what the sentiment is on disabling verification by default (and e.g. only enabling it on ALT builds with assertions). This does not seem terribly useful outside of rustc development and it does seem to show up in profiles (at something like 3%).
**EDIT:** A table showing the various configurations and what is enabled when.
| Configuration | Dynamic verification performed | LLVM static assertions compiled in |
| --- | --- | --- |
| alt builds | | yes |
| nightly builds | | no |
| stable builds | | no |
| CI builds | | |
| dev builds in a checkout | | |
This commit updates the stage0 build of tools to use the libraries of the stage0
compiler instead of the compiled libraries by the stage0 compiler. This should
enable us to avoid any stage0 hacks (like missing SIMD).
ship LLVM tools with the toolchain
this PR adds llvm-{nm,objcopy,objdump,size} to the rustc sysroot (right next to LLD)
this slightly increases the size of the rustc component. I measured these numbers on x86_64 Linux:
- rustc-1.27.0-dev-x86_64-unknown-linux-gnu.tar.gz 180M -> 193M (+7%)
- rustc-1.27.0-dev-x86_64-unknown-linux-gnu.tar.xz 129M -> 137M (+6%)
r? @alexcrichton
cc #49584
Since they are unlikely to fail and are almost never going to fail
except with bootstrap changes (which would be tested locally anyway) it
makes sense to run these tests close to last.
Switch to bootstrapping from 1.27
It's possible the Float trait could be removed from core, but I couldn't tell whether it was intended to be removed or not. @SimonSapin may be able to comment more here; we can presumably also do that in a follow up PR as this one is already quite large.
Issue #50786 shows a case with local rebuild where the libraries built
by stage0 had the same suffix as stage0's own, and were accidentally
loaded by that stage0 rustc when compiling `librustc_trans`.
Now we set `__CARGO_DEFAULT_LIB_METADATA` to "bootstrap" during stage0,
rather than the release channel like usual, so the library suffix will
always be completely distinct from the stage0 compiler.
Use the correct crt*.o files when linking musl targets.
This is supposed to support optionally using the system copy of musl
libc instead of the included one if supported. This currently only
affects the start files, which is enough to allow building rustc on musl
targets.
Most of the changes are analogous to crt-static.
Excluding the start files is something musl based distributions usually patch into their copy of rustc:
- https://github.com/alpinelinux/aports/blob/eb064c8/community/rust/musl-fix-linux_musl_base.patch
- https://github.com/voidlinux/void-packages/blob/77400fc/srcpkgs/rust/patches/link-musl-dynamically.patch
For third-party distributions that not yet carry those patches it would be nice if it was supported without the need to patch upstream sources.
## Reasons
### What breaks?
Some start files were missed when originally writing the logic to swap in musl start files (gcc comes with its own start files, which are suppressed by -nostdlib, but not manually included later on). This caused #36710, which also affects rustc with the internal llvm copy or any other system libraries that need crtbegin/crtend.
### How is it fixed?
The system linker already has all the logic to decide which start files to include, so we can just defer to it (except of course if it doesn't target musl).
### Why is it optional?
In #40113 it was first tried to remove the start files, which broke compiling musl-targeting static binaries with a glibc-targeting compiler. This is why it eventually landed without removing the start files. Being an option side-steps the issue.
### Why are the start files still installed?
This has the nice side-effect, that the produced rust-std-* binaries can still be used by on a glibc-targeting system with a rustc built against glibc.
## Does it work?
With the following build script (using [musl-cross-make](https://github.com/richfelker/musl-cross-make)): https://shadowice.org/~mixi/rust-musl/build.sh, I was able to cross-compile a musl-host musl-targeting rustc on a glibc-based system. The resulting binaries are at https://shadowice.org/~mixi/rust-musl/binaries/. This also requires #50103 and #50104 (which are also applied to the branch the build script uses).
./x.py test should be able to run individual tests
Allows user to be able to run individual tests by specifying filename i.e `./x.py test src/test/run-pass/foo.rs`
Fixes#48483
Currently on CI we predominately compile LLVM with the default system compiler
which means gcc on Linux, some version of Clang on OSX, MSVC on Windows, and
gcc on MinGW. This commit switches Linux, OSX, and Windows to all use Clang
6.0.0 to build LLVM (aka the C/C++ compiler as part of the bootstrap). This
looks to generate faster code according to #49879 which translates to a faster
rustc (as LLVM internally is faster)
The major changes here were to the containers that build Linux releases,
basically adding a new step that uses the previous gcc 4.8 compiler to compile
the next Clang 6.0.0 compiler. Otherwise the OSX and Windows scripts have been
updated to download precompiled versions of Clang 6 and configure the build to
use them.
Note that `cc` was updated here to fix using `clang-cl` with `cc-rs` on MSVC, as
well as an update to `sccache` on Windows which was needed to correctly work
with `clang-cl`. Finally the MinGW compiler is entirely left out here
intentionally as it's currently thought that Clang can't generate C++ code for
MinGW and we need to use gcc, but this should be verified eventually.
trim and pass relative test paths as test-args
use collect for getting test_args
move suite_path to ShouldRun and make Option
append existing args to test_args
use enum for PathSet
handle Suites differently from Paths
Error out if part of test suite but not file
refactor, make requested changes
rustbuild: Allow quick testing of libstd and libcore at stage0
This PR implemented two features:
1. Added a `--no-doc` flag to allow testing a crate *without* doc tests. In this mode, we don't need to build rustdoc, and thus we can skip building the stage2 compiler. (Ideally stage0 test should use the bootstrap rustdoc, but I don't want to mess up the core builder logic here)
2. Moved all libcore tests externally and added a tidy test to ensure we don't accidentally add `#[test]` into libcore.
After this PR, one could run `./x.py test --stage 0 --no-doc src/libstd` to test `libstd` without building the compiler, thus enables us to quickly test new library features.
Add "the Rustc book"
This PR introduces a new book into the documentation, "The rustc book". We already have books for Cargo, and for Rustdoc, rustc should have some too. This book is focused on *users* of rustc, and provides a nice place to write documentation for users.
I haven't put content here, but plan on scaffolding it out very soon, and wanted this PR open for a few discussions first. One of those is "what exactly should said TOC be?" I plan on having a proposed one up tomorrow, but figured I'd let people know to start thinking about it now.
The big one is that we also will want to put https://github.com/rust-lang-nursery/rustc-guide in-tree as well, and the naming is... tough. I'm proposing:
* doc.rust-lang.org/rustc is "The Rustc book", to mirror the other tools' books.
* doc.rust-lang.org/rustc-contribution is "The Rustc contribution guide", and contains that book
@nikomatsakis et al, any thoughts on this? I'm not attached to it in particular, but had to put something together to get this discussion going. I think mirroring the other tools is a good idea for this work, but am not sure where exactly that leaves yours.
Fixes https://github.com/rust-docs/team/issues/11
Add rustc_trans to x.py check
r? @Mark-Simulacrum
I looked at `bootstrap/compile.rs` and `bootstrap/check.rs` to try to work out which steps were appropriate, but I'm sure I've overlooked some details here, so it's worth checking carefully I've got all the steps right (e.g. I wasn't sure whether we want to build LLVM if necessary with `x.py check`, though I thought it was probably better to than to not).
From a quick test, it seems to be working, though.
Only emit save-analysis data for `cargo build` tasks
Previously, we were emittinng analysis data for all tasks, including `doc`. That meant we got two sets of save-analysis data, one from the normal build and one from the docs. That means indexing with the RLS took twice as long and made downloads larger and build times longer.
cc https://github.com/rust-lang-nursery/rls/issues/826
r? @Mark-Simulacrum
Previously, we were emittinng analysis data for all tasks, including `doc`. That meant we got two sets of save-analysis data, one from the normal build and one from the docs. That means indexing with the RLS took twice as long and made downloads larger and build times longer.
cc https://github.com/rust-lang-nursery/rls/issues/826
Debugging information for the extended tools is currently disabled for
concerns about the size. This patch adds `--enable-debuginfo-tools` to
let one opt into having that debuginfo.
This is useful for debugging the tools in distro packages. We always
strip debuginfo into separate packages anyway, so the extra size is not
a concern in regular use.
This ensures that each build will support the testing design of "dry
running" builds. It's also checked that a dry run build is equivalent
step-wise to a "wet" run build; the graphs we generate when running are
directly compared node/node and edge/edge, both for order and contents.
In order to run tests, previous commits have cfg'd out various parts of
rustbuild. Generally speaking, these are filesystem-related operations
and process-spawning related parts. Then, rustbuild is run "as normal"
and the various steps that where run are retrieved from the cache and
checked against the expected results.
Note that this means that the current implementation primarily tests
"what" we build, but doesn't actually test that what we build *will*
build. In other words, it doesn't do any form of dependency verification
for any crate. This is possible to implement, but is considered future
work.
This implementation strives to cfg out as little code as possible; it
also does not currently test anywhere near all of rustbuild. The current
tests are also not checked for "correctness," rather, they simply
represent what we do as of this commit, which may be wrong.
Test cases are drawn from the old implementation of rustbuild, though
the expected results may vary.
When the RUSTFLAGS_STAGE_{1,2} is not set, including a space means
the string will always be non-empty and RUSTFLAGS will be always be
reset which breaks other ways of setting these such as through config
in CARGO_HOME.
This commit is an implementation of adding custom sections to wasm artifacts in
rustc. The intention here is to expose the ability of the wasm binary format to
contain custom sections with arbitrary user-defined data. Currently neither our
version of LLVM nor LLD supports this so the implementation is currently custom
to rustc itself.
The implementation here is to attach a `#[wasm_custom_section = "foo"]`
attribute to any `const` which has a type like `[u8; N]`. Other types of
constants aren't supported yet but may be added one day! This should hopefully
be enough to get off the ground with *some* custom section support.
The current semantics are that any constant tagged with `#[wasm_custom_section]`
section will be *appended* to the corresponding section in the final output wasm
artifact (and this affects dependencies linked in as well, not just the final
crate). This means that whatever is interpreting the contents must be able to
interpret binary-concatenated sections (or each constant needs to be in its own
custom section).
To test this change the existing `run-make` test suite was moved to a
`run-make-fulldeps` folder and a new `run-make` test suite was added which
applies to all targets by default. This test suite currently only has one test
which only runs for the wasm target (using a node.js script to use `WebAssembly`
in JS to parse the wasm output).
This commit updates CI configuration to inform rustbuild that it should print
out how long each step takes on CI. This'll hopefully allow us to track the
duration of steps over time and follow regressions a bit more closesly (as well
as have closer analysis of differences between two builds).
cc #48829
Support extra-verbose builds
- The bootstrap crate currently passes -v to Cargo if itself invoked with -vv. But Cargo supports -vv (to show build script output), so make bootstrap pass that if itself invoked with -vvv. (More specifically, pass N '-v's to Cargo if invoked with N+1 of them.)
- bootstrap.py currently tries to pass on up to two '-v's to cargo when building bootstrap, but incorrectly ('-v' is marked as 'store_true', so argparse stores either False or True, ignoring multiple '-v's). Fix this, allow passing any number of '-v's, and make it consistent with bootstrap's invocation of Cargo (i.e. subtract one from the number of '-v's).
- Also improve bootstrap.py's config.toml 'parsing' to support arbitrary verbosity levels, + allow command line to override it.
- The bootstrap crate currently passes -v to Cargo if itself invoked
with -vv. But Cargo supports -vv (to show build script output), so make
bootstrap pass that if itself invoked with -vvv. (More specifically,
pass N '-v's to Cargo if invoked with N+1 of them.)
- bootstrap.py currently tries to pass on up to two '-v's to cargo when
building bootstrap, but incorrectly ('-v' is marked as 'store_true', so
argparse stores either False or True, ignoring multiple '-v's). Fix
this, allow passing any number of '-v's, and make it consistent with
bootstrap's invocation of Cargo (i.e. subtract one from the number of
'-v's).
- Also improve bootstrap.py's config.toml 'parsing' to support arbitrary
verbosity levels, + allow command line to override it.
rustbuild: Remove ThinLTO-related configuration
This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.
This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
Required moving all fulldeps tests depending on `rand` to different locations as
now there's multiple `rand` crates that can't be implicitly linked against.
This commit removes some ThinLTO/codegen unit cruft primarily only needed during
the initial phase where we were adding ThinLTO support to rustc itself. The
current bootstrap compiler knows about ThinLTO and has it enabled by default for
multi-CGU builds which are also enabled by default. One CGU builds (aka
disabling ThinLTO) can be achieved by configuring the number of codegen units to
1 for a particular builds.
This also changes the defaults for our dist builders to go back to multiple
CGUs. Unfortunately we're seriously bleeding for cycle time on the bots right
now so we need to recover any time we can.
rustbuild: Pass `ccache` to build scripts
This is a re-attempt at #48192 hopefully this time with 100% less randomly
[blocking builds for 20 minutes][block]. To work around #48192 the sccache
server is started in the `run.sh` script very early on in the compilation
process.
[block]: https://github.com/rust-lang/rust/issues/48192
This commit imports the LLD project from LLVM to serve as the default linker for
the `wasm32-unknown-unknown` target. The `binaryen` submoule is consequently
removed along with "binaryen linker" support in rustc.
Moving to LLD brings with it a number of benefits for wasm code:
* LLD is itself an actual linker, so there's no need to compile all wasm code
with LTO any more. As a result builds should be *much* speedier as LTO is no
longer forcibly enabled for all builds of the wasm target.
* LLD is quickly becoming an "official solution" for linking wasm code together.
This, I believe at least, is intended to be the main supported linker for
native code and wasm moving forward. Picking up support early on should help
ensure that we can help LLD identify bugs and otherwise prove that it works
great for all our use cases!
* Improvements to the wasm toolchain are currently primarily focused around LLVM
and LLD (from what I can tell at least), so it's in general much better to be
on this bandwagon for bugfixes and new features.
* Historical "hacks" like `wasm-gc` will soon no longer be necessary, LLD
will [natively implement][gc] `--gc-sections` (better than `wasm-gc`!) which
means a postprocessor is no longer needed to show off Rust's "small wasm
binary size".
LLD is added in a pretty standard way to rustc right now. A new rustbuild target
was defined for building LLD, and this is executed when a compiler's sysroot is
being assembled. LLD is compiled against the LLVM that we've got in tree, which
means we're currently on the `release_60` branch, but this may get upgraded in
the near future!
LLD is placed into rustc's sysroot in a `bin` directory. This is similar to
where `gcc.exe` can be found on Windows. This directory is automatically added
to `PATH` whenever rustc executes the linker, allowing us to define a `WasmLd`
linker which implements the interface that `wasm-ld`, LLD's frontend, expects.
Like Emscripten the LLD target is currently only enabled for Tier 1 platforms,
notably OSX/Windows/Linux, and will need to be installed manually for compiling
to wasm on other platforms. LLD is by default turned off in rustbuild, and
requires a `config.toml` option to be enabled to turn it on.
Finally the unstable `#![wasm_import_memory]` attribute was also removed as LLD
has a native option for controlling this.
[gc]: https://reviews.llvm.org/D42511
This is a re-attempt at #48192 hopefully this time with 100% less randomly
[blocking builds for 20 minutes][block]. To work around #48192 the sccache
server is started in the `run.sh` script very early on in the compilation
process.
[block]: https://github.com/rust-lang/rust/issues/48192
This re-introduces a `Config.libdir_relative` field, now derived from
`libdir` and made relative to `prefix` if necessary.
This fixes a regression from #46592 when `--libdir` is given an absolute
path. `Builder::sysroot_libdir` should always use a relative path so
its callers don't clobber system locations, and `librustc` also asserts
that `CFG_LIBDIR_RELATIVE` is really relative.
We'd previously assumed that these paths would be relative to the src
dir, and that for example our various CI scripts would, when calling
x.py, use `../x.py build ../src/tools/...` but this isn't the case --
they use `../x.py` without using the relevant source-relative path.
We eventually may want to make this (actually somewhat logical) change,
but this is not that time.
Primarily for CI purposes; this is intended to avoid cases where we
update rustbuild and unintentionally make CI stop running some builds to
the arguments being passed no longer applying for some reason.
Implement excluding a build-step via --exclude
First step to fixing https://github.com/rust-lang/rust/issues/47911. This doesn't change any CI configuration, but implements what I believe necessary to make that feasible in rustbuild.
In theory this should be sufficient to allow someone to open a PR against .travis.yml and appveyor.yml which splits the Windows 32-bit tests and maybe the OS X tests into multiple builders (depending on what our cost-concerns are) to reduce runtimes.
r? @alexcrichton
cc @kennytm
Some Steps are by-default run but don't have any paths associated with
them. We need to have at least one PathSet per each Step, though, so we
add an empty one on calls to `never()`.
Previously, a Step would be able to tell on its own when it was invoked
"by-default" (that is, `./x.py test` was called instead of `./x.py test
some/path`). This commit replaces that functionality, invoking each Step
with each of the paths it has specified as "should be invoked by."
For example, if a step calls `path("src/tools/cargo")` and
`path("src/doc/cargo")` then it's make_run will be called twice, with
"src/tools/cargo" and "src/doc/cargo." This makes it so that default
handling logic is in builder, instead of spread across various Steps.
However, this meant that some Step specifications needed to be updated,
since for example `rustdoc` can be built by `./x.py build
src/librustdoc` or `./x.py build src/tools/rustdoc`. A `PathSet`
abstraction is added that handles this: now, each Step can not only list
`path(...)` but also `paths(&[a, b, ...])` which will make it so that we
don't invoke it with each of the individual paths, instead invoking it
with the first path in the list (though this shouldn't be depended on).
Future work likely consists of implementing a better/easier way for a
given Step to work with "any" crate in-tree, especially those that want
to run tests, build, or check crates in the std, test, or rustc crate
trees. Currently this is rather painful to do as most of the logic is
duplicated across should_run and make_run. It seems likely this can be
abstracted away into builder somehow.
rustbuild: Pass `ccache` to build scripts
Right now the ccache setting is only used for LLVM, but this tweaks it to also
be used for build scripts so C++ builds like `librustc_llvm` can be a bit
speedier.
Right now the ccache setting is only used for LLVM, but this tweaks it to also
be used for build scripts so C++ builds like `librustc_llvm` can be a bit
speedier.
Dist builds should always be as fast as we can make them, and since
those run on CI we don't care quite as much for the build being somewhat
slower. As such, we don't automatically enable ThinLTO on builds for the
dist builders.
Right now this directory is located under:
$sysroot/lib/rustlib/$target/lib/codegen-backends
but after seeing what we do in a few other places it seems that a more
appropriate location would be:
$sysroot/lib/rustlib/$target/codegen-backends
so this commit moves it!
This currently only supports a limited subset of the full compilation,
but is likely 90% of what people will want and is possible without
building a full compiler (i.e., running LLVM). In theory, this means
that contributors who don't want to build LLVM now have an easy way to
compile locally, though running tests won't work.
Test rustdoc js
Add tests for the rustdoc search. It was heavily required because of all the recent breaking changes that happened while I went through improvements in doc search (add search in/for generic search for example).
* Bump the release version to 1.25
* Bump the bootstrap compiler to the recent beta
* Allow using unstable rustdoc features on beta - this fix has been applied to
the beta branch but needed to go to the master branch as well.
Run the miri test suite on the aux builder and travis
Reopen of #38350
see https://github.com/rust-lang/rust/pull/43340#issuecomment-316940762 for earlier discussion
Rationale for running miri's test suite in rustc's CI is that miri currently contains many features that we want in const eval in the future, and these features would break if the test suite is not run.
fixes#44077
r? @nikomatsakis
cc @eddyb
