Set `download-ci-llvm = "if-available"` by default when `channel = dev`
See https://github.com/rust-lang/compiler-team/issues/566. The motivation for changing the default is to avoid downloading and building LLVM when someone runs `x build` before running `x setup`. The motivation for only doing it on `channel = "dev"` is to avoid breaking distros or users installing from source. It works because `dev` is also the default channel.
The diff looks larger than it is; most of it is moving the `llvm` branch below the `rust` so `config.channel` is set.
r? `@Mark-Simulacrum` cc `@oli-obk` `@bjorn3` `@cuviper`
See https://github.com/rust-lang/compiler-team/issues/566.
The motivation for changing the default is to avoid downloading and building LLVM when someone runs `x build` before running `x setup`.
The motivation for only doing it on `channel = "dev"` is to avoid breaking distros or users installing from source. It works because `dev` is also the default channel.
The diff looks larger than it is; most of it is moving the `llvm` branch below the `rust` so `config.channel` is set.
Usually, we do want to use the static C++ library when building rustc_llvm, but do not want to have that dependency at compiler runtime. Change the defaults to Make It So.
This reverts commit 3acb505ee5
(PR #101833).
The changes in this commit caused several bugs or at least
incompatibilies. For now we're reverting this commit and will re-land it
alongside fixes for those bugs.
fix: use git-commit-info for version information
Fixes#33286.
Fixes#86587.
This PR changes the current `git-commit-hash` file that `./x.py` dist puts in the `rustc-{version}-src.tar.{x,g}z` to contain the hash, the short hash, and the commit date from which the tarball was created, assuming git was available when it was. It uses this for reading the version so that rustc has all the appropriate metadata.
# Testing
Testing this is kind of a pain. I did it with something like
```sh
./x.py dist # ensure that `ignore-git` is `false` in config.toml
cp ./build/dist/rustc-1.65.0-dev-src.tar.gz ../rustc-1.65.0-dev-src.tar.gz
cd .. && tar -xzf rustc-1.65.0-dev-src && cd rustc-1.65.0-dev-src
./x.py build
```
Then, the output of `rustc -vV` with the stage1 compiler should have the `commit-hash` and `commit-date` fields filled, rather than be `unknown`. To be completely sure, you can use `rustc --sysroot` with the stdlib that the original `./x.py dist` made, which will require that the metadata matches.
The build script for `compiler_builtins` doesn't support cross-compilation. I tried fixing it, but the cc crate itself
doesn't appear to support cross-compiling to windows either unless you use the -gnu toolchain:
```
error occurred: Failed to find tool. Is `lib.exe` installed?
```
Rather than trying to fix it or special-case the platforms without bugs,
make it opt-in instead of automatic.
bootstrap: Add llvm-has-rust-patches target option
This is so you can check out an upstream commit in src/llvm-project and
have everything just work.
This simplifies the logic in `is_rust_llvm` a bit; it doesn't need to
check for download-ci-llvm because we would have already errored if both
that and llvm-config were specified on the host platform.
Add build metrics to rustbuild
This PR adds a new module of rustbuild, `ci_profiler`, whose job is to gather as much information as possible about the CI build as possible and store it in a JSON file uploaded to `ci-artifacts`. Right now for each step it collects:
* Type name and debug representation of the `Step` object.
* Duration of the step (excluding child steps).
* Systemwide CPU stats for the duration of the step (both single core and all cores).
* Which child steps were executed.
This is capable of replacing both the scripts to collect CPU stats and the `[TIMING]` lines in build logs (not yet removed, until we port our tooling to use the CI profiler). The format is also extensible to be able in the future to collect more information.
r? `@Mark-Simulacrum`
I don't know why anyone would turn this off; doing so makes builds much slower (nearly a 60x slowdown according to #49057).
Remove the option to do so, which makes bootstrap a little easier to maintain.
Bootstrap continues to allow you to manage submodules manually by setting `submodules = false`.
This tool will generate a JSON file with statistics about each
individual step to disk. It will be used in rust-lang/rust's CI to
replace the mix of scripts and log scraping we currently have to gather
this data.
Previously, the static-libstdcpp setting was tied to llvm-tools such
that enabling the latter always enabled the latter. This seems
unfortunate, since it is entirely reasonable for someone to want to
_not_ statically link stdc++, but _also_ want to build the llvm-tools.
This patch therefore separates the two settings such that neither
implies the other.
On its own, that would change the default behavior in a way that's
likely to surprise users. Specifically, users who build llvm-tools
_likely_ want those tools to be statically compiled against libstdc++,
since otherwise users with older GLIBCXX will be unable to run the
vended tools. So, we also flip the default for the `static-libstdcpp`
setting such that builds always link statically against libstdc++ by
default, but it's _possible_ to opt out.
See also #94719.
It is currently hard coded to llvm if enabled and cranelift otherwise.
This made some sense when cranelift was the only alternative codegen
backend. Since the introduction of the gcc backend this doesn't make
much sense anymore. Before this PR bootstrapping rustc using a backend
other than llvm or cranelift required changing the source of
rustc_interface. With this PR it becomes a matter of putting the right
backend as first enabled backend in config.toml.
Support custom options for LLVM build
The LLVM build has a lot of options that rustbuild doesn't need to know about. We should allow the user to customize the LLVM build directly.
Here are some [example customizations][recipe] we'd like to do.
[recipe]: 90105e5e4e/recipes/contrib/clang_toolchain.py (579)
Continue supporting -Z instrument-coverage for compatibility for now,
but show a deprecation warning for it.
Update uses and documentation to use the -C option.
Move the documentation from the unstable book to stable rustc
documentation.
Make new symbol mangling scheme default for compiler itself.
As suggest in https://github.com/rust-lang/rust/pull/89917#issuecomment-945888574, this PR enables the new symbol mangling scheme for the compiler itself. The standard library is still compiled using the legacy mangling scheme so that the new symbol format does not show up in user code (yet).
r? `@Mark-Simulacrum`
I'm working on some LLVM patches in concert with a Rust patch, and it's
helping me quite a bit to have this as an option. It doesn't seem that
hard, so I figured I'd formalize it in x.py and send it upstream.
fix the stage0 tools config file path in `config.toml.example`
in #88362 , the `stage0.txt ` have been switched to `stage0.json` , but in `config.toml.example` the guide didn't change , this PR fix this issue
On NixOS systems, bootstrap will patch rustc used in bootstrapping after
checking `/etc/os-release` (to confirm the current distribution is NixOS).
However, when using Nix on a non-NixOS system, it can be desirable for
bootstrap to patch rustc. In this commit, a `patch-binaries-for-nix`
option is added to `config.toml`, which allows for user opt-in to
bootstrap's Nix patching.
Signed-off-by: David Wood <david.wood@huawei.com>
BPF target support
This adds `bpfel-unknown-none` and `bpfeb-unknown-none`, two new no_std targets that generate little and big endian BPF. The approach taken is very similar to the cuda target, where `TargetOptions::obj_is_bitcode` is enabled and code generation is done by the linker.
I added the targets to `dist-various-2`. There are [some tests](https://github.com/alessandrod/bpf-linker/tree/main/tests/assembly) in bpf-linker and I'm planning to add more. Those are currently not ran as part of rust CI.
Cleanup option parsing and config.toml.example
- Add an assertion that `link-shared = true` when `thin-lto = true`.
Previously, link-shared would be silently overwritten.
- Get rid of `Option<bool>` in bootstrap/config.rs. Set defaults
immediately instead of delaying until later in bootstrap. This makes
it easier to find what the default value is.
- Remove redundant `config.x = false` when the default was already false
- Set defaults for `bindir` in `default_opts()` instead of `parse()`
- Update `download-ci-llvm = if-supported` option to match bootstrap.py
- Remove redundant check for link_shared. Previously, it was checked twice.
- Update various options in config.toml.example to their defaults.
Previously, some options showed an example value instead of the
default value.
- Fix incorrect defaults in config.toml.example
+ `use-libcxx` defaults to false
+ Add missing `check-stage = 0`
+ Update several defaults to be conditional (e.g. `if incremental { 10 } else { 100 }`)
- Remove redundant defaults in prose
- Use the same comment for the default and target-dependent `musl-root`
- Fix typos
- Link to `cc_detect` for `cc` and `cxx`, since the logic is ... complicated.
- Update more defaults to better reflect how they actually get set
- Remove ignored `gpg-password-file` option
This stopped being used in
7704d35,
but was never removed from config.toml.
- Remove unused flags from `config.toml`
+ Disallow `infodir` and `localstatedir` in `config.toml`
+ Allow the flags in `./configure`, but give a warning that they will be
ignored.
+ Fix incorrect comment that `datadir` will be ignored.
Example output:
```
$ ./configure --set install.infodir=xxx
configure: processing command line
configure:
configure: install.infodir := xxx
configure: build.configure-args := ['--set', 'install.infodir=xxx']
warning: infodir will be ignored
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python /home/joshua/rustc3/x.py --help`
configure:
```
- Update CHANGELOG
cc https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/bootstrap.20defaults
- Add an assertion that `link-shared = true` when `thin-lto = true`.
Previously, link-shared would be silently overwritten.
- Get rid of `Option<bool>` in bootstrap/config.rs. Set defaults
immediately instead of delaying until later in bootstrap. This makes
it easier to find what the default value is.
- Remove redundant `config.x = false` when the default was already false
- Set defaults for `bindir` in `default_opts()` instead of `parse()`
- Update `download-ci-llvm = if-supported` option to match bootstrap.py
- Remove redundant check for link_shared. Previously, it was checked twice.
- Update various options in config.toml.example to their defaults.
Previously, some options showed an example value instead of the
default value.
- Fix incorrect defaults in config.toml.example
+ `use-libcxx` defaults to false
+ Add missing `check-stage = 0`
+ Update several defaults to be conditional (e.g. `if incremental { 10 } else { 100 }`)
- Remove redundant defaults in prose
- Use the same comment for the default and target-dependent `musl-root`
- Fix typos
- Link to `cc_detect` for `cc` and `cxx`, since the logic is ... complicated.
- Update more defaults to better reflect how they actually get set
- Remove ignored `gpg-password-file` option
This stopped being used in
7704d35acc,
but was never removed from config.toml.
- Remove unused flags from `config.toml`
+ Disallow `infodir` and `localstatedir` in `config.toml`
+ Allow the flags in `./configure`, but give a warning that they will be
ignored.
+ Fix incorrect comment that `datadir` will be ignored.
Example output:
```
$ ./configure --set install.infodir=xxx
configure: processing command line
configure:
configure: install.infodir := xxx
configure: build.configure-args := ['--set', 'install.infodir=xxx']
warning: infodir will be ignored
configure:
configure: writing `config.toml` in current directory
configure:
configure: run `python /home/joshua/rustc3/x.py --help`
configure:
```
- Update CHANGELOG
- Add "as an example" where appropriate
- Link to an issue instead of to ephemeral chats
Add `download-rustc = "if-unchanged"`
This allows keeping the setting to a fixed value without having to
toggle it when you want to work on the compiler instead of on tools.
This sets `BOOTSTRAP_DOWNLOAD_RUSTC` in bootstrap.py so rustbuild doesn't have to try and replicate its logic.
Helps with https://github.com/rust-lang/rust/issues/81930.
r? `@Mark-Simulacrum` cc `@camelid`
Adds bootstrap rules to support installing rust-demangler.
When compiling with `-Z instrument-coverage`, the coverage reports are
generated by `llvm-cov`. `llvm-cov` includes a built-in demangler for
C++, and an option to supply an alternate demangler. For Rust, we have
`rust-demangler`, currently used in `rustc` coverage tests.
Fuchsia's toolchain for Rust is built via `./x.py install`. Fuchsia is
adding support for Rust coverage, and we need to include the
`rust-demangler` in the installed `bin` directory.
Configured rust-demangler as an in-tree extended tool.
Added tests to support `./x.py test rust-demangler`.
Install with extended tools by default only if `profiler = true`.
