Support absolute source paths in bootstrap
Fixes https://github.com/rust-lang/rust/issues/126765
`x test [PATHS]` should work when each path
1. Is the name of a build step, such as `tidy` in `x test tidy` or
2. Points to an existing file that is a descendant of the builder's source directory (root of this repository).
Try to clarify the confusingly-named `RustDev` and `RustcDev` steps
When trying to track down how the downloaded LLVM gets built and bundled, I was greatly confused by these step names.
I wasn't sure whether I could just rename them (since they correspond to the filename of the resulting tarball), but I at least wanted to leave behind some signposts to reduce confusion.
Ignore `branch-protection-check-IBT` run-make test
The old Makefile implementation (#110304) had an improper comparison which caused the test to never run. However, both the updated Makefile implementation and the rmake implementation fail (missing `.note.gnu.property`). This could be a bug in the original implementation or test flakiness.
Edit: Manually recreating the test case shows that `.note.gnu.property` does not appear in nightly.
```rust
// main.rs
fn main() {
println!("hello world");
}
```
```sh
$ rustc +nightly -V
rustc 1.81.0-nightly (c1b336cb6 2024-06-21)
$ rustc +stable -V
rustc 1.79.0 (129f3b996 2024-06-10)
```
```sh
$ rustc +nightly -Zcf-protection=branch -Clink-args=-nostartfiles -Csave-temps "-L$PWD" main.rs -o main
$ llvm-readobj --elf-output-style=GNU -nW main
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000008 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: bcae34e6431b2a37
```
Compiling without the other flags still does not show `.note.gnu.property`.
```sh
$ rustc +nightly main.rs -o main
$ llvm-readobj --elf-output-style=GNU -nW main
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 4.4.0
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000008 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: d60d5f108b63bf3a
```
Compiling on stable shows `.note.gnu.property`.
```sh
$ rustc +stable main.rs -o main
$ llvm-readobj --elf-output-style=GNU -nW main
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0 (property note)
Properties: x86 ISA needed: x86-64-baseline
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 4a494eb578123314e6ff1caf1c8877e27004664f
Displaying notes found in: .note.ABI-tag
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 4.4.0
```
Part of #121876.
r? `@jieyouxu`
rustdoc: Add support for `missing_unsafe_on_extern` feature
Follow-up of https://github.com/rust-lang/rust/pull/124482.
Not sure if the `safe` keyword is supposed to be displayed or not though? For now I didn't add it in the generated doc, only `unsafe` as usual.
cc `@spastorino`
r? `@fmease`
Rollup of 3 pull requests
Successful merges:
- #126140 (Rename `std::fs::try_exists` to `std::fs::exists` and stabilize fs_try_exists)
- #126318 (Add a `x perf` command for integrating bootstrap with `rustc-perf`)
- #126552 (Remove use of const traits (and `feature(effects)`) from stdlib)
r? `@ghost`
`@rustbot` modify labels: rollup
Remove use of const traits (and `feature(effects)`) from stdlib
The current uses are already unsound because they are using non-const impls in const contexts. We can reintroduce them by reverting the commit in this PR, after #120639 lands.
Also, make `effects` an incomplete feature.
cc `@rust-lang/project-const-traits`
r? `@compiler-errors`
Add a `x perf` command for integrating bootstrap with `rustc-perf`
This PR adds a new `x perf` command to bootstrap. The idea is to let rustc developers profile (`profile_local`) and benchmark (`bench_local`) a stage1/stage2 compiler directly from within `rust`.
Before, if you wanted to use `rustc-perf`, you had to clone it, set it up, copy the `rustc` sysroot after every change to `rust` etc. This is an attempt to automate that.
I opened this PR mostly for discussion. My idea is to offer an interface that looks something like this (a random sample of commands):
```bash
x perf --stage 2 profile eprintln
x perf --stage1 profile cachegrind
x perf benchmark --id baseline
x perf benchmark --id after-edit
x perf cmp baseline after-edit
```
In this PR, I'd like to only implement the simplest case (`profile_local (eprintln)`), because that only requires a single sysroot (you don't compare anything), and it's relatively easy to set up. Also, I'd like to avoid forcing developers to deal with the rustc-perf UI, so more complex use-cases (like benchmarking two sysroots and comparing the results) should probably wait for https://github.com/rust-lang/rustc-perf/issues/1734 (which is hopefully coming along soon-ish).
I'm not sure if it's better to do this in bootstrap directly, or if I should create some shim tool that will receive a `rustc` sysroot, and offer a simplified CLI on top of `rustc-perf`.
## Why is a separate CLI needed?
We definitely need to add some support to bootstrap to automate preparing `rustc-perf` and the `rustc` sysroot, but in theory after that we could just let people invoke `rustc-perf` manually. While that is definitely possible, you'd need to manually figure out where is your sysroot located, which seems annoying to me. The `rustc-perf` CLI is also relatively complex, and for this use-case it makes sense to only use a subset of it. So I thought that it would be better to offer a simplified interface on top of it that would make life easier for contributors. But maybe it's not worth it.
CC `@onur-ozkan`
Generalize `{Rc,Arc}::make_mut()` to unsized types.
* `{Rc,Arc}::make_mut()` now accept any type implementing the new unstable trait `core::clone::CloneToUninit`.
* `CloneToUninit` is implemented for `T: Clone` and for `[T] where T: Clone`.
* `CloneToUninit` is a generalization of the existing internal trait `alloc::alloc::WriteCloneIntoRaw`.
* New feature gate: `clone_to_uninit`
This allows performing `make_mut()` on `Rc<[T]>` and `Arc<[T]>`, which was not previously possible.
---
Previous PR description, now obsolete:
> Add `{Rc, Arc}::make_mut_slice()`
>
> These functions behave identically to `make_mut()`, but operate on `Arc<[T]>` instead of `Arc<T>`.
>
> This allows performing the operation on slices, which was not previously possible because `make_mut()` requires `T: Clone` (and slices, being `!Sized`, do not and currently cannot implement `Clone`).
>
> Feature gate: `make_mut_slice`
try-job: test-various
This requires introducing a new internal type `RcUninit` (and
`ArcUninit`), which can own an `RcBox<T>` without requiring it to be
initialized, sized, or a slice. This is similar to `UniqueRc`, but
`UniqueRc` doesn't support the allocator parameter, and there is no
`UniqueArc`.
This trait allows cloning DSTs, but is unsafe to implement and use
because it writes to possibly-uninitialized memory which must be of the
correct size, and must initialize that memory.
It is only implemented for `T: Clone` and `[T] where T: Clone`, but
additional implementations could be provided for specific `dyn Trait`
or custom-DST types.
Bootstrap command refactoring: refactor `BootstrapCommand` (step 1)
This PR is a first step towards https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap.
It refactors `BoostrapCommand` to get it closer to a state where it is an actual command wrapper that can be routed through a central place of command execution, and also to make the distinction between printing output vs handling output programatically clearer (since now it's a mess).
The existing usages of `BootstrapCommand` are complicated primarily because of different ways of handling output. There are commands that:
1) Want to eagerly print stdout/stderr of the executed command, plus print an error message if the command fails (output mode `PrintAll`). Note that this error message attempts to print stdout/stderr of the command when `-v` is enabled, but that will always be empty, since this mode uses `.status()` and not `.output()`.
2) Want to eagerly print stdout/stderr of the executed command, but do not print any additional error message if it fails (output mode `PrintOutput`)
3) Want to capture stdout/stderr of the executed command, but print an error message if it fails (output mode `PrintFailure`). This means that the user wants to either ignore the output or handle it programatically, but that's not obvious from the name.
The difference between 1) and 2) (unless explicitly specified) is determined dynamically based on the bootstrap verbosity level.
It is very difficult for me to wrap my head around all these modes. I think that in a future PR, we should split these axes into e.g. this:
1) Do I want to handle the output programmatically or print it to the terminal? This should be a separate axis, true/false. (Note that "hiding the output" essentially just means saying that I handle it programmatically, and then I ignore the output).
2) Do I want to print a message if the command fails? Yes/No/Based on verbosity (which would be the default).
Then there is also the failure mode, but that is relatively simple to handle, the command execution will just shutdown bootstrap (either eagerly or late) when the command fails.
Note that this is just a first refactoring steps, there are a lot of other things to be done, so some things might not look "final" yet. The next steps are (not necessarily in this order):
- Remove `run` and `run_cmd` and implement everything in terms of `run_tracked` and rename `run_tracked` to `run`
- Implement the refactoring specified above (change how output modes work)
- Modify `BootstrapCmd` so that it stores `Command` instead of `&mut Command` and remove all the annoying `BootstrapCmd::from` by changing `Command::new` to `BootstrapCmd::new`
- Refactor the rest of command executions not currently using `BootstrapCmd` that can access Builder to use the correct output and failure modes. This will include passing Builder to additional places.
- Handle the most complex cases, such as output streaming. That will probably need to be handled separately.
- Refactor the rest of commands that cannot access builder (e.g. `Config::parse`) by introducing a new command context that will be passed to these places, and then stored in `Builder`. Move certain fields (such as `fail_fast`) from `Builder` to the command context.
- Handle the co-operation of `Builder`, `Build`, `Config` and command context. There are some fields and logic used during command execution that are distributed amongst `Builder/Build/Config`, so it will require some refactoring to make it work if the execution will happen on a separate place (in the command context).
- Refactor logging of commands, so that it is either logged to a file or printed in a nice hierarchical way that cooperates with the `Step` debug hierarchical output.
- Implement profiling of commands (add command durations to the command log, print a log of slowest commands and their execution counts at the end of bootstrap execution, perhaps store command executions to `metrics.json`).
- Implement caching of commands.
- Implement testing of commands through snapshot tests/mocking.
Best reviewed commit by commit.
r? ``@onur-ozkan``
Fix `...` in multline code-skips in suggestions
When we have long code skips, we write `...` in the line number gutter.
For suggestions, we were "centering" the `...` with the line, but that was inconsistent with what we do in every other case *and* off-center.
Add `f16` inline ASM support for 32-bit ARM
Adds `f16` inline ASM support for 32-bit ARM. SIMD vector types are taken from [here](https://developer.arm.com/architectures/instruction-sets/intrinsics/#f:`@navigationhierarchiesreturnbasetype=[float]&f:@navigationhierarchieselementbitsize=[16]&f:@navigationhierarchiesarchitectures=[A32]).`
Relevant issue: #125398
Tracking issue: #116909
`@rustbot` label +F-f16_and_f128
Stop using `unlikely` in `strict_*` methods
The `strict_*` methods don't need (un)likely, because the `overflow_panic` calls are all `#[cold]`, [meaning](https://llvm.org/docs/LangRef.html#function-attributes) that LLVM knows any branch to them is unlikely without us needing to say so.
r? libs
Rollup of 7 pull requests
Successful merges:
- #126530 (Add `f16` inline ASM support for RISC-V)
- #126712 (Migrate `relocation-model`, `error-writing-dependencies` and `crate-name-priority` `run-make` tests to rmake)
- #126722 (Add method to get `FnAbi` of function pointer)
- #126787 (Add direct accessors for memory addresses in `Machine` (for Miri))
- #126798 ([fuchsia-test-runner] Remove usage of kw_only)
- #126809 (Remove stray `.` from error message)
- #126811 (Add a tidy rule to check that fluent messages and attrs don't end in `.`)
r? `@ghost`
`@rustbot` modify labels: rollup
Add a tidy rule to check that fluent messages and attrs don't end in `.`
This adds a new dependency on `fluent-parse` to `tidy` -- we already rely on it in rustc so I feel like it's not that big of a deal.
This PR also adjusts many error messages that currently end in `.`; not all of them since I added an `ALLOWLIST`, excluded `rustc_codegen_*` ftl files, and `.teach_note` attributes.
r? ``@estebank`` ``@oli-obk``
Migrate `relocation-model`, `error-writing-dependencies` and `crate-name-priority` `run-make` tests to rmake
Part of #121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html).
Needs MSVC try-job due to #28026, almost guaranteed to fail, but let's see anyways.
try-job: aarch64-gnu
`/* try-job: x86_64-msvc */`
try-job: x86_64-apple-1
try-job: armhf-gnu
try-job: test-various
Add `f16` inline ASM support for RISC-V
This PR adds `f16` inline ASM support for RISC-V. A `FIXME` is left for `f128` support as LLVM does not support the required `Q` (Quad-Precision Floating-Point) extension yet.
Relevant issue: #125398
Tracking issue: #116909
`@rustbot` label +F-f16_and_f128
Add PidFd::{kill, wait, try_wait}
#117957 changed `Child` kill/wait/try_wait to use its pidfd instead of the pid, when one is available.
This PR extracts those implementations and makes them available on `PidFd` directly.
The `PidFd` implementations differ significantly from the corresponding `Child` methods:
* the methods can be called after the child has been reaped, which will result in an error but will be safe. This state is not observable in `Child` unless something stole the zombie child
* the `ExitStatus` is not kept, meaning that only the first time a wait succeeds it will be returned
* `wait` does not close stdin
* `wait` only requires `&self` instead of `&mut self` since there is no state to maintain and subsequent calls are safe
Tracking issue: #82971
