Add support for tidy linting via external tools for non-rust files
This change adds the flag `--check-extras` to `tidy`. It accepts a comma separated list of any of the options:
* py (test everything applicable for python files)
* py:lint (lint python files using `ruff`)
* py:fmt (check formatting for python files using `black`)
* shell or shell:lint (lint shell files using `shellcheck`)
Specific files to check can also be specified via positional args. Examples:
* `./x test tidy --check-extras=shell,py`
* `./x test tidy --check-extras=py:fmt -- src/bootstrap/bootstrap.py`
* `./x test tidy --check-extras=shell -- src/ci/*.sh`
* Python formatting can be applied with bless: `./x test tidy --ckeck-extras=py:fmt --bless`
`ruff` and `black` need to be installed via pip; this tool manages these within a virtual environment at `build/venv`. `shellcheck` needs to be installed on the system already.
---
This PR doesn't fix any of the errors that show up (I will likely go through those at some point) and it doesn't enforce anything new in CI. Relevant zulip discussion: https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/Other.20linters.20in.20CI
Rollup of 6 pull requests
Successful merges:
- #110435 (rustdoc-json: Add test for field ordering.)
- #111891 (feat: `riscv-interrupt-{m,s}` calling conventions)
- #114377 (test_get_dbpath_for_term(): handle non-utf8 paths (fix FIXME))
- #114469 (Detect method not found on arbitrary self type with different mutability)
- #114587 (Convert Const to Allocation in smir)
- #114670 (Don't use `type_of` to determine if item has intrinsic shim)
Failed merges:
- #114599 (Add impl trait declarations to SMIR)
r? `@ghost`
`@rustbot` modify labels: rollup
Don't use `type_of` to determine if item has intrinsic shim
When we're calling `resolve_instance` on an inline const, we were previously looking at the `type_of` for that const, seeing that it was an `extern "intrinsic"` fn def, and treating it as if we were computing the instance of that intrinsic itself. This is incorrect.
Instead, we should be using the def-id of the item we're computing to determine if it's an intrinsic.
Fixes#114660
Detect method not found on arbitrary self type with different mutability
```
error[E0599]: no method named `x` found for struct `Pin<&S>` in the current scope
--> $DIR/arbitrary_self_type_mut_difference.rs:11:18
|
LL | Pin::new(&S).x();
| ^ help: there is a method with a similar name: `y`
|
note: method is available for `Pin<&mut S>`
--> $DIR/arbitrary_self_type_mut_difference.rs:6:5
|
LL | fn x(self: Pin<&mut Self>) {}
| ^^^^^^^^^^^^^^^^^^^^^^^^^^
```
Related to #57994, as one of the presented cases can lead to code like this.
test_get_dbpath_for_term(): handle non-utf8 paths (fix FIXME)
Removes a FIXME for #9639
Part of #44366 which is E-help-wanted
The remaining two FIXMEs for #9639 are considerably more complicated, so I will create separate PRs for them.
feat: `riscv-interrupt-{m,s}` calling conventions
Similar to prior support added for the mips430, avr, and x86 targets this change implements the rough equivalent of clang's [`__attribute__((interrupt))`][clang-attr] for riscv targets, enabling e.g.
```rust
static mut CNT: usize = 0;
pub extern "riscv-interrupt-m" fn isr_m() {
unsafe {
CNT += 1;
}
}
```
to produce highly effective assembly like:
```asm
pub extern "riscv-interrupt-m" fn isr_m() {
420003a0: 1141 addi sp,sp,-16
unsafe {
CNT += 1;
420003a2: c62a sw a0,12(sp)
420003a4: c42e sw a1,8(sp)
420003a6: 3fc80537 lui a0,0x3fc80
420003aa: 63c52583 lw a1,1596(a0) # 3fc8063c <_ZN12esp_riscv_rt3CNT17hcec3e3a214887d53E.0>
420003ae: 0585 addi a1,a1,1
420003b0: 62b52e23 sw a1,1596(a0)
}
}
420003b4: 4532 lw a0,12(sp)
420003b6: 45a2 lw a1,8(sp)
420003b8: 0141 addi sp,sp,16
420003ba: 30200073 mret
```
(disassembly via `riscv64-unknown-elf-objdump -C -S --disassemble ./esp32c3-hal/target/riscv32imc-unknown-none-elf/release/examples/gpio_interrupt`)
This outcome is superior to hand-coded interrupt routines which, lacking visibility into any non-assembly body of the interrupt handler, have to be very conservative and save the [entire CPU state to the stack frame][full-frame-save]. By instead asking LLVM to only save the registers that it uses, we defer the decision to the tool with the best context: it can more accurately account for the cost of spills if it knows that every additional register used is already at the cost of an implicit spill.
At the LLVM level, this is apparently [implemented by] marking every register as "[callee-save]," matching the semantics of an interrupt handler nicely (it has to leave the CPU state just as it found it after its `{m|s}ret`).
This approach is not suitable for every interrupt handler, as it makes no attempt to e.g. save the state in a user-accessible stack frame. For a full discussion of those challenges and tradeoffs, please refer to [the interrupt calling conventions RFC][rfc].
Inside rustc, this implementation differs from prior art because LLVM does not expose the "all-saved" function flavor as a calling convention directly, instead preferring to use an attribute that allows for differentiating between "machine-mode" and "superivsor-mode" interrupts.
Finally, some effort has been made to guide those who may not yet be aware of the differences between machine-mode and supervisor-mode interrupts as to why no `riscv-interrupt` calling convention is exposed through rustc, and similarly for why `riscv-interrupt-u` makes no appearance (as it would complicate future LLVM upgrades).
[clang-attr]: https://clang.llvm.org/docs/AttributeReference.html#interrupt-risc-v
[full-frame-save]: 9281af2ecf/src/lib.rs (L440-L469)
[implemented by]: b7fb2a3fec/llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp (L61-L67)
[callee-save]: 973f1fe7a8/llvm/lib/Target/RISCV/RISCVCallingConv.td (L30-L37)
[rfc]: https://github.com/rust-lang/rfcs/pull/3246
Sync rustc_codegen_cranelift
A couple of small bug fixes this time. In addition I fixed the test suite after the introduction of `#![deny(internal_feature)]` broke it.
r? `@ghost`
`@rustbot` label +A-codegen +A-cranelift +T-compiler
Rollup of 6 pull requests
Successful merges:
- #113939 (open pidfd in child process and send to the parent via SOCK_SEQPACKET+CMSG)
- #114548 (Migrate a trait selection error to use diagnostic translation)
- #114606 (fix: not insert missing lifetime for `ConstParamTy`)
- #114634 (Mention riscv64-linux-android support in Android documentation)
- #114638 (Remove old RPITIT tests (revisions were removed))
- #114641 (Rename copying `ascii::Char` methods from `as_` to `to_`)
r? `@ghost`
`@rustbot` modify labels: rollup
Rename copying `ascii::Char` methods from `as_` to `to_`
Tracking issue: #110998.
The [API guidelines][naming] describe `as_` as used for borrowed -> borrowed operations, and `to_` for
owned -> owned operations on `Copy` types.
[naming]: https://rust-lang.github.io/api-guidelines/naming.html
Mention riscv64-linux-android support in Android documentation
This CL brings the android.md file in-line with the list of supported targets from platform-support.md.
Followup to https://github.com/rust-lang/rust/pull/112858
r? `@Mark-Simulacrum`
fix: not insert missing lifetime for `ConstParamTy`
Fixes#113462
We should ignore the missing lifetime, as it's illegal to include a lifetime in a const param.
r? ``@compiler-errors``
open pidfd in child process and send to the parent via SOCK_SEQPACKET+CMSG
This avoids using `clone3` when a pidfd is requested while still getting it in a 100% race-free manner by passing it up from the child process.
This should solve most concerns in #82971
The change in 07f855d781 introduced a
trailing numeral of some kind after the `extern crate
compiler_builtins`, which appears to have caused at least two false
negatives (654b924 and 657fd24). Instead, this change normalizes the
test output to ignore the number (of symbols rustc recognizes?) to avoid
needing to re-`--bless` these two tests for unrelated changes.
These new interrupt calling conventions are not themselves stabilized,
but there are other unstable calling conventions present in the SMIR
mapping (e.g. AVR interrupts) and the mapping appears to be "complete"
so far, with no obvious way to represent unstable conventions separately
from the stable ones.
Similar to prior support added for the mips430, avr, and x86 targets
this change implements the rough equivalent of clang's
[`__attribute__((interrupt))`][clang-attr] for riscv targets, enabling
e.g.
```rust
static mut CNT: usize = 0;
pub extern "riscv-interrupt-m" fn isr_m() {
unsafe {
CNT += 1;
}
}
```
to produce highly effective assembly like:
```asm
pub extern "riscv-interrupt-m" fn isr_m() {
420003a0: 1141 addi sp,sp,-16
unsafe {
CNT += 1;
420003a2: c62a sw a0,12(sp)
420003a4: c42e sw a1,8(sp)
420003a6: 3fc80537 lui a0,0x3fc80
420003aa: 63c52583 lw a1,1596(a0) # 3fc8063c <_ZN12esp_riscv_rt3CNT17hcec3e3a214887d53E.0>
420003ae: 0585 addi a1,a1,1
420003b0: 62b52e23 sw a1,1596(a0)
}
}
420003b4: 4532 lw a0,12(sp)
420003b6: 45a2 lw a1,8(sp)
420003b8: 0141 addi sp,sp,16
420003ba: 30200073 mret
```
(disassembly via `riscv64-unknown-elf-objdump -C -S --disassemble ./esp32c3-hal/target/riscv32imc-unknown-none-elf/release/examples/gpio_interrupt`)
This outcome is superior to hand-coded interrupt routines which, lacking
visibility into any non-assembly body of the interrupt handler, have to
be very conservative and save the [entire CPU state to the stack
frame][full-frame-save]. By instead asking LLVM to only save the
registers that it uses, we defer the decision to the tool with the best
context: it can more accurately account for the cost of spills if it
knows that every additional register used is already at the cost of an
implicit spill.
At the LLVM level, this is apparently [implemented by] marking every
register as "[callee-save]," matching the semantics of an interrupt
handler nicely (it has to leave the CPU state just as it found it after
its `{m|s}ret`).
This approach is not suitable for every interrupt handler, as it makes
no attempt to e.g. save the state in a user-accessible stack frame. For
a full discussion of those challenges and tradeoffs, please refer to
[the interrupt calling conventions RFC][rfc].
Inside rustc, this implementation differs from prior art because LLVM
does not expose the "all-saved" function flavor as a calling convention
directly, instead preferring to use an attribute that allows for
differentiating between "machine-mode" and "superivsor-mode" interrupts.
Finally, some effort has been made to guide those who may not yet be
aware of the differences between machine-mode and supervisor-mode
interrupts as to why no `riscv-interrupt` calling convention is exposed
through rustc, and similarly for why `riscv-interrupt-u` makes no
appearance (as it would complicate future LLVM upgrades).
[clang-attr]: https://clang.llvm.org/docs/AttributeReference.html#interrupt-risc-v
[full-frame-save]: 9281af2ecf/src/lib.rs (L440-L469)
[implemented by]: b7fb2a3fec/llvm/lib/Target/RISCV/RISCVRegisterInfo.cpp (L61-L67)
[callee-save]: 973f1fe7a8/llvm/lib/Target/RISCV/RISCVCallingConv.td (L30-L37)
[rfc]: https://github.com/rust-lang/rfcs/pull/3246
Restrict linker version script of proc-macro crates to just its two symbols
Restrict linker version script of proc-macro crates to just the two symbols of each proc-macro crate.
The main known effect of doing this is to stop including `#[no_mangle]` symbols in the linker version script.
Background:
The combination of a proc-macro crate with an import of another crate that itself exports a no_mangle function was broken for a period of time, because:
* In PR #99944 we stopped exporting no_mangle symbols from proc-macro crates; proc-macro crates have a very limited interface and are meant to be treated as a blackbox to everything except rustc itself. However: he constructed linker version script still referred to them, but resolving that discrepancy was left as a FIXME in the code, tagged with issue #99978.
* In PR #108017 we started telling the linker to check (via the`--no-undefined-version` linker invocation flag) that every symbol referenced in the "linker version script" is provided as linker input. So the unresolved discrepancy from #99978 started surfacing as a compile-time error (e.g. #111888).
Fix#111888Fix#99978.
tests: Uncomment now valid GAT code behind FIXME
The code fails to parse with `nightly-2021-02-05`:
$ cargo +nightly-2021-02-05 build
error: generic associated types in trait paths are currently not implemented
--> src/main.rs:9:42
|
9 | fn _bar<T: for<'a> StreamingIterator<Item<'a> = &'a [i32]>>(_iter: T) { /* ... */
| ^^^^
but parses with `nightly-2021-02-06`:
$ cargo +nightly-2021-02-06 build
warning: the feature `generic_associated_types` is incomplete and may not be safe to use and/or cause compiler crashes
warning: 1 warning emitted
because it was (with high probability) fixed by #79554 which was merged within that nightly range.
This PR is part of #44366 which is E-help-wanted.
Allowing re-implementation of mir_drops_elaborated query
For our use case of the rust compiler interface (a rust verifier called [Prusti](https://github.com/viperproject/prusti-dev/)), it would be extremely useful if we were able to "copy" the implementation of the `mir_drops_elaborated_and_const_checked` query to override it. This would mean that the following items would need to be made public:
>6d55184d05/compiler/rustc_mir_transform/src/lib.rs (L434)
>6d55184d05/compiler/rustc_mir_transform/src/inline.rs (L32)
(for the latter its module needs to be public or it needs to be re-exported)
To explain why (we think) this is necessary: I am currently working on a new feature, where we try to modify the generated executables by inserting certain additional checks, and potentially perform some optimizations based on verification results.
We are using the rust compiler interface and most of our goals can be achieved by overriding queries, in our case this is currently `mir_drops_elaborated_and_const_checked`.
However, at the moment this approach is somewhat limited. When overriding queries, we can call and steal the base-query and then modify the results before allocating and returning those.
The problem is that the verification works with a copy of `mir_promoted`. For the modifications we want to make to the mir, we would often want to rely on results of the verifier that refer to Locations in the `mir_promoted`. We can not modify the `mir_promoted` query using these results, because to run the verification we also need the results of `mir_borrowck()`, which means `mir_promoted` will already be constructed and cached.
The Locations we get from the verifier are also no longer usable to modify `mir_drops_elaborated_and_const_checked`, because the MIR obviously changes between those 2 phases. Tracking all Locations between the two seems to be pretty much unfeasible, and would also be extremely unstable.
By being able to override the query with its original implementation, we could modify the MIR before drop elaboration and the various other passes are performed.
I have spent quite a bit of time investigating other solutions, and didn't find any other way solving this problem. If I still missed something I would of course be happy to hear any suggestions that do not require exposing more internal compiler functionality. However, I think being able to re-implement certain queries could also benefit other use cases in the future, for example in PR #108328 one of the approaches discussed involved doing the same thing for `mir_promoted`.
Prevent constant rebuilds of `rustc-main` (and thus everything else)
PR #114305 changed bootstrap to run `strip -g` on `librustc_driver.so` and `libllvm.so` on Linux when no debuginfo was requested. Unfortunately, that PR resulted in bootstrap always rebuilding everything starting from stage 1 `rustc-main` (including stage 1 libraries and tests) when invoking bootstrap multiple times.
We noticed this because Ferrocene's CI times increased to between 2x and 3x total execution time, but the regression can also be reproduced locally by running `./x build library/sysroot --stage 1` twice.
The explanation of the problem is in the code comments.
r? ```@lqd```
cc ```@ozkanonur```
update llvm-wrapper include to silence deprecation warning
Includes of `include/llvm/Support/Host.h` now emit a deprecated warning: `warning: This header is deprecated, please use llvm/TargetParser/Host.h`.
I don't believe we are using this include.
I don't believe we need to bump the `download-ci-llvm` stamp since these warnings are emitted while building the `llvm-wrapper`.
r? ```@nikic```
CFI: Fix error compiling core with LLVM CFI enabled
Fix#90546 by filtering out global value function pointer types from the type tests, and adding the LowerTypeTests pass to the rustc LTO optimization pipelines.
