CI: add a script for dynamically computing CI job matrix
It would be great if was easier to run specific CI workflows locally, and also to allow us to spawn a specific CI workflow by bors, to enable running arbitrary try builds. See discussion [here](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/CI.20workflows.20refactoring).
This PR is a first step in that direction.
- Moves the definition of CI runners and (for now) PR jobs into a separate `jobs.yml` file.
- Adds a simple Python script that reads the file, decides which jobs should be active for the current CI workflow, and prints them as JSON to their output.
- The PR job then reads this output and generates its job matrix based on it.
By moving the job definitions from `ci.yml` into a separate file, we can handle it programmatically, which should make it easier to both do local execution of CI jobs and also to do arbitrary try builds.
Rollup of 4 pull requests
Successful merges:
- #123900 (Stop using `PolyTraitRef` for closure/coroutine predicates already instantiated w placeholders)
- #123924 (Fix various bugs in `ty_kind_suggestion`)
- #123943 (Use the rustc_private libc less in tests)
- #123970 (zkvm: fix references to `os_str` module)
r? `@ghost`
`@rustbot` modify labels: rollup
zkvm: fix references to `os_str` module
The `os_str` module has been moved to `sys`. This change fixes build issues by changing `use` to point to `crate::sys::os_str`.
Use the rustc_private libc less in tests
I started looking into our use of `rustc_private` + `extern crate libc;` in tests because of https://github.com/rust-lang/rust/pull/123938 and it looks like some fraction of the users of libc simply don't need the libc crate anymore.
Fix various bugs in `ty_kind_suggestion`
Consolidates two implementations of `ty_kind_suggestion`
Fixes some misuse of the empty param-env
Fixes a problem where we suggested `(42)` instead of `(42,)` for tuple suggestions
Suggest a value when `return;`, making it consistent with `break;`
Fixes#123906
Rollup of 12 pull requests
Successful merges:
- #123423 (Distribute LLVM bitcode linker as a preview component)
- #123548 (libtest: also measure time in Miri)
- #123666 (Fix some typos in doc)
- #123864 (Remove a HACK by instead inferring opaque types during expected/formal type checking)
- #123896 (Migrate some diagnostics in `rustc_resolve` to session diagnostic)
- #123919 (builtin-derive: tag → discriminant)
- #123922 (Remove magic constants when using `base_n`.)
- #123931 (Don't leak unnameable types in `-> _` recover)
- #123933 (move the LargeAssignments lint logic into its own file)
- #123934 (`rustc_data_structures::graph` mini refactor)
- #123941 (Fix UB in LLVM FFI when passing zero or >1 bundle)
- #123957 (disable create_dir_all_bare test on all(miri, windows))
r? `@ghost`
`@rustbot` modify labels: rollup
Fix UB in LLVM FFI when passing zero or >1 bundle
Rust passes a `*const &OperandBundleDef` to these APIs, usually from a `Vec<&OperandBundleDef>` or so. Previously we were dereferencing that pointer and passing it to the ArrayRef constructor with some length (N).
This meant that if the length was 0, we were dereferencing a pointer to nowhere (if the vector on the Rust side didn't actually get allocated or so), and if the length was >1 then loading the *second* element somewhere in LLVM would've been reading past the end.
Since Rust can't hold OperandBundleDef by-value we're forced to indirect through a vector that copies out the OperandBundleDefs from the by-reference list on the Rust side in order to match the LLVM expected API.
move the LargeAssignments lint logic into its own file
The collector is a file full of very subtle logic, so let's try to keep that separate from the logic that only serves to implement this lint.
Remove magic constants when using `base_n`.
Some use cases of `base_n` use number literals instead of the predefined constants. The latter are more descriptive so it might be better to use those instead.
builtin-derive: tag → discriminant
As far as I can tell, all of this operates on the discriminant, not the tag. After all, with something like `Option<&T>`, the "tag" of the `Some` variant is basically just the reference value, which is never what you want to compare when figuring out which variant the enum is in.
See [here](https://rustc-dev-guide.rust-lang.org/appendix/glossary.html) for an explanation of the difference between tag and discriminant.
Migrate some diagnostics in `rustc_resolve` to session diagnostic
Hello, I migrated some diagnostics in `rustc_resolve` to session diagnostic.
r? ``@davidtwco``
Remove a HACK by instead inferring opaque types during expected/formal type checking
I was wondering why I couldn't come up with a test that hits the code path of the argument check checking the types we inferred from the return type... Turns out we reject those attempts early during fudging.
I have absolutely no information for you as to what kind of type inference changes this may incur, but I think we should just land this out of two reasons:
* had I found the other place to use opaque type inference on before I added the hack, we'd be using that today and this PR would never have happened
* if it is possible to hit this path, it requires some god awful recursive RPIT logic that I doubt anyone would have written without actively trying to write obscure code
r? ``@ghost``
libtest: also measure time in Miri
A long time ago we disabled timekeeping of the default test harness in Miri, as otherwise it would fail to run without `-Zmiri-disable-isolation`. However, since then Miri gained a "fake clock" that lets it present some deterministic notion of time when isolation is enabled.
So we could now let libtest do timekeeping again when running in Miri. That's nice as it can help detect tests that run too long. However it can also be confusing as the results with isolation can be quite different than the real time.
``@rust-lang/miri`` what do you think?
Distribute LLVM bitcode linker as a preview component
The self-contained LLVM bitcode linker was recently added in #117458. It is currently only in use to link the Nvidia ptx assembly tests when running rustc tests (local or CI). In fact, the linker itself is currently only usable for the `nvptx64-nvidia-cuda` target, but more targets will be supported in the future.
The reason a new linker was needed for the ptx format is that the [old one](https://github.com/denzp/rust-ptx-linker) has not been updated the last few years. It worked fine for a while, but as LLVM changed it broke and the nvptx tests was [disabled in rustc back in 2019](f8f9a2869c). It was ad-hoc patched and have been used in a sub-optimal state by the community until now.
If this PR is merged, the LLVM bitcode linker will be distributed as a preview component that can be used as a replacement for the old ptx-linker for development in addition to rustc tests. In addition to installing the `llvm-bitcode-linker` component, also the `llvm-tools` component must be installed as the `llvm-bitcode-linker` works by calling llvm tools.
Even though the LLVM bitcode linker is in its early stages it already now provides a lot of value over the old ptx-linker just by working and using up-to-date llvm tooling. By shipping it as a component it will be easier to gather user experience and improving it.
``@petrochenkov`` when installing as a component it will be installed in the self-contained folder and will not work with `-Clink-self-contained=no` (although for some reason I expect to be a bug `-Clink-self-contained=-linker` doesn't properly disable it). However, when building using `x.py build` it will be placed in the `lib/rustlib/<target>/bin` directory and will be available for internal tests even if `-Clink-self-contained=no` is passed.
CC: ``@Mark-Simulacrum`` as I very briefly discussed it with you some months ago https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/.E2.9C.94.20How.20to.20ship.20a.20new.20tool.20.28embedded.20linker.29.20to.20users.3F
Miri on Windows: run .CRT$XLB linker section on thread-end
Hopefully fixes https://github.com/rust-lang/rust/issues/123583
First commit is originally by `@bjorn3`
r? `@oli-obk`
Cc `@ChrisDenton`
Add the missing inttoptr when we ptrtoint in ptr atomics
Ralf noticed this here: https://github.com/rust-lang/rust/pull/122220#discussion_r1535172094
Our previous codegen forgot to add the cast back to integer type. The code compiles anyway, because of course all locals are in-memory to start with, so previous codegen would do the integer atomic, store the integer to a local, then load a pointer from that local. Which is definitely _not_ what we wanted: That's an integer-to-pointer transmute, so all pointers returned by these `AtomicPtr` methods didn't have provenance. Yikes.
Here's the IR for `AtomicPtr::fetch_byte_add` on 1.76: https://godbolt.org/z/8qTEjeraY
```llvm
define noundef ptr `@atomicptr_fetch_byte_add(ptr` noundef nonnull align 8 %a, i64 noundef %v) unnamed_addr #0 !dbg !7 {
start:
%0 = alloca ptr, align 8, !dbg !12
%val = inttoptr i64 %v to ptr, !dbg !12
call void `@llvm.lifetime.start.p0(i64` 8, ptr %0), !dbg !28
%1 = ptrtoint ptr %val to i64, !dbg !28
%2 = atomicrmw add ptr %a, i64 %1 monotonic, align 8, !dbg !28
store i64 %2, ptr %0, align 8, !dbg !28
%self = load ptr, ptr %0, align 8, !dbg !28
call void `@llvm.lifetime.end.p0(i64` 8, ptr %0), !dbg !28
ret ptr %self, !dbg !33
}
```
r? `@RalfJung`
cc `@nikic`
compiletest ice tracking
see https://rust-lang.zulipchat.com/#narrow/stream/131828-t-compiler/topic/where.20to.20mass-add.20known.20ices.20.2F.20merging.20glacier.20into.20rust/near/429082963
This will allow us to sunset most of https://github.com/rust-lang/glacier
The rustc ices will be tracked directly inside the rust testsuite
There are a couple of .sh tests remaining that I have not ported over yet.
This adds `tests/crashes`, a file inside this directory MUST ice, otherwise it is considered test-fail.
This will be used to track ICEs from glacier and the bugtracker.
When someones pr accidentally fixes one of these ICEs, they can move the test from `crashes` into `ui` for example.
I also added a new tidy lint that warns when a test inside `tests/crashes` does not have a `//@ known-bug: ` line
the env var `COMPILETEST_VERBOSE_CRASHES` can be set to get exit code, stderr and stdout of a crash-test to aid debugging/adding tests.
Rust passes a *const &OperandBundleDef to these APIs, usually from a
Vec<&OperandBundleDef> or so. Previously we were dereferencing that
pointer and passing it to the ArrayRef constructor with some length (N).
This meant that if the length was 0, we were dereferencing a pointer to
nowhere, and if the length was >1 then loading the *second* element
somewhere in LLVM would've been reading past the end.
Since Rust can't hold OperandBundleDef by-value we're forced to indirect
through a vector that copies out the OperandBundleDefs from the
by-reference list on the Rust side in order to match the LLVM expected
API.
ci: test cargo on `aarch64-gnu`
Since `aarch64-unknown-linux-gnu` is a tier-1 target, we should also test cargo on it, especially since cargo's own CI doesn't cover this yet. This might have helped us discover #123733 sooner, which is not a cargo problem but was uncovered by a new cargo test (which we'll have to skip for now). Everything else passes in my local run, so at least we'll have a guard against future regressions.
Discard overflow obligations in `impl_may_apply`
Hacky fix for #123493. Throws away obligations that are overflowing in `impl_may_apply` when we recompute if an impl applies, since those will lead to fatal overflow if processed during fulfillment.
Something about #114811 (I think it's the predicate reordering) caused us to evaluate predicates differently in error reporting leading to fatal overflow, though I believe the underlying overflow is possible to hit since this code was rewritten to use fulfillment.
Fixes#123493
skip Codegen{GCC,Cranelift} when using CI rustc
CI rustc uses the default codegen backend, therefore we can't run `CodegenGCC` and `CodegenCranelift` tests when using it.
cc `@bjorn3` (to make sure I am not doing anything wrong)
Fixes#123331
