use key-value format in stage0 file
Currently, we are working on the python removal task on bootstrap. Which means we have to extract some data from the stage0 file using shell scripts. However, parsing values from the stage0.json file is painful because shell scripts don't have a built-in way to parse json files.
This change simplifies the stage0 file format to key-value pairs, which makes it easily readable from any environment.
See the zulip thread for more details: https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Using.20different.20format.20in.20the.20stage0.20file
CI: implement job skipping in Python matrix calculation
This removes the `step` YAML anchor and the corresponding bash script.
Best reviewed commit-by-commit.
r? ```@pietroalbini```
CI: add a script for dynamically computing CI job matrix
It would be great if was easier to run specific CI workflows locally, and also to allow us to spawn a specific CI workflow by bors, to enable running arbitrary try builds. See discussion [here](https://rust-lang.zulipchat.com/#narrow/stream/242791-t-infra/topic/CI.20workflows.20refactoring).
This PR is a first step in that direction.
- Moves the definition of CI runners and (for now) PR jobs into a separate `jobs.yml` file.
- Adds a simple Python script that reads the file, decides which jobs should be active for the current CI workflow, and prints them as JSON to their output.
- The PR job then reads this output and generates its job matrix based on it.
By moving the job definitions from `ci.yml` into a separate file, we can handle it programmatically, which should make it easier to both do local execution of CI jobs and also to do arbitrary try builds.
Cancel `cargo update` job if there's no updates
Previously there were always updates so we didn't hit this. Since #122489, this job runs on a more frequent schedule and causes errors if there have been no changes in that timespan.
This led to a weird error on https://github.com/rust-lang/rust/pull/122646#issuecomment-2004339093 - because of this I've replaced the `exit 1`s here with `gh run cancel` so we don't have false 'failed' jobs in the logs.
Improvements to building and CI for mingw/msys
I was getting error messages when trying to follow the build instructions the mingw build for Rust, and managed to track the issue down to an incomparability of Rust's bootstrap program with MSYS2's version of git. Essentially, the problem is that MSYS2's git works in emulated unix-y paths, but bootstrap expects a Windows path. I found a workaround for this by using relative paths instead of absolute paths.
Along with that fix, this PR also updates the build instructions for MinGW to be compatible with modern versions of MSYS2, and some changes to CI to make sure that MSYS2's version of git is tested. In particular, I'm suggesting using the [MSYS2 github action](https://github.com/marketplace/actions/setup-msys2) specially made for this purpose, which is much less hacky than the old approach and gives us more control of what packages are installed. I also cleaned up as many alternate versions of key tools as I could find from PATH, to avoid accidental usage, and cleaned up some abuses of the `CUSTOM_MINGW` environment variable.
This fixes https://github.com/rust-lang/rust/issues/105696 and fixes https://github.com/rust-lang/rust/issues/117567
This seems to fix two sporadic errors that have been appearing in CI.
One is an issue with cmake being unable to verify that cmake is able to
build a simple test program. The other is a `invalid r_symbolnum`
linking error when trying to build one of cranelift's tests.
This is intended as a temporary fix until we can figure out how to
resolve those issues.
Bump Fuchsia, build tests, and use 8 core bots
- Build Fuchsia on 8 cores instead of 16
- Skip building cranelift for Fuchsia
- Bump Fuchsia (includes building tests)
This includes a change to the upstream build_fuchsia_from_rust_ci script that builds a minimal set of tests, to improve coverage on this builder. This would have caught https://github.com/rust-lang/rust-clippy/issues/11952 and #119593.
See prior discussion on #119400 about building on 8 cores instead of 16. This PR combines changes from that and #119399, plus clean up.
r? `@Mark-Simulacrum`
