fix broken build cache caused by rustdoc builds
Currently rustdoc breaks the build cache (due to having different rustflags) when building rustdoc before building another tool (e.g., `x test miri && x test rustdoc && x test miri`).
This change fixes that by moving `on-broken-pipe` into `prepare_cargo_tool` so it is set for all tools.
cc `@RalfJung`
Fixes#123177
Bump black, ruff and platformdirs
Mainly because I encountered the following error, and we have no reason to prevent our upgrade.
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed.
This behaviour is the source of the following dependency conflicts.
virtualenv 20.25.3 requires platformdirs<5,>=3.9.1, but you have platformdirs 3.6.0 which is incompatible.
```
r? `@Kobzol`
miri: make sure we can find link_section statics even for the local crate
Miri needs some way to iterate all the exported functions and "used" statics of all crates. For dependency crates, this already works fine since we can overwrite the query resonsible for computing `exported_symbols`, but it turns out for local binary crates this does not work: for binaries, `reachable_set` skips a lot of its logic and only checks `contains_extern_indicator()` and `RUSTC_STD_INTERNAL_SYMBOL`. Other flags like `CodegenFnAttrFlags::USED` are entirely ignored.
This PR proposes to use the same check, `has_custom_linkage`, in binaries that we already use to drive the main workqueue of the reachability recursive traversal. I have no idea why binaries used a slightly different check that ignores `USED` -- was that deliberate or does it just not matter most of the time?
Currently rustdoc breaks the build cache (due to having different rustflags) when building
rustdoc before building another tool (e.g., `x test miri && x test rustdoc && x test miri`).
This change fixes that by moving `on-broken-pipe` into `prepare_cargo_tool` so it is
set for all tools.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Update cargo
10 commits in bc89bffa5987d4af8f71011c7557119b39e44a65..4ed7bee47f7dd4416b36fada1909e9a62c546246
2024-06-22 00:36:36 +0000 to 2024-06-25 16:28:22 +0000
- test: omit target-dir name (rust-lang/cargo#14142)
- test: migrate offline, open_namespaces and owner to snapbox (rust-lang/cargo#14138)
- docs: remove stray comment (rust-lang/cargo#14133)
- Change tests to support `rustc` wording changes (rust-lang/cargo#14135)
- deps: dont use tracing-attributes for workspace, use for cargo only (rust-lang/cargo#14122)
- Include vcs_info even if workspace is dirty (rust-lang/cargo#13960)
- test: migrate profile* to snapbox (rust-lang/cargo#14128)
- test: migrate path and paths to snapbox (rust-lang/cargo#14109)
- test: migrate rustflags, rustup and script to snapbox (rust-lang/cargo#14126)
- test: migrate rename_deps, replace, required_features and run to snapbox (rust-lang/cargo#14127)
Rollup of 7 pull requests
Successful merges:
- #126618 (Mark assoc tys live only if the corresponding trait is live)
- #126746 (Deny `use<>` for RPITITs)
- #126868 (not use offset when there is not ends with brace)
- #126884 (Do not ICE when suggesting dereferencing closure arg)
- #126893 (Eliminate the distinction between PREC_POSTFIX and PREC_PAREN precedence level)
- #126915 (Don't suggest awaiting in closure patterns)
- #126943 (De-duplicate all consecutive native libs regardless of their options)
r? `@ghost`
`@rustbot` modify labels: rollup
Mainly because I encountered the following error, and we have no reason to prevent our upgrade.
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed.
This behaviour is the source of the following dependency conflicts.
virtualenv 20.25.3 requires platformdirs<5,>=3.9.1, but you have platformdirs 3.6.0 which is incompatible.
```
Specify target specific linker for `riscv64gc-gnu` job
In https://github.com/rust-lang/rust/pull/125669 I updated the `riscv64gc-gnu` job to be more in line with other jobs, like the `armhf-gnu` one.
I missed something important, though! Since the `riscv64gc-unknown-linux-gnu` builds go through a cross compiler, invoking `cc` is not correct! That would invoke the `x86_64-unknown-linux-gnu` linker, instead of the `riscv64gc-unknown-linux-gnu` linker.
While developing #125669 I assumed this line would have been sufficient, however it was not:
553a69030e/src/ci/docker/host-x86_64/disabled/riscv64gc-gnu/Dockerfile (L27-L28)
Unfortunately, without the linker set, `cc` is invoked, and the following linker error occurs:
<details>
<summary>Long output</summary>
```
2024-06-21T14:37:54.4786899Z === HAYSTACK ===
2024-06-21T14:37:54.4787240Z error: linking with `cc` failed: exit status: 1
2024-06-21T14:37:54.4787589Z |
2024-06-21T14:37:54.4806781Z = note: LC_ALL="C" PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" VSLANG="1033" "cc" "/tmp/rustc7Ju6va/symbols.o" "inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o" "inaccessible/tmp/program.dv9uftjrq86w5xa7l2eo7g9l7.rcgu.o" "-Wl,--as-needed" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/inaccessible-temp-dir/inaccessible-temp-dir/rmake_out" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libstd-bb9ed017b7d70437.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libpanic_unwind-5b34ef68c22a7b9b.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libobject-4b9224407e296f51.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libmemchr-1eaaa8d74bbe0b36.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libaddr2line-b34292f7086867e9.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libgimli-1927540c5dff6690.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/librustc_demangle-ab493befe3742702.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libstd_detect-41195694489636c4.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libhashbrown-6905da49c8a1542a.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/librustc_std_workspace_alloc-bbc6290f03f92349.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libminiz_oxide-7a4683d4a82cf47c.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libadler-071583516cdbce90.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libunwind-d84b176d2f4c1334.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libcfg_if-92b7de0160f87442.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/liblibc-cff1533ea7c26263.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/liballoc-49e374c13f9e6132.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/librustc_std_workspace_core-9389dcc15b2f7f7b.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libcore-46f0fa14905fdc5c.rlib" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib/libcompiler_builtins-20e1af401f90cd1c.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/riscv64gc-unknown-linux-gnu/lib" "-o" "program" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
2024-06-21T14:37:54.4823900Z = note: /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4825032Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4826130Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4827200Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4828279Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4829365Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4830474Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4831554Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4833002Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4834116Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: Relocations in generic ELF (EM: 243)
2024-06-21T14:37:54.4835237Z /usr/bin/ld: inaccessible/tmp/program.program.45572bc5f2b14090-cgu.0.rcgu.o: error adding symbols: file in wrong format
2024-06-21T14:37:54.4835971Z collect2: error: ld returned 1 exit status
2024-06-21T14:37:54.4836449Z
2024-06-21T14:37:54.4836584Z
2024-06-21T14:37:54.4836744Z error: aborting due to 1 previous error
2024-06-21T14:37:54.4836983Z
2024-06-21T14:37:54.4836990Z
2024-06-21T14:37:54.4837083Z === NEEDLE ===
2024-06-21T14:37:54.4837518Z failed to find or create the directory specified by `--temps-dir`
2024-06-21T14:37:54.4838250Z thread 'main' panicked at /checkout/tests/run-make/inaccessible-temp-dir/rmake.rs:35:14:
2024-06-21T14:37:54.4838801Z needle was not found in haystack
2024-06-21T14:37:54.4839270Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
2024-06-21T14:37:54.4840125Z thread 'main' panicked at /checkout/tests/run-make/inaccessible-temp-dir/rmake.rs:27:5:
2024-06-21T14:37:54.4840759Z called `Result::unwrap()` on an `Err` value: Any { .. }
2024-06-21T14:37:54.4841194Z ------------------------------------------
```
</details>
By setting `--set target.riscv64gc-unknown-linux-gnu.linker=riscv64-linux-gnu-gcc` the correct linker is invoked.
## Testing
> [!NOTE]
> `riscv64gc-unknown-linux-gnu` is a [**Tier 2 with Host Tools** platform](https://doc.rust-lang.org/beta/rustc/platform-support.html), all tests may not necessarily pass! This change should only make fix existing broken tests (like `inaccessible-temp-dir`) while introducing no new breakage.
You can test out the job locally:
```sh
mv src/ci/docker/host-x86_64/disabled/riscv64gc-gnu src/ci/docker/host-x86_64/riscv64gc-gnu
DEPLOY=1 ./src/ci/docker/run.sh riscv64gc-gnu
```
Eliminate the distinction between PREC_POSTFIX and PREC_PAREN precedence level
I have been tangling with precedence as part of porting some pretty-printer improvements from syn back to rustc (related to parenthesization of closures, returns, and breaks by the AST pretty-printer).
As far as I have been able to tell, there is no difference between the 2 different precedence levels that rustc identifies as `PREC_POSTFIX` (field access, square bracket index, question mark, method call) and `PREC_PAREN` (loops, if, paths, literals).
There are a bunch of places that look at either `prec < PREC_POSTFIX` or `prec >= PREC_POSTFIX`. But there is nothing that needs to distinguish PREC_POSTFIX and PREC_PAREN from one another.
d49994b060/compiler/rustc_ast/src/util/parser.rs (L236-L237)d49994b060/compiler/rustc_hir_typeck/src/fn_ctxt/suggestions.rs (L2829)d49994b060/compiler/rustc_hir_typeck/src/fn_ctxt/suggestions.rs (L1290)
In the interest of eliminating a distinction without a difference, this PR collapses these 2 levels down to 1.
There is exactly 1 case where an expression with PREC_POSTFIX precedence needs to be parenthesized in a location that an expression with PREC_PAREN would not, and that's when the receiver of ExprKind::MethodCall is ExprKind::Field. `x.f()` means a different thing than `(x.f)()`. But this does not justify having separate precedence levels because this special case in the grammar is not governed by precedence. Field access does not have "lower precedence than" method call syntax — you can tell because if it did, then `x.f[0].f()` wouldn't be able to have its unparenthesized field access in the receiver of a method call. Because this Field/MethodCall special case is not governed by precedence, it already requires special handling and is not affected by eliminating the PREC_POSTFIX precedence level.
d49994b060/compiler/rustc_ast_pretty/src/pprust/state/expr.rs (L217-L221)
ast: Standardize visiting order for attributes and node IDs
This should only affect `macro_rules` scopes and order of diagnostics.
Also add a deprecation lint for `macro_rules` called outside of their scope, like in https://github.com/rust-lang/rust/issues/124535.
Add `SliceLike` to `rustc_type_ir`, use it in the generic solver code (+ some other changes)
First, we split out `TraitRef::new_from_args` which takes *just* `ty::GenericArgsRef` from `TraitRef::new` which takes `impl IntoIterator<Item: Into<GenericArg>>`. I will explain in a minute why.
Second, we introduce `SliceLike`, which allows us to be generic over `List<T>` and `[T]`. This trait has an `as_slice()` and `into_iter()` method, and some other convenience functions. However, importantly, since types like `I::GenericArgs` now implement `SliceLike` rather than `IntoIter<Item = I::GenericArg>`, we can't use `TraitRef::new` on this directly. That's where `new_from_args` comes in.
Finally, we adjust all the code to use these slice operators. Some things get simpler, some things get a bit more annoying since we need to use `as_slice()` in a few places. 🤷
r? lcnr
Deprecate no-op codegen option `-Cinline-threshold=...`
This deprecates `-Cinline-threshold` since using it has no effect. This has been the case since the new LLVM pass manager started being used, more than 2 years ago.
Recommend using `-Cllvm-args=--inline-threshold=...` instead.
Closes#89742 which is E-help-wanted.
std: refactor the TLS implementation
As discovered by Mara in #110897, our TLS implementation is a total mess. In the past months, I have simplified the actual macros and their expansions, but the majority of the complexity comes from the platform-specific support code needed to create keys and register destructors. In keeping with #117276, I have therefore moved all of the `thread_local_key`/`thread_local_dtor` modules to the `thread_local` module in `sys` and merged them into a new structure, so that future porters of `std` can simply mix-and-match the existing code instead of having to copy the same (bad) implementation everywhere. The new structure should become obvious when looking at `sys/thread_local/mod.rs`.
Unfortunately, the documentation changes associated with the refactoring have made this PR rather large. That said, this contains no functional changes except for two small ones:
* the key-based destructor fallback now, by virtue of sharing the implementation used by macOS and others, stores its list in a `#[thread_local]` static instead of in the key, eliminating one indirection layer and drastically simplifying its code.
* I've switched over ZKVM (tier 3) to use the same implementation as WebAssembly, as the implementation was just a way worse version of that
Please let me know if I can make this easier to review! I know these large PRs aren't optimal, but I couldn't think of any good intermediate steps.
`@rustbot` label +A-thread-locals
Rollup of 4 pull requests
Successful merges:
- #125241 (Add `rust_analyzer` as a predefined tool)
- #126213 (Update docs for AtomicBool/U8/I8 with regard to alignment)
- #126414 (Tier 2 std support must always be known)
- #126882 (Special case when a code line only has multiline span starts)
r? `@ghost`
`@rustbot` modify labels: rollup
Tier 2 std support must always be known
We should never have a tier 2 target without knowing its support status so I think this line in the tier 2 section is a bit wrong:
> ? indicates the standard library support is unknown or a work-in-progress.
My first inclination was just to drop the "unknown or" part. However, after thinking about it some more, I think we should just use `✓` for this. The only affected targets are UEFI and frankly there are targets with worse std support that are marked with `✓` (e.g. wasm).
I think a `✓` should mean "this supports building with std (and is checked in CI for tier 2+)". The target errata can detail the current limitations or special requirements for doing so.
Remove stray println from rustfmt's `rewrite_static`
r? `@calebcartwright` `@ytmimi` -- though anyone should probably r+ this so it gets into nightly sooner than later, since it's obviously wrong.
This can just be fixed in-tree, since I don't think we want to wait until the next sync to fix this.
Fix https://github.com/rust-lang/rustfmt/issues/6210
Fix https://github.com/rust-lang/rust/issues/126887
Implement `use<>` formatting in rustfmt
This PR implements formatting for precise-capturing `use<>` syntax as proposed in https://github.com/rust-lang/rust/pull/126753.
The syntax is implemented as-if the `use<>` bound were a trait bound but with the `use` keyword as its path segment identifier.
I opted to develop this in the rust-lang/rust tree since I'm not certain when the next rustfmt subtree sync is going to be, and I'd rather not block landing nightly support for `use<>` on something I have no control over. If ``@rust-lang/rustfmt`` would rather I move this PR over to that repository, then I would at least like to know when the next rustfmt->rust subtree sync is going to be, since stabilizing `precise_capturing` without formatting will be disruptive.
This implementation is otherwise rather straightforward.
Tracking:
- https://github.com/rust-lang/rust/issues/123432
Add needs-symlink directive to compiletest
This is an alternative to #126846 that allows running symlink tests on Windows in CI but will ignore them locally if symlinks aren't available. A future improvement would be to check that the `needs-symlink` directive is used in rmake files that call `create_symlink` but this is just a quick PR to unblock Windows users who want to run tests locally without enabling symlinks.
Rework pattern and expression nonterminal kinds.
Some tweaks to `NonterminalKind` that will assist with #124141. Details in the individual commits.
r? compiler-errors
cc ```@eholk```
less bootstrap warnings
This is how the build logs looks like currently:
```sh
$ x build
Building bootstrap
Compiling bootstrap v0.0.0 (/home/nimda/devspace/onur-ozkan/rust/src/bootstrap)
Finished `dev` profile [unoptimized] target(s) in 3.43s
WARNING: no codegen-backends config matched the requested path to build a codegen backend. HELP: add backend to codegen-backends in config.toml.
WARNING: creating symbolic link `/home/nimda/devspace/.other/rustc-builds/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/rustc-src/rust` to `/home/nimda/devspace/onur-ozkan/rust` failed with File exists (os
error 17)
Creating a sysroot for stage2 compiler (use `rustup toolchain link 'name' build/host/stage2`)
WARNING: creating symbolic link `/home/nimda/devspace/.other/rustc-builds/build/x86_64-unknown-linux-gnu/ci-rustc-sysroot/lib/rustlib/rustc-src/rust` to `/home/nimda/devspace/onur-ozkan/rust` failed with File e
xists (os error 17)
Building tool rustdoc (stage1 -> stage2, x86_64-unknown-linux-gnu)
Compiling rustdoc v0.0.0 (/home/nimda/devspace/onur-ozkan/rust/src/librustdoc)
Compiling rustdoc-tool v0.0.0 (/home/nimda/devspace/onur-ozkan/rust/src/tools/rustdoc)
Finished `release` profile [optimized + debuginfo] target(s) in 13.57s
Build completed successfully in 0:00:17
```
This PR removes artifact linking warnings and only shows the codegen-backend warning if explicitly called or during Dist or Install steps.
tidy: skip submodules if not present for non-CI environments
Right now tidy requires rustc-perf to be fetched as it checks its license, but this doesn't make sense for most contributors since rustc-perf is a dist-specific tool and its license will not change frequently, especially during compiler development. This PR makes tidy to skip rustc-perf if it's not fetched and if it's not running in CI.
Applies https://github.com/rust-lang/rust/pull/126225#issuecomment-2158143674 and reverts #126225.
