Autodiff batching
Enzyme supports batching, which is especially known from the ML side when training neural networks.
There we would normally have a training loop, where in each iteration we would pass in some data (e.g. an image), and a target vector. Based on how close we are with our prediction we compute our loss, and then use backpropagation to compute the gradients and update our weights.
That's quite inefficient, so what you normally do is passing in a batch of 8/16/.. images and targets, and compute the gradients for those all at once, allowing better optimizations.
Enzyme supports batching in two ways, the first one (which I implemented here) just accepts a Batch size,
and then each Dual/Duplicated argument has not one, but N shadow arguments. So instead of
```rs
for i in 0..100 {
df(x[i], y[i], 1234);
}
```
You can now do
```rs
for i in 0..100.step_by(4) {
df(x[i+0],x[i+1],x[i+2],x[i+3], y[i+0], y[i+1], y[i+2], y[i+3], 1234);
}
```
which will give the same results, but allows better compiler optimizations. See the testcase for details.
There is a second variant, where we can mark certain arguments and instead of having to pass in N shadow arguments, Enzyme assumes that the argument is N times longer. I.e. instead of accepting 4 slices with 12 floats each, we would accept one slice with 48 floats. I'll implement this over the next days.
I will also add more tests for both modes.
For any one preferring some more interactive explanation, here's a video of Tim's llvm dev talk, where he presents his work. https://www.youtube.com/watch?v=edvaLAL5RqU
I'll also add some other docs to the dev guide and user docs in another PR.
r? ghost
Tracking:
- https://github.com/rust-lang/rust/issues/124509
- https://github.com/rust-lang/rust/issues/135283
Rename `is_like_osx` to `is_like_darwin`
Replace `is_like_osx` with `is_like_darwin`, which more closely describes reality (OS X is the pre-2016 name for macOS, and is by now quite outdated; Darwin is the overall name for the OS underlying Apple's macOS, iOS, etc.).
``@rustbot`` label O-apple
r? compiler
Improve `xcrun` error handling
The compiler invokes `xcrun` on macOS when linking Apple targets, to find the Xcode SDK which contain all the necessary linker stubs. The error messages that `xcrun` outputs aren't always that great though, so this PR tries to improve that by providing extra context when an error occurs.
Fixes https://github.com/rust-lang/rust/issues/56829.
Fixes https://github.com/rust-lang/rust/issues/84534.
Part of https://github.com/rust-lang/rust/issues/129432.
See also the alternative https://github.com/rust-lang/rust/pull/131433.
Tested on:
- `x86_64-apple-darwin`, MacBook Pro running Mac OS X 10.12.6
- With no tooling installed
- With Xcode 9.2
- With Xcode 9.2 Commandline Tools
- `aarch64-apple-darwin`, MacBook M2 Pro running macOS 14.7.4
- With Xcode 13.4.1
- With Xcode 16.2
- Inside `nix-shell -p xcbuild` (nixpkgs' `xcrun` shim)
- `aarch64-apple-darwin`, VM running macOS 15.3.1
- With no tooling installed
- With Xcode 16.2 Commandline Tools
``@rustbot`` label O-apple
r? compiler
CC ``@BlackHoleFox`` ``@thomcc``
add FCW to warn about wasm ABI transition
See https://github.com/rust-lang/rust/issues/122532 for context: the "C" ABI on wasm32-unk-unk will change. The goal of this lint is to warn about any function definition and calls whose behavior will be affected by the change. My understanding is the following:
- scalar arguments are fine
- including 128 bit types, they get passed as two `i64` arguments in both ABIs
- `repr(C)` structs (recursively) wrapping a single scalar argument are fine (unless they have extra padding due to over-alignment attributes)
- all return values are fine
`@bjorn3` `@alexcrichton` `@Manishearth` is that correct?
I am making this a "show up in future compat reports" lint to maximize the chances people become aware of this. OTOH this likely means warnings for most users of Diplomat so maybe we shouldn't do this?
IIUC, wasm-bindgen should be unaffected by this lint as they only pass scalar types as arguments.
Tracking issue: https://github.com/rust-lang/rust/issues/138762
Transition plan blog post: https://github.com/rust-lang/blog.rust-lang.org/pull/1531
try-job: dist-various-2
Remove InstanceKind::generates_cgu_internal_copy
This PR should not contain any behavior changes. Before this PR, the logic for selecting instantiation mode is spread across all of
* `instantiation_mode`
* `cross_crate_inlinable`
* `generates_cgu_internal_copy`
* `requires_inline`
The last two of those functions are not well-designed. The function that actually decides if we generate a CGU-internal copy is `instantiation_mode`, _not_ `generates_cgu_internal_copy`. The function `requires_inline` documents that it is about the LLVM `inline` attribute and that it is a hint. The LLVM attribute is called `inlinehint`, this function is also used by other codegen backends, and since it is part of instantiation mode selection it is *not* a hint.
The goal of this PR is to start cleaning up the logic into a sequence of checks that have a more logical flow and are easier to customize in the future (to do things like improve incrementality or improve optimizations without causing obscure linker errors because you forgot to update another part of the compiler).
Lower to a memset(undef) when Rvalue::Repeat repeats uninit
Fixes https://github.com/rust-lang/rust/issues/138625.
It is technically correct to just do nothing. But if we actually do nothing, we may miss that this is de-initializing something, so instead we just lower to a single memset that writes undef. This is still superior to the memcpy loop, in both quality of code we hand to the backend and LLVM's final output.
Lower BinOp::Cmp to llvm.{s,u}cmp.* intrinsics
Lowers `mir::BinOp::Cmp` (`three_way_compare` intrinsic) to the corresponding LLVM `llvm.{s,u}cmp.i8.*` intrinsics.
These are the intrinsics mentioned in https://github.com/rust-lang/rust/pull/118310, which are now available in LLVM 19.
I couldn't find any follow-up PRs/discussions about this, please let me know if I missed something.
r? `@scottmcm`
Don't attempt to export compiler-builtins symbols from rust dylibs
They are marked with hidden visibility to prevent them from getting exported, so we shouldn't ask the linker to export them anyway. The only thing that does it cause a warning on macOS.
Part of https://github.com/rust-lang/rust/issues/136096
cc `@jyn514`
Avoid no-op unlink+link dances in incr comp
Incremental compilation scales quite poorly with the number of CGUs. This PR improves one reason for that.
The incr comp process hard-links all the files from an old session into a new one, then it runs the backend, which may just hard-link the new session files into the output directory. Then codegen hard-links all the output files back to the new session directory.
This PR (perhaps unimaginatively) fixes the silliness that ensues in the last step. The old `link_or_copy` implementation would be passed pairs of paths which are already the same inode, then it would blindly delete the destination and re-create the hard-link that it just deleted. This PR lets us skip both those operations. We don't skip the other two hard-links.
`cargo +stage1 b && touch crates/core/main.rs && strace -cfw -elink,linkat,unlink,unlinkat cargo +stage1 b` before and then after on `ripgrep-13.0.0`:
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
52.56 0.024950 25 978 485 unlink
34.38 0.016318 22 727 linkat
13.06 0.006200 24 249 unlinkat
------ ----------- ----------- --------- --------- ----------------
100.00 0.047467 24 1954 485 total
```
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
42.83 0.014521 57 252 unlink
38.41 0.013021 26 486 linkat
18.77 0.006362 25 249 unlinkat
------ ----------- ----------- --------- --------- ----------------
100.00 0.033904 34 987 total
```
This reduces the number of hard-links that are causing perf troubles, noted in https://github.com/rust-lang/rust/issues/64291 and https://github.com/rust-lang/rust/issues/137560
They are marked with hidden visibility to prevent them from getting
exported, so we shouldn't ask the linker to export them anyway. The only
thing that does it cause a warning on macOS.
Rollup of 7 pull requests
Successful merges:
- #138384 (Move `hir::Item::ident` into `hir::ItemKind`.)
- #138508 (Clarify "owned data" in E0515.md)
- #138531 (Store test diffs in job summaries and improve analysis formatting)
- #138533 (Only use `DIST_TRY_BUILD` for try jobs that were not selected explicitly)
- #138556 (Fix ICE: attempted to remap an already remapped filename)
- #138608 (rustc_target: Add target feature constraints for LoongArch)
- #138619 (Flatten `if`s in `rustc_codegen_ssa`)
r? `@ghost`
`@rustbot` modify labels: rollup
Mangle rustc_std_internal_symbols functions
This reduces the risk of issues when using a staticlib or rust dylib compiled with a different rustc version in a rust program. Currently this will either (in the case of staticlib) cause a linker error due to duplicate symbol definitions, or (in the case of rust dylibs) cause rustc_std_internal_symbols functions to be silently overridden. As rust gets more commonly used inside the implementation of libraries consumed with a C interface (like Spidermonkey, Ruby YJIT (curently has to do partial linking of all rust code to hide all symbols not part of the C api), the Rusticl OpenCL implementation in mesa) this is becoming much more of an issue. With this PR the only symbols remaining with an unmangled name are rust_eh_personality (LLVM doesn't allow renaming it) and `__rust_no_alloc_shim_is_unstable`.
Helps mitigate https://github.com/rust-lang/rust/issues/104707
try-job: aarch64-gnu-debug
try-job: aarch64-apple
try-job: x86_64-apple-1
try-job: x86_64-mingw-1
try-job: i686-mingw-1
try-job: x86_64-msvc-1
try-job: i686-msvc-1
try-job: test-various
try-job: armhf-gnu
Denote `ControlFlow` as `#[must_use]`
I've repeatedly hit bugs in the compiler due to `ControlFlow` not being marked `#[must_use]`. There seems to be an accepted ACP to make the type `#[must_use]` (https://github.com/rust-lang/libs-team/issues/444), so this PR implements that part of it.
Most of the usages in the compiler that trigger this new warning are "root" usages (calling into an API that uses control-flow internally, but for which the callee doesn't really care) and have been suppressed by `let _ = ...`, but I did legitimately find one instance of a missing `?` and one for a never-used `ControlFlow` value in #137448.
Presumably this needs an FCP too, so I'm opening this and nominating it for T-libs-api.
This PR also touches the tools (incl. rust-analyzer), but if this went into FCP, I'd split those out into separate PRs which can land before this one does.
r? libs-api
`@rustbot` label: T-libs-api I-libs-api-nominated
Rollup of 5 pull requests
Successful merges:
- #138283 (Enforce type of const param correctly in MIR typeck)
- #138439 (feat: check ARG_MAX on Unix platforms)
- #138502 (resolve: Avoid some unstable iteration)
- #138514 (Remove fake borrows of refs that are converted into non-refs in `MakeByMoveBody`)
- #138524 (Mark myself as unavailable for reviews temporarily)
r? `@ghost`
`@rustbot` modify labels: rollup
On Unix the limits can be gargantuan anyway so we're pretty
unlikely to hit them, but might still exceed it.
We consult ARG_MAX here to get an estimate.
fix: remove the check of lld not supporting @response-file
In LLVM v9, lld has supported `@response-file.`
LLVM v9 was released on 2019-09-19.
The check was added back to 2018-03-14 (1.26.0) via 04442af18b.
It has been more than five years, and we ship our own lld regardlessly.
This should be happily removed.
See also:
* <bb12396f91>
* <https://reviews.llvm.org/D63024>
atomic intrinsics: clarify which types are supported and (if applicable) what happens with provenance
The provenance semantics match what Miri implements and what the `AtomicPtr` API expects.
Don't `alloca` just to look at a discriminant
Today we're making LLVM do a bunch of extra work when you match on trivial stuff like `Option<bool>` or `ControlFlow<u8>`.
This PR changes that so that simple types like `Option<u32>` or `Result<(), Box<Error>>` can stay as `OperandValue::ScalarPair` and we can still read the discriminant from them, rather than needing to write them into memory to have a `PlaceValue` just to get the discriminant out.
Fixes#137503
attempt to support `BinaryFormat::Xcoff` in `naked_asm!`
Fixes https://github.com/rust-lang/rust/issues/137219
So, the inline assembly support for xcoff is extremely limited. The LLVM [XCOFFAsmParser](1b25c0c4da/llvm/lib/MC/MCParser/XCOFFAsmParser.cpp) does not support many of the attributes that LLVM itself emits, and that should exist based on [the assembler docs](https://www.ibm.com/docs/en/ssw_aix_71/assembler/assembler_pdf.pdf). It also does accept some that should not exist based on those docs.
So, I've tried to do the best I can given those limitations. At least it's better than emitting the directives for elf and having that fail somewhere deep in LLVM. Given that inline assembly for this target is incomplete (under `asm_experimental_arch`), I think that's OK (and again I don't see how we can do better given the limitations in LLVM).
r? ```@Noratrieb``` (given that you reviewed https://github.com/rust-lang/rust/pull/136637)
It seems reasonable to ping the [`powerpc64-ibm-aix` target maintainers](https://doc.rust-lang.org/rustc/platform-support/aix.html), hopefully they have thoughts too: ```@daltenty``` ```@gilamn5tr```
naked functions: on windows emit `.endef` without the symbol name
tracking issue: https://github.com/rust-lang/rust/issues/90957
fixes https://github.com/rust-lang/rust/issues/138320
The `.endef` directive does not take the name as an argument. Apparently the LLVM x86_64 parser does accept this, but on i686 it's rejected. In general `i686` does some special name mangling stuff, so it's good to include it in the naked function tests.
r? ````@ChrisDenton```` (because windows)
metadata: Ignore sysroot when doing the manual native lib search in rustc
This is the opposite alternative to https://github.com/rust-lang/rust/pull/138170 and another way to make native library search consistent between rustc and linker.
This way the directory list searched by rustc is still a prefix of the directory list considered by linker, but it's a shorter prefix than in #138170.
We can include the sysroot directories into rustc's search again later if the issues with #138170 are resolved, it will be a backward compatible change.
This may break some code doing weird things on unstable rustc, or tier 2-3 targets, like bundling `libunwind.a` or sanitizers into something.
Note that this doesn't affect shipped `libc.a`, because it lives in `self-contained` directories in sysroot, and `self-contained` sysroot is already not included into the rustc's search. All libunwind and sanitizer libs should be moved to `self-contained` sysroot too eventually.
With the consistent search directory list between rustc and linker we can make rustc own the native library search (at least for static libs) and use linker search only as a fallback (like in #123436). This will allow addressing issues like https://github.com/rust-lang/rust/pull/132394 once and for all on all targets.
r? ``@bjorn3``
In LLVM v9, lld has supported @response-file
LLVM v9 was released on 2019-09-19.
And the check was added back to 2018-03-14 (1.26.0) via 04442af18b.
It has been more than five years, and we ship our own lld regardlessly.
This should be happily removed.
See also:
* <bb12396f91>
* <https://reviews.llvm.org/D63024>
Continuing the work from #137350.
Removes the unused methods: `expect_variant`, `expect_field`,
`expect_foreign_item`.
Every method gains a `hir_` prefix.
Prevent ICE in autodiff validation by emitting user-friendly errors
This PR moves `valid_ret_activity` and `valid_input_activity` checks to the macro expansion phase in compiler/rustc_builtin_macros/src/autodiff.rs, replacing the following internal compiler error (ICE):
```
error: internal compiler error:
compiler/rustc_codegen_ssa/src/codegen_attrs.rs:935:13:
Invalid input activity Dual for Reverse mode
```
with a more user-friendly message.
The issue specifically affected the test file `tests/ui/autodiff/autodiff_illegal.rs`, impacting the functions `f5` and `f6`.
The ICE can be reproduced by following [Enzyme's Rustbook](https://enzymead.github.io/rustbook/installation.html) installation guide.
Additionally, this PR adds tests for invalid return activity in `autodiff_illegal.rs`, which previously triggered an unnoticed ICE before these fixes.
r? ``@oli-obk``
Speed up target feature computation
The LLVM backend calls `LLVMRustHasFeature` twice for every feature. In short-running rustc invocations, this accounts for a surprising amount of work.
r? `@bjorn3`
Revert <https://github.com/rust-lang/rust/pull/138084> to buy time to
consider options that avoids breaking downstream usages of cargo on
distributed `rustc-src` artifacts, where such cargo invocations fail due
to inability to inherit `lints` from workspace root manifest's
`workspace.lints` (this is only valid for the source rust-lang/rust
workspace, but not really the distributed `rustc-src` artifacts).
This breakage was reported in
<https://github.com/rust-lang/rust/issues/138304>.
This reverts commit 48caf81484, reversing
changes made to c6662879b2.
compiler: Use `size_of` from the prelude instead of imported
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the prelude instead of importing or qualifying them. Apply this change across the compiler.
These functions were added to all preludes in Rust 1.80.
r? ``@compiler-errors``
Don't re-`assume` in `transmute`s that don't change niches
I noticed in nightly 2025-02-21 that `transmute` is emitting way more `assume`s than necessary for newtypes.
For example, the three transmutes in <https://rust.godbolt.org/z/fW1KaTc4o> emits
```rust
define noundef range(i32 1, 0) i32 `@repeatedly_transparent_transmute(i32` noundef range(i32 1, 0) %_1) unnamed_addr {
start:
%0 = sub i32 %_1, 1
%1 = icmp ule i32 %0, -2
call void `@llvm.assume(i1` %1)
%2 = sub i32 %_1, 1
%3 = icmp ule i32 %2, -2
call void `@llvm.assume(i1` %3)
%4 = sub i32 %_1, 1
%5 = icmp ule i32 %4, -2
call void `@llvm.assume(i1` %5)
%6 = sub i32 %_1, 1
%7 = icmp ule i32 %6, -2
call void `@llvm.assume(i1` %7)
%8 = sub i32 %_1, 1
%9 = icmp ule i32 %8, -2
call void `@llvm.assume(i1` %9)
%10 = sub i32 %_1, 1
%11 = icmp ule i32 %10, -2
call void `@llvm.assume(i1` %11)
ret i32 %_1
}
```
But those are all just newtypes that don't change size or niches, so none of it's needed.
After this PR it's down to just
```rust
define noundef range(i32 1, 0) i32 `@repeatedly_transparent_transmute(i32` noundef range(i32 1, 0) %_1) unnamed_addr {
start:
ret i32 %_1
}
```
because none of those `assume`s in the original actually did anything.
(Transmuting to something with a difference niche, though, still has the assumes -- the other tests continue to pass checking that.)
self-contained linker: conservatively default to `-znostart-stop-gc` on x64 linux
To help stabilization, this PR disables an LLD optimization on x64 linux with respect to `--gc-sections` and encapsulation symbols: it will reduce the number of crates needing to opt-out of lld due to this bfd / lld difference. For example, all the people using [linkme](https://github.com/dtolnay/linkme), which [doesn't work with lld](https://github.com/dtolnay/linkme/issues/63) or on nightly, need to disable lld.
More information about all this, and the historical differences, can be found in:
- https://maskray.me/blog/2021-01-31-metadata-sections-comdat-and-shf-link-order
- https://lld.llvm.org/ELF/start-stop-gc
This optimization has [no visible impact](https://github.com/rust-lang/rust/pull/137685#issuecomment-2686116312) on our benchmarks, so we can use it by default and have a safer/more conservative starting point to remove friction during migration. We can them emit an FCW for the cases where lld detects reliance on encapsulation symbols without `-znostart-stop-gc`, and then revert back to lld's default after a while. No one compiling on nightly relies on this difference, obviously, so doing an FCW is not necessary until after lld is used on stable.
I've tested that this correctly links on `linkme` examples. I've also quickly tried to crate an rmake test but the setup with encapsulation symbols is annoying to reproduce: a few link section/name attributes is not enough, we also need to collect symbols between the encapsulation symbols, without referencing them in code, for `-znostart-stop-gc` to only impact this... It should of course be doable though, maybe ````@Kobzol```` will look into it if they have time.
r? ````@petrochenkov````
Add verbatim linker to AIXLinker
This adds support for the "verbatim" native link modifier on AIX, will successfully pass the `native-link-modifier-verbatim-linker test case`
Use `std::mem::{size_of, size_of_val, align_of, align_of_val}` from the
prelude instead of importing or qualifying them.
These functions were added to all preludes in Rust 1.80.
Clean up various LLVM FFI things in codegen_llvm
cc ```@ZuseZ4``` I touched some autodiff parts
The major change of this PR is [bfd88ce](bfd88cead0) which makes `CodegenCx` generic just like `GenericBuilder`
The other commits mostly took advantage of the new feature of making extern functions safe, but also just used some wrappers that were already there and shrunk unsafe blocks.
best reviewed commit-by-commit
Currently it is called twice, once with `allow_unstable` set to true and
once with it set to false. This results in some duplicated work. Most
notably, for the LLVM backend, `LLVMRustHasFeature` is called twice for
every feature, and it's moderately slow. For very short running
compilations on platforms with many features (e.g. a `check` build of
hello-world on x86) this is a significant fraction of runtime.
This commit changes `target_features_cfg` so it is only called once, and
it now returns a pair of feature sets. This halves the number of
`LLVMRustHasFeature` calls.
Support raw-dylib link kind on ELF
raw-dylib is a link kind that allows rustc to link against a library without having any library files present.
This currently only exists on Windows. rustc will take all the symbols from raw-dylib link blocks and put them in an import library, where they can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient to have this same functionality. Not having the libraries present at build-time can be convenient for several reasons, especially cross-compilation. With raw-dylib, code linking against a library can be cross-compiled without needing to have these libraries available on the build machine. If the libc crate makes use of this, it would allow cross-compilation without having any libc available on the build machine. This is not yet possible with this implementation, at least against libc's like glibc that use symbol versioning. The raw-dylib kind could be extended with support for symbol versioning in the future.
This implementation is very experimental and I have not tested it very well. I have tested it for a toy example and the lz4-sys crate, where it was able to successfully link a binary despite not having a corresponding library at build-time.
I was inspired by Björn's comments in https://internals.rust-lang.org/t/bundle-zig-cc-in-rustup-by-default/22096/27
Tracking issue: #135694
r? bjorn3
try-job: aarch64-apple
try-job: x86_64-msvc-1
try-job: x86_64-msvc-2
try-job: test-various
Rollup of 12 pull requests
Successful merges:
- #135767 (Future incompatibility warning `unsupported_fn_ptr_calling_conventions`: Also warn in dependencies)
- #137852 (Remove layouting dead code for non-array SIMD types.)
- #137863 (Fix pretty printing of unsafe binders)
- #137882 (do not build additional stage on compiler paths)
- #137894 (Revert "store ScalarPair via memset when one side is undef and the other side can be memset")
- #137902 (Make `ast::TokenKind` more like `lexer::TokenKind`)
- #137921 (Subtree update of `rust-analyzer`)
- #137922 (A few cleanups after the removal of `cfg(not(parallel))`)
- #137939 (fix order on shl impl)
- #137946 (Fix docker run-local docs)
- #137955 (Always allow rustdoc-json tests to contain long lines)
- #137958 (triagebot.toml: Don't label `test/rustdoc-json` as A-rustdoc-search)
r? `@ghost`
`@rustbot` modify labels: rollup
rustdoc: when merging target features, keep the highest stability
This addresses https://github.com/rust-lang/rust/issues/137366. (Not closing since we might consider a backport.)
rustdoc wants to pretend that it runs for all targets at once and has all target features, so `tcx.rust_target_features()` will actually be all the target features. For target features that exist on multiple targets, the stability info for one of the targets will be picked (first or last in the list, I guess). All the code consuming that query has to be aware that the data is basically nonsense when running in rustdoc, but the logic checking for unstable or forbidden `#[target_feature]` attributes was not aware of that.
This PR makes the `tcx.rust_target_features()` info in rustdoc slightly less nonsensical (and decidedly less random) by having the "most stable" target feature take precedent. That deals with #137366 (a conflict between a stable and a "forbidden" target feature of the same name for different targets), and also deals with the situation (that we did not seem to have yet) of a conflict between a stable and an unstable target feature of the same name. Note that if there are two unstable target features of the same name, rustdoc might still require the "wrong" nightly feature to be enabled -- but this can only possibly affect unstable code so I guess we can wait until that actually happens, and then someone will have to rewrite this entire thing to be less hacky.
improve `simd_select` error message when used with invalid mask type
followup to https://github.com/rust-lang/rust/pull/137828
This PR improves the error message for an invalid `simd_select` mask type, and adds testing for `simd_scatter` and `simd_gather` being used with invalid mask types.
the `simd_masked_load` and `simd_masked_store` intrinsics already generated a better error message:
0c72c0d11a/tests/ui/simd/masked-load-store-build-fail.rs (L24-L37)
r? `@workingjubilee`
Fix link failure on AVR (incompatible ISA error)
Fixes#137739. A reproducer of the issue is present there. I believe the root cause was introducing the avr-none target (which has no CPU by default) while also trying to get the ISA revision from the target spec. This commit uses the `target-cpu` option instead, which is already required to be present for the target.
r? compiler
cc ``@Patryk27``
rename BackendRepr::Vector → SimdVector
For many Rustaceans, "vector" does not imply "SIMD", so let's be more clear in this type that is used pervasively in the compiler.
r? `@workingjubilee`
The embedded bitcode should always be prepared for LTO/ThinLTO
Fixes#115344. Fixes#117220.
There are currently two methods for generating bitcode that used for LTO. One method involves using `-C linker-plugin-lto` to emit object files as bitcode, which is the typical setting used by cargo. The other method is through `-C embed-bitcode=yes`.
When using with `-C embed-bitcode=yes -C lto=no`, we run a complete non-LTO LLVM pipeline to obtain bitcode, then the bitcode is used for LTO. We run the Call Graph Profile Pass twice on the same module.
This PR is doing something similar to LLVM's `buildFatLTODefaultPipeline`, obtaining the bitcode for embedding after running `buildThinLTOPreLinkDefaultPipeline`.
r? nikic
Fixes#137739. A reproducer of the issue is present there. I believe the
root cause was introducing the avr-none target (which has no CPU by
default) and trying to get the ISA revision from there. This commit
uses the `target-cpu` option instead, which is already required to be
present for the target.
Co-authored-by: tones111 <tones111@users.noreply.github.com>
linker: Fix escaping style for response files on Windows
If we use a С/С++ compiler as linker, then Posix-style escaping should be used.
Also temporarily fixup rustbuild to not fail at least in common scenarios, until the bootstrap compiler is updated.
Fixes https://github.com/rust-lang/rust/issues/137498
raw-dylib is a link kind that allows rustc to link against a library
without having any library files present.
This currently only exists on Windows. rustc will take all the symbols
from raw-dylib link blocks and put them in an import library, where they
can then be resolved by the linker.
While import libraries don't exist on ELF, it would still be convenient
to have this same functionality. Not having the libraries present at
build-time can be convenient for several reasons, especially
cross-compilation. With raw-dylib, code linking against a library can be
cross-compiled without needing to have these libraries available on the
build machine. If the libc crate makes use of this, it would allow
cross-compilation without having any libc available on the build
machine. This is not yet possible with this implementation, at least
against libc's like glibc that use symbol versioning.
The raw-dylib kind could be extended with support for symbol versioning
in the future.
This implementation is very experimental and I have not tested it very
well. I have tested it for a toy example and the lz4-sys crate, where it
was able to successfully link a binary despite not having a
corresponding library at build-time.
Teach structured errors to display short `Ty<'_>`
Make it so that in every structured error annotated with `#[derive(Diagnostic)]` that has a field of type `Ty<'_>`, the printing of that value into a `String` will look at the thread-local storage `TyCtxt` in order to shorten to a length appropriate with the terminal width. When this happen, the resulting error will have a note with the file where the full type name was written to.
```
error[E0618]: expected function, found `((..., ..., ..., ...), ..., ..., ...)``
--> long.rs:7:5
|
6 | fn foo(x: D) { //~ `x` has type `(...
| - `x` has type `((..., ..., ..., ...), ..., ..., ...)`
7 | x(); //~ ERROR expected function, found `(...
| ^--
| |
| call expression requires function
|
= note: the full name for the type has been written to 'long.long-type-14182675702747116984.txt'
= note: consider using `--verbose` to print the full type name to the console
```
Follow up to and response to the comments on #136898.
r? ``@oli-obk``
ssa/mono: deduplicate `type_has_metadata`
The implementation of the `type_has_metadata` function is duplicated in `rustc_codegen_ssa` and `rustc_monomorphize`, so move this to `rustc_middle`.
Make it so that every structured error annotated with `#[derive(Diagnostic)]` that has a field of type `Ty<'_>`, the printing of that value into a `String` will look at the thread-local storage `TyCtxt` in order to shorten to a length appropriate with the terminal width. When this happen, the resulting error will have a note with the file where the full type name was written to.
```
error[E0618]: expected function, found `((..., ..., ..., ...), ..., ..., ...)``
--> long.rs:7:5
|
6 | fn foo(x: D) { //~ `x` has type `(...
| - `x` has type `((..., ..., ..., ...), ..., ..., ...)`
7 | x(); //~ ERROR expected function, found `(...
| ^--
| |
| call expression requires function
|
= note: the full name for the type has been written to 'long.long-type-14182675702747116984.txt'
= note: consider using `--verbose` to print the full type name to the console
```
Make `#[used]` work when linking with `ld64`
To make `#[used]` work in static libraries, we use the `symbols.o` trick introduced in https://github.com/rust-lang/rust/pull/95604.
However, the linker shipped with Xcode, ld64, works a bit differently from other linkers; in particular, [it completely ignores undefined symbols by themselves](https://github.com/apple-oss-distributions/ld64/blob/ld64-954.16/src/ld/parsers/macho_relocatable_file.cpp#L2455-L2468), and only consider them if they have relocations (something something atoms something fixups, I don't know the details).
So to make the `symbols.o` file work on ld64, we need to actually insert a relocation. That's kinda cumbersome to do though, since the relocation must be valid, and hence must point to a valid piece of machine code, and is hence very architecture-specific.
Fixes https://github.com/rust-lang/rust/issues/133491, see that for investigation.
---
Another option would be to pass `-u _foo` to the final linker invocation. This has the problem that `-u` causes the linker to not be able to dead-strip the symbol, which is undesirable. (If we did this, we would possibly also want to do it by putting the arguments in a file by itself, and passing that file via ``@`,` e.g. ``@undefined_symbols.txt`,` similar to https://github.com/rust-lang/rust/issues/52699, though that [is only supported since Xcode 12](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes#Linking), and I'm not sure we wanna bump that).
Various other options that are probably all undesirable as they affect link time performance:
- Pass `-all_load` to the linker.
- Pass `-ObjC` to the linker (the Objective-C support in the linker has different code paths that load more of the binary), and instrument the binaries that contain `#[used]` symbols.
- Pass `-force_load` to libraries that contain `#[used]` symbols.
Failed attempt: Embed `-u _foo` in the object file with `LC_LINKER_OPTION`, akin to https://github.com/rust-lang/rust/issues/121293. Doesn't work, both because `ld64` doesn't read that from archive members unless it already has a reason to load the member (which is what this PR is trying to make it do), and because `ld64` only support the `-l`, `-needed-l`, `-framework` and `-needed_framework` flags in there.
---
TODO:
- [x] Support all Apple architectures.
- [x] Ensure that this works regardless of the actual type of the symbol.
- [x] Write up more docs.
- [x] Wire up a few proper tests.
`@rustbot` label O-apple
New attribute parsing infrastructure
Another step in the plan outlined in https://github.com/rust-lang/rust/issues/131229
introduces infrastructure for structured parsers for attributes, as well as converting a couple of complex attributes to have such structured parsers.
This PR may prove too large to review. I left some of my own comments to guide it a little. Some general notes:
- The first commit is basically standalone. It just preps some mostly unrelated sources for the rest of the PR to work. It might not have enormous merit on its own, but not negative merit either. Could be merged alone, but also doesn't make the review a whole lot easier. (but it's only +274 -209)
- The second commit is the one that introduces new infrastructure. It's the important one to review.
- The 3rd commit uses the new infrastructure showing how some of the more complex attributes can be parsed using it. Theoretically can be split up, though the parsers in this commit are the ones that really test the new infrastructure and show that it all works.
- The 4th commit fixes up rustdoc and clippy. In the previous 2 they didn't compile yet while the compiler does. Separated them out to separate concerns and make the rest more palatable.
- The 5th commit blesses some test outputs. Sometimes that's just because a diagnostic happens slightly earlier than before, which I'd say is acceptable. Sometimes a diagnostic is now only emitted once where it would've been twice before (yay! fixed some bugs). One test I actually moved from crashes to fixed, because it simply doesn't crash anymore. That's why this PR Closes#132391. I think most choices I made here are generally reasonable, but let me know if you disagree anywhere.
- The 6th commit adds a derive to pretty print attributes
- The 7th removes smir apis for attributes, for the time being. The api will at some point be replaced by one based on `rustc_ast_data_structures::AttributeKind`
In general, a lot of the additions here are comments. I've found it very important to document new things in the 2nd commit well so other people can start using it.
Closes#132391Closes#136717
Add a span to `CompilerBuiltinsCannotCall`
Currently, this error emit a diagnostic with no context like:
error: `compiler_builtins` cannot call functions through upstream monomorphizations; encountered invalid call from `<math::libm::support::hex_float::Hexf<i32> as core::fmt::LowerHex>::fmt` to `core::fmt::num::<impl core::fmt::LowerHex for i32>::fmt`
With this change, it at least usually points to the problematic function:
error: `compiler_builtins` cannot call functions through upstream monomorphizations; encountered invalid call from `<math::libm::support::hex_float::Hexf<i32> as core::fmt::LowerHex>::fmt` to `core::fmt::num::<impl core::fmt::LowerHex for i32>::fmt`
--> src/../libm/src/math/support/hex_float.rs:270:5
|
270 | fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
Allow `IndexSlice` to be indexed by ranges.
This comes with some annoyances as the index type can no longer inferred from indexing expressions. The biggest offender for this is `IndexVec::from_fn_n(|idx| ..., n)` where the index type won't be inferred from the call site or any index expressions inside the closure.
My main use case for this is mapping a `Place` to `Range<Idx>` for value tracking where the range represents all the values the place contains.
Currently, this error emit a diagnostic with no context like:
error: `compiler_builtins` cannot call functions through upstream monomorphizations; encountered invalid call from `<math::libm::support::hex_float::Hexf<i32> as core::fmt::LowerHex>::fmt` to `core::fmt::num::<impl core::fmt::LowerHex for i32>::fmt`
With this change, it at least usually points to the problematic
function:
error: `compiler_builtins` cannot call functions through upstream monomorphizations; encountered invalid call from `<math::libm::support::hex_float::Hexf<i32> as core::fmt::LowerHex>::fmt` to `core::fmt::num::<impl core::fmt::LowerHex for i32>::fmt`
--> src/../libm/src/math/support/hex_float.rs:270:5
|
270 | fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
Emit getelementptr inbounds nuw for pointer::add()
Lower pointer::add (via intrinsic::offset with unsigned offset) to getelementptr inbounds nuw on LLVM versions that support it. This lets LLVM make use of the pre-condition that the offset addition does not wrap in an unsigned sense. Together with inbounds, this also implies that the offset is non-negative.
Fixes https://github.com/rust-lang/rust/issues/137217.
Misc. `rustc_codegen_ssa` cleanups 🧹
Just a bunch of stuff I found while reading the crate's code.
Each commit can stand on its own.
Maybe r? `@Noratrieb` because I saw you did some similar cleanups on these files a while ago? (feel free to re-assign, I'm just guessing)
Give `global_asm` a fake body to store typeck results, represent `sym fn` as a hir expr to fix `sym fn` operands with lifetimes
There are a few intertwined problems with `sym fn` operands in both inline and global asm macros.
Specifically, unlike other anon consts, they may evaluate to a type with free regions in them without actually having an item-level type annotation to give them a "proper" type. This is in contrast to named constants, which always have an item-level type annotation, or unnamed constants which are constrained by their position (e.g. a const arg in a turbofish, or a const array length).
Today, we infer the type of the operand by looking at the HIR typeck results; however, those results are region-erased, so during borrowck we ICE since we don't expect to encounter erased regions. We can't just fill this type with something like `'static`, since we may want to use real (free) regions:
```rust
fn foo<'a>() {
asm!("/* ... */", sym bar::<&'a ()>);
}
```
The first idea may be to represent `sym fn` operands using *inline* consts instead of anon consts. This makes sense, since inline consts can reference regions from the parent body (like the `'a` in the example above). However, this introduces a problem with `global_asm!`, which doesn't *have* a parent body; inline consts *must* be associated with a parent body since they are not a body owner of their own. In #116087, I attempted to fix this by using two separate `sym` operands for global and inline asm. However, this led to a lot of confusion and also some unattractive code duplication.
In this PR, I adjust the lowering of `global_asm!` so that it's lowered in a "fake" HIR body. This body contains a single expression which is `ExprKind::InlineAsm`; we don't *use* this HIR body, but it's used in typeck and borrowck so that we can properly infer and validate the the lifetimes of `sym fn` operands.
I then adjust the lowering of `sym fn` to instead be represented with a HIR expression. This is both because it's no longer necessary to represent this operand as an anon const, since it's *just* a path expression, and also more importantly to sidestep yet another ICE (https://github.com/rust-lang/rust/issues/137179), which has to do with the existing code breaking an invariant of def-id creation and anon consts. Specifically, we are not allowed to synthesize a def-id for an anon const when that anon const contains expressions with def-ids whose parent is *not* that anon const. This is somewhat related to https://github.com/rust-lang/rust/pull/130443#issuecomment-2445678945, which is also a place in the compiler where synthesizing anon consts leads to def-id parenting issue.
As a side-effect, this consolidates the type checking for inline and global asm, so it allows us to simplify `InlineAsmCtxt` a bit. It also allows us to delete a bit of hacky code from anon const `type_of` which was there to detect `sym fn` operands specifically. This also could be generalized to support `const` asm operands with types with lifetimes in them. Since we specifically reject these consts today, I'm not going to change the representation of those consts (but they'd just be turned into inline consts).
r? oli-obk -- mostly b/c you're patient and also understand the breadth of the code that this touches, please reassign if you don't want to review this.
Fixes#111709Fixes#96304Fixes#137179
