Remove more `#[macro_use] extern crate tracing`
Because explicit importing of macros via use items is nicer (more standard and readable) than implicit importing via `#[macro_use]`. Continuing the work from #124511 and #124914.
r? `@jackh726`
Add a footer in FileEncoder and check for it in MemDecoder
We have a few reports of ICEs due to decoding failures, where the fault does not lie with the compiler. The goal of this PR is to add some very lightweight and on-by-default validation to the compiler's outputs. If validation fails, we emit a fatal error for rmeta files in general that mentions the path that didn't load, and for incremental compilation artifacts we emit a verbose warning that tries to explain the situation and treat the artifacts as outdated.
The validation currently implemented here is very crude, and yet I think we have 11 ICE reports currently open (you can find them by searching issues for `1002111927320821928687967599834759150`) which this simple validation would have detected. The structure of the code changes here should permit the addition of further validation code, such as a checksum, if it is merited. I would like to have code to detect corruption such as reported in https://github.com/rust-lang/rust/issues/124719, but I'm not yet sure how to do that efficiently, and this PR is already a good size.
The ICE reports I have in mind that this PR would have smoothed over are:
https://github.com/rust-lang/rust/issues/124469https://github.com/rust-lang/rust/issues/123352https://github.com/rust-lang/rust/issues/123376 [^1]
https://github.com/rust-lang/rust/issues/99763https://github.com/rust-lang/rust/issues/93900.
---
[^1]: This one might be a compiler bug, but even if it is I think the workflow described is pushing the envelope of what we can support. This issue is one of the reasons this warning still asks people to file an issue.
always print nice 'std not found' error when std is not found
Fixes https://github.com/rust-lang/miri/issues/3529
Arguably Miri is doing something odd by letting people create no-std sysroots for arbitrary targets -- but equally arguably, there's no good reason for rustc to special-case the host triple here. Being a non-host triple does not imply the target is a no-std target, after all.
Enable `CrateNum` query feeding via `TyCtxt`
Instead of having a magic function that violates some `TyCtxtFeed` invariants, add a `create_def` equivalent for `CrateNum`s.
Note that this still isn't tracked by the query system (unlike `create_def`), and that feeding most `CrateNum` queries for crates other than the local one will likely cause performance regressions.
These things should be attempted on their own separately, but this PR should stand on its own
weak lang items are not allowed to be #[track_caller]
For instance the panic handler will be called via this import
```rust
extern "Rust" {
#[lang = "panic_impl"]
fn panic_impl(pi: &PanicInfo<'_>) -> !;
}
```
A `#[track_caller]` would add an extra argument and thus make this the wrong signature.
The 2nd commit is a consistency rename; based on the docs [here](https://doc.rust-lang.org/unstable-book/language-features/lang-items.html) and [here](https://rustc-dev-guide.rust-lang.org/lang-items.html) I figured "lang item" is more widely used. (In the compiler output, "lang item" and "language item" seem to be pretty even.)
Replace some `CrateStore` trait methods with hooks.
Just like with the `CrateStore` trait, this avoids the cyclic definition issues with `CStore` being
defined after TyCtxt, but needing to be used in TyCtxt.
Fixed the `private-dependency` bug
Fixed the private-dependency bug: If the directly dependent crate is loaded last and is not configured with `--extern`, it may be incorrectly set to `private-dependency`
Fixes#122756
With associated type bounds enabled, the implied_predicates and super_predicates
queries may differ for traits, since associated type bounds are also
implied but are not counted as super predicates.
Split an item bounds and an item's super predicates
This is the moral equivalent of #107614, but instead for predicates this applies to **item bounds**. This PR splits out the item bounds (i.e. *all* predicates that are assumed to hold for the alias) from the item *super predicates*, which are the subset of item bounds which share the same self type as the alias.
## Why?
Much like #107614, there are places in the compiler where we *only* care about super-predicates, and considering predicates that possibly don't have anything to do with the alias is problematic. This includes things like closure signature inference (which is at its core searching for `Self: Fn(..)` style bounds), but also lints like `#[must_use]`, error reporting for aliases, computing type outlives predicates.
Even in cases where considering all of the `item_bounds` doesn't lead to bugs, unnecessarily considering irrelevant bounds does lead to a regression (#121121) due to doing extra work in the solver.
## Example 1 - Trait Aliases
This is best explored via an example:
```
type TAIT<T> = impl TraitAlias<T>;
trait TraitAlias<T> = A + B where T: C;
```
The item bounds list for `Tait<T>` will include:
* `Tait<T>: A`
* `Tait<T>: B`
* `T: C`
While `item_super_predicates` query will include just the first two predicates.
Side-note: You may wonder why `T: C` is included in the item bounds for `TAIT`? This is because when we elaborate `TraitAlias<T>`, we will also elaborate all the predicates on the trait.
## Example 2 - Associated Type Bounds
```
type TAIT<T> = impl Iterator<Item: A>;
```
The `item_bounds` list for `TAIT<T>` will include:
* `Tait<T>: Iterator`
* `<Tait<T> as Iterator>::Item: A`
But the `item_super_predicates` will just include the first bound, since that's the only bound that is relevant to the *alias* itself.
## So what
This leads to some diagnostics duplication just like #107614, but none of it will be user-facing. We only see it in the UI test suite because we explicitly disable diagnostic deduplication.
Regarding naming, I went with `super_predicates` kind of arbitrarily; this can easily be changed, but I'd consider better names as long as we don't block this PR in perpetuity.
Print the crates not available as static
This prints out the crates not available to be statically linked when static linking is preferred and we run into an error with duplicated crates.
various clippy fixes
We need to keep the order of the given clippy lint rules before passing them.
Since clap doesn't offer any useful interface for this purpose out of the box,
we have to handle it manually.
Additionally, this PR makes `-D` rules work as expected. Previously, lint rules were limited to `-W`. By enabling `-D`, clippy began to complain numerous lines in the tree, all of which have been resolved in this PR as well.
Fixes#121481
cc `@matthiaskrgr`
rustc-metadata: Store crate name in self-profile of metadata_register_crate
When profiling a build of Zed, I found myself in need of names of crates that take the longest to register in downstream crates.
Add `wasm_c_abi` `future-incompat` lint
This is a warning that will tell users to update to `wasm-bindgen` v0.2.88, which supports spec-compliant C ABI.
The idea is to prepare for a future where Rust will switch to the spec-compliant C ABI by default; so not to break everyone's world, this warning is introduced.
Addresses #71871.
Move metadata header and version checks together
This will make it easier to report rustc versions for older metadata formats.
Split out of https://github.com/rust-lang/rust/pull/120855
Avoid invoking the `intrinsic` query for DefKinds other than `Fn` or `AssocFn`
fixes the perf regression from https://github.com/rust-lang/rust/pull/120675 by only invoking (and thus inserting into the dep graph) the `intrinsic` query if the `DefKind` matches items that can actually be intrinsics
Make not finding core a fatal error
Similar to https://github.com/rust-lang/rust/pull/120472, this prevents terminal spam. In particular, it makes the good diagnostic visible when you try to use a target that's not installed.
Rework `untranslatable_diagnostic` lint
Currently it only checks calls to functions marked with `#[rustc_lint_diagnostics]`. This PR changes it to check calls to any function with an `impl Into<{D,Subd}iagnosticMessage>` parameter. This greatly improves its coverage and doesn't rely on people remembering to add `#[rustc_lint_diagnostics]`. It also lets us add `#[rustc_lint_diagnostics]` to a number of functions that don't have an `impl Into<{D,Subd}iagnosticMessage>`, such as `Diag::span`.
r? ``@davidtwco``
Embed length of offset/position into Span tag byte
This cuts the average bytes/relative span from 3.5 to 3.2 on libcore, ultimately saving ~400kb of data.
Currently it only checks calls to functions marked with
`#[rustc_lint_diagnostics]`. This commit changes it to check calls to
any function with an `impl Into<{D,Subd}iagMessage>` parameter. This
greatly improves its coverage and doesn't rely on people remembering to
add `#[rustc_lint_diagnostics]`.
The commit also adds `#[allow(rustc::untranslatable_diagnostic)`]
attributes to places that need it that are caught by the improved lint.
These places that might be easy to convert to translatable diagnostics.
Finally, it also:
- Expands and corrects some comments.
- Does some minor formatting improvements.
- Adds missing `DecorateLint` cases to
`tests/ui-fulldeps/internal-lints/diagnostics.rs`.
Fix duplicated path in the "not found dylib" error
While working on the gcc backend, I couldn't figure out why I had this error:
```
error: couldn't load codegen backend /checkout/compiler/rustc_codegen_gcc/target/release/librustc_codegen_gcc.so/checkout/compiler/rustc_codegen_gcc/target/release/librustc_codegen_gcc.so: cannot open shared object file: No such file or directory
```
As you can see, the path is duplicated for some reason. After investigating a bit more, I realized that `libloading::Error::LoadLibraryExW` starts with the path of the not found dylib, making it appear twice in our error afterward (because we do render it like this: `{path}{err}`, and since the `err` starts with the path...).
Thanks to `````@bjorn3````` for linking me to https://github.com/rust-lang/rust/pull/121392. :)
Add a scheme for moving away from `extern "rust-intrinsic"` entirely
All `rust-intrinsic`s can become free functions now, either with a fallback body, or with a dummy body and an attribute, requiring backends to actually implement the intrinsic.
This PR demonstrates the dummy-body scheme with the `vtable_size` intrinsic.
cc https://github.com/rust-lang/rust/issues/63585
follow-up to #120500
MCP at https://github.com/rust-lang/compiler-team/issues/720
Existing names for values of this type are `sess`, `parse_sess`,
`parse_session`, and `ps`. `sess` is particularly annoying because
that's also used for `Session` values, which are often co-located, and
it can be difficult to know which type a value named `sess` refers to.
(That annoyance is the main motivation for this change.) `psess` is nice
and short, which is good for a name used this much.
The commit also renames some `parse_sess_created` values as
`psess_created`.
Unify dylib loading between proc macros and codegen backends
As bonus this makes the errors when failing to load a proc macro more informative to match the backend loading errors. In addition it makes it slightly easier to patch rustc to work on platforms that don't support dynamic linking like wasm.
Currently `has_errors` excludes lint errors. This commit changes it to
include lint errors.
The motivation for this is that for most places it doesn't matter
whether lint errors are included or not. But there are multiple places
where they must be includes, and only one place where they must not be
included. So it makes sense for `has_errors` to do the thing that fits
the most situations, and the new `has_errors_excluding_lint_errors`
method in the one exceptional place.
The same change is made for `err_count`. Annoyingly, this requires the
introduction of `err_count_excluding_lint_errs` for one place, to
preserve existing error printing behaviour. But I still think the change
is worthwhile overall.
As bonus this makes the errors when failing to load a proc macro more
informative to match the backend loading errors. In addition it makes it
slightly easier to patch rustc to work on platforms that don't support
dynamic linking like wasm.
Currently many diagnostic modifier methods are available on both
`Diagnostic` and `DiagnosticBuilder`. This commit removes most of them
from `Diagnostic`. To minimize the diff size, it keeps them within
`diagnostic.rs` but changes the surrounding `impl Diagnostic` block to
`impl DiagnosticBuilder`. (I intend to move things around later, to give
a more sensible code layout.)
`Diagnostic` keeps a few methods that it still needs, like `sub`,
`arg`, and `replace_args`.
The `forward!` macro, which defined two additional methods per call
(e.g. `note` and `with_note`), is replaced by the `with_fn!` macro,
which defines one additional method per call (e.g. `with_note`). It's
now also only used when necessary -- not all modifier methods currently
need a `with_*` form. (New ones can be easily added as necessary.)
All this also requires changing `trait AddToDiagnostic` so its methods
take `DiagnosticBuilder` instead of `Diagnostic`, which leads to many
mechanical changes. `SubdiagnosticMessageOp` gains a type parameter `G`.
There are three subdiagnostics -- `DelayedAtWithoutNewline`,
`DelayedAtWithNewline`, and `InvalidFlushedDelayedDiagnosticLevel` --
that are created within the diagnostics machinery and appended to
external diagnostics. These are handled at the `Diagnostic` level, which
means it's now hard to construct them via `derive(Diagnostic)`, so
instead we construct them by hand. This has no effect on what they look
like when printed.
There are lots of new `allow` markers for `untranslatable_diagnostics`
and `diagnostics_outside_of_impl`. This is because
`#[rustc_lint_diagnostics]` annotations were present on the `Diagnostic`
modifier methods, but missing from the `DiagnosticBuilder` modifier
methods. They're now present.
fixes#117448
For example unnecessary imports in std::prelude that can be eliminated:
```rust
use std::option::Option::Some;//~ WARNING the item `Some` is imported redundantly
use std::option::Option::None; //~ WARNING the item `None` is imported redundantly
```
Implement intrinsics with fallback bodies
fixes#93145 (though we can port many more intrinsics)
cc #63585
The way this works is that the backend logic for generating custom code for intrinsics has been made fallible. The only failure path is "this intrinsic is unknown". The `Instance` (that was `InstanceDef::Intrinsic`) then gets converted to `InstanceDef::Item`, which represents the fallback body. A regular function call to that body is then codegenned. This is currently implemented for
* codegen_ssa (so llvm and gcc)
* codegen_cranelift
other backends will need to adjust, but they can just keep doing what they were doing if they prefer (though adding new intrinsics to the compiler will then require them to implement them, instead of getting the fallback body).
cc `@scottmcm` `@WaffleLapkin`
### todo
* [ ] miri support
* [x] default intrinsic name to name of function instead of requiring it to be specified in attribute
* [x] make sure that the bodies are always available (must be collected for metadata)
Merge `impl_polarity` and `impl_trait_ref` queries
Hopefully this is perf neutral. I want to finish https://github.com/rust-lang/rust/pull/120835 and stop using the HIR in `coherent_trait`, which should then give us a perf improvement.
Invert diagnostic lints.
That is, change `diagnostic_outside_of_impl` and `untranslatable_diagnostic` from `allow` to `deny`, because more than half of the compiler has been converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow` attributes, which proves that this change is warranted.
r? ````@davidtwco````
resolve: Unload speculatively resolved crates before freezing cstore
Name resolution sometimes loads additional crates to improve diagnostics (e.g. suggest imports).
Not all of these diagnostics result in errors, sometimes they are just warnings, like in #117772.
If additional crates loaded speculatively stay and gets listed by things like `query crates` then they may produce further errors like duplicated lang items, because lang items from speculatively loaded crates are as good as from non-speculatively loaded crates.
They can probably do things like adding unintended impls from speculatively loaded crates to method resolution as well.
The extra crates will also get into the crate's metadata as legitimate dependencies.
In this PR I remove the speculative crates from cstore when name resolution is finished and cstore is frozen.
This is better than e.g. filtering away speculative crates in `query crates` because things like `DefId`s referring to these crates and leaking to later compilation stages can produce ICEs much easier, allowing to detect them.
The unloading could potentially be skipped if any errors were reported (to allow using `DefId`s from speculatively loaded crates for recovery), but I didn't do it in this PR because I haven't seen such cases of recovery. We can reconsider later if any relevant ICEs are reported.
Unblocks https://github.com/rust-lang/rust/pull/117772.
That is, change `diagnostic_outside_of_impl` and
`untranslatable_diagnostic` from `allow` to `deny`, because more than
half of the compiler has be converted to use translated diagnostics.
This commit removes more `deny` attributes than it adds `allow`
attributes, which proves that this change is warranted.
Because it's almost always static.
This makes `impl IntoDiagnosticArg for DiagnosticArgValue` trivial,
which is nice.
There are a few diagnostics constructed in
`compiler/rustc_mir_build/src/check_unsafety.rs` and
`compiler/rustc_mir_transform/src/errors.rs` that now need symbols
converted to `String` with `to_string` instead of `&str` with `as_str`,
but that' no big deal, and worth it for the simplifications elsewhere.
Error codes are integers, but `String` is used everywhere to represent
them. Gross!
This commit introduces `ErrCode`, an integral newtype for error codes,
replacing `String`. It also introduces a constant for every error code,
e.g. `E0123`, and removes the `error_code!` macro. The constants are
imported wherever used with `use rustc_errors::codes::*`.
With the old code, we have three different ways to specify an error code
at a use point:
```
error_code!(E0123) // macro call
struct_span_code_err!(dcx, span, E0123, "msg"); // bare ident arg to macro call
\#[diag(name, code = "E0123")] // string
struct Diag;
```
With the new code, they all use the `E0123` constant.
```
E0123 // constant
struct_span_code_err!(dcx, span, E0123, "msg"); // constant
\#[diag(name, code = E0123)] // constant
struct Diag;
```
The commit also changes the structure of the error code definitions:
- `rustc_error_codes` now just defines a higher-order macro listing the
used error codes and nothing else.
- Because that's now the only thing in the `rustc_error_codes` crate, I
moved it into the `lib.rs` file and removed the `error_codes.rs` file.
- `rustc_errors` uses that macro to define everything, e.g. the error
code constants and the `DIAGNOSTIC_TABLES`. This is in its new
`codes.rs` file.
Suggest Upgrading Compiler for Gated Features
This PR addresses #117318
I have a few questions:
1. Do we want to specify the current version and release date of the compiler? I have added this in via environment variables, which I found in the code for the rustc cli where it handles the `--version` flag
a. How can I handle the changing message in the tests?
3. Do we want to only show this message when the compiler is old?
a. How can we determine when the compiler is old?
I'll wait until we figure out the message to bless the tests
Varargs support for system ABI
This PR allows functions with the `system` ABI to be variadic (under the `extended_varargs_abi_support` feature tracked in #100189). On x86 windows, the `system` ABI is equivalent to `C` for variadic functions. On other platforms, `system` is already equivalent to `C`.
Fixes#110505
Avoid specialization in the metadata serialization code
With the exception of a perf-only specialization for byte slices and byte vectors.
This uses the same trick of introducing a new trait and having the Encodable and Decodable derives add a bound to it as used for TyEncoder/TyDecoder. The new code is clearer about which encoder/decoder uses which impl and it reduces the dependency of rustc on specialization, making it easier to remove support for specialization entirely or turn it into a construct that is only allowed for perf optimizations if we decide to do this.
Rollup of 8 pull requests
Successful merges:
- #119151 (Hide foreign `#[doc(hidden)]` paths in import suggestions)
- #119350 (Imply outlives-bounds on lazy type aliases)
- #119354 (Make `negative_bounds` internal & fix some of its issues)
- #119506 (Use `resolutions(()).effective_visiblities` to avoid cycle errors in `report_object_error`)
- #119554 (Fix scoping for let chains in match guards)
- #119563 (Check yield terminator's resume type in borrowck)
- #119589 (cstore: Remove unnecessary locking from `CrateMetadata`)
- #119622 (never patterns: Document behavior of never patterns with macros-by-example)
r? `@ghost`
`@rustbot` modify labels: rollup
cstore: Remove unnecessary locking from `CrateMetadata`
Locks and atomics in `CrateMetadata` fields were necessary before https://github.com/rust-lang/rust/pull/107765 when `CStore` was cloneable, but now they are not necessary and can be removed after restructuring the code a bit to please borrow checker.
All remaining locked fields in `CrateMetadata` are lazily populated caches.
Replace a number of FxHashMaps/Sets with stable-iteration-order alternatives
This PR replaces almost all of the remaining `FxHashMap`s in query results with either `FxIndexMap` or `UnordMap`. The only case that is missing is the `EffectiveVisibilities` struct which turned out to not be straightforward to transform. Once that is done too, we can remove the `HashStable` implementation from `HashMap`.
The first commit adds the `StableCompare` trait which is a companion trait to `StableOrd`. Some types like `Symbol` can be compared in a cross-session stable way, but their `Ord` implementation is not stable. In such cases, a `StableCompare` implementation can be provided to offer a lightweight way for stable sorting. The more heavyweight option is to sort via `ToStableHashKey`, but then sorting needs to have access to a stable hashing context and `ToStableHashKey` can also be expensive as in the case of `Symbol` where it has to allocate a `String`.
The rest of the commits are rather mechanical and don't overlap, so they are best reviewed individually.
Part of [MCP 533](https://github.com/rust-lang/compiler-team/issues/533).
Report I/O errors from rmeta encoding with emit_fatal
https://github.com/rust-lang/rust/issues/119456 reminded me that I never did systematic testing to provoke the out-of-disk ICEs so I grepped through a recent crater run (https://github.com/rust-lang/rust/pull/119440#issuecomment-1873393963) for more out-of-disk ICEs on current master and yep there's 2 in there.
So I finally cooked up a way to provoke for these crashes. I wrote a little `cdylib` crate that has a `#[no_mangle] pub extern "C" fn write` which occasionally reports `ENOSPC`, and prints a backtrace when it does.
<details><summary><strong>code for the dylib</strong></summary>
<p>
```rust
// cargo add libc rand backtrace
use rand::Rng;
#[no_mangle]
pub extern "C" fn write(
fd: libc::c_int,
buf: *const libc::c_void,
count: libc::size_t,
) -> libc::ssize_t {
if fd > 2 && rand::thread_rng().gen::<u8>() == 0 {
let mut count = 0;
backtrace::trace(|frame| {
backtrace::resolve_frame(frame, |symbol| {
if let Some(name) = symbol.name() {
if count > 3 {
eprintln!("{}", name);
}
}
count += 1;
});
true
});
unsafe {
*libc::__errno_location() = libc::ENOSPC;
}
return -1;
} else {
unsafe {
let res =
libc::syscall(libc::SYS_write, fd as usize, buf as usize, count as usize) as isize;
if res < 0 {
*libc::__errno_location() = -res as i32;
-1
} else {
res
}
}
}
}
```
</p>
</details>
Then `LD_PRELOAD` that dylib and repeatedly build a big project until it ICEs, such as with this:
```bash
while true; do
cargo clean
LD_PRELOAD=/home/ben/evil/target/release/libevil.so cargo +stage1 check 2> errors
if grep "thread 'rustc' panicked" errors; then
break
fi
done
```
My "big project" for testing was an otherwise-empty project with `cargo add axum`.
Before this PR, the above procedure finds a crash in between 1 and 15 minutes. With this PR, I have not found a crash in 30 minutes, and I'll be leaving this to run overnight (starting now). (A night has now passed, no crashes were found)
I believe the problem is that even though since https://github.com/rust-lang/rust/pull/117301 we correctly check `FileEncoder` for errors on all paths, we use `emit_err`, so there is a window of time between the call to `emit_err` and the full error reporting where rustc believes it has emitted a valid rmeta file and will permit Cargo to launch a build for a dependent crate. Changing these calls to `emit_fatal` closes that window.
I think there are a number of other cases where `emit_err` has been used instead of the more-correct `emit_fatal` such as e51e98dde6/compiler/rustc_codegen_ssa/src/back/write.rs (L542) but unlike rmeta encoding I am not aware of those cases of those causing problems.
r? ``@WaffleLapkin``
`Diagnostic` has 40 methods that return `&mut Self` and could be
considered setters. Four of them have a `set_` prefix. This doesn't seem
necessary for a type that implements the builder pattern. This commit
removes the `set_` prefixes on those four methods.
This involves lots of breaking changes. There are two big changes that
force changes. The first is that the bitflag types now don't
automatically implement normal derive traits, so we need to derive them
manually.
Additionally, bitflags now have a hidden inner type by default, which
breaks our custom derives. The bitflags docs recommend using the impl
form in these cases, which I did.
Spans are now stored in a more compact form which cuts down on at least
1 byte per span (indirect/direct encoding) and at most 3 bytes per span
(indirect/direct encoding, context byte, length byte). As a result,
libcore metadata shrinks by 1.5MB.
Support encoding spans with relative offsets
The relative offset is often smaller than the absolute offset, and with
the LEB128 encoding, this ends up cutting the overall metadata size
considerably (~1.5 megabytes on libcore). We can support both relative
and absolute encodings essentially for free since we already take a full
byte to differentiate between direct and indirect encodings (so an extra
variant is quite cheap).
