Fix argument removal suggestion around macros
Fixes#112437.
Fixes#113866.
Helps with #114255.
The issue was that `span.find_ancestor_inside(outer)` could previously return a span with a different expansion context from `outer`.
This happens for example for the built-in macro `panic!`, which expands to another macro call of `panic_2021!` or `panic_2015!`. Because the call site of `panic_20xx!` has not associated source code, its span currently points to the call site of `panic!` instead.
Something similar also happens items that get desugared in AST->HIR lowering. For example, `for` loops get two spans: One "inner" span that has the `.desugaring_kind()` kind set to `DesugaringKind::ForLoop` and one "outer" span that does not. Similar to the macro situation, both of these spans point to the same source code, but have different expansion contexts.
This causes problems, because joining two spans with different expansion contexts will usually[^1] not actually join them together to avoid creating "spaghetti" spans that go from the macro definition to the macro call. For example, in the following snippet `full_span` might not actually contain the `adjusted_start` and `adjusted_end`. This caused the broken suggestion / debug ICE in the linked issues.
```rust
let adjusted_start = start.find_ancestor_inside(shared_ancestor);
let adjusted_end = end.find_ancestor_inside(shared_ancestor);
let full_span = adjusted_start.to(adjusted_end)
```
To fix the issue, this PR introduces a new method, `find_ancestor_inside_same_ctxt`, which combines the functionality of `find_ancestor_inside` and `find_ancestor_in_same_ctxt`: It finds an ancestor span that is contained within the parent *and* has the same syntax context, and is therefore safe to extend. This new method should probably be used everywhere, where the returned span is extended, but for now it is just used for the argument removal suggestion.
Additionally, this PR fixes a second issue where the function call itself is inside a macro but the arguments come from outside the macro. The test is added in the first commit to include stderr diff, so this is best reviewed commit by commit.
[^1]: If one expansion context is the root context and the other is not.
It lints against features that are inteded to be internal to the
compiler and standard library. Implements MCP #596.
We allow `internal_features` in the standard library and compiler as those
use many features and this _is_ the standard library from the "internal to the compiler and
standard library" after all.
Marking some features as internal wasn't exactly the most scientific approach, I just marked some
mostly obvious features. While there is a categorization in the macro,
it's not very well upheld (should probably be fixed in another PR).
We always pass `-Ainternal_features` in the testsuite
About 400 UI tests and several other tests use internal features.
Instead of throwing the attribute on each one, just always allow them.
There's nothing wrong with testing internal features^^
Mark `ErrorGuaranteed` constructor as deprecated so people don't use it
You should never ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever ever use this function unless you know what you're doing, so make it harder to accidentally use it!
Alternatives are to change the name to sound scarier, make it `unsafe` (though it's not really a soundness thing), or work on deeper refactors to make it private.
r? `@BoxyUwU`
Make `impl Debug for Span` not panic on not having session globals.
I hit the panic that this patch avoids while messing with the early lints in `rustc_session::config::build_session_options()`. The rest of that project is not finished, but this seemed like a self-contained improvement.
(Should changes like this add tests? I don't see similar unit tests.)
Remove some suspicious cast truncations
These truncations were added a long time ago, and as best I can tell without a perf justification. And with rust-lang/rust#110410 it has become perf-neutral to not truncate anymore. We worked hard for all these bits, let's use them.
Rename `with_source_map` as `set_source_map`. Because `with` functions
(e.g. `with_session_globals`, `scoped_tls::ScopedKey::with`) are for
*getting* a value for the duration of a closure, and `set` functions
(e.g. `set_session_globals_then` `scoped_tls::ScopedKey::with`) are for
*setting* a value for the duration of a closure.
Also fix up the comment, which is wrong:
- The bit about `TyCtxt` is wrong.
- `span_debug1` doesn't exist any more.
- There's only one level of fallback, not two.
(This is effectively a follow-up to the changes in #93936.)
Also add a comment explaining that `SessionGlobals::source_map` should
only be used when absolutely necessary.
Convert all the crates that have had their diagnostic migration
completed (except save_analysis because that will be deleted soon and
apfloat because of the licensing problem).
Fix line numbers for MIR inlined code
`should_collapse_debuginfo` detects if the specified span is part of a
macro expansion however it does this by checking if the span is anything
other than a normal (non-expanded) kind, then the span sequence is
walked backwards to the root span.
This doesn't work when the MIR inliner inlines code as it creates spans
with expansion information set to `ExprKind::Inlined` and results in the
line number being attributed to the inline callsite rather than the
normal line number of the inlined code.
Fixes#103068
Shorten the `lookup_line` code slightly
The `match` looks like it's exactly the same as `checked_sub(1)`, so we might as well see if perf says we can just do that to save a couple lines.
`should_collapse_debuginfo` detects if the specified span is part of a
macro expansion however it does this by checking if the span is anything
other than a normal (non-expanded) kind, then the span sequence is
walked backwards to the root span.
This doesn't work when the MIR inliner inlines code as it creates spans
with expansion information set to `ExprKind::Inlined` and results in the
line number being attributed to the inline callsite rather than the
normal line number of the inlined code.
Inline SyntaxContext in both encoded span representation.
The current interned representation for spans does not use the `ctxt_or_zero: u16` field. This PR proposes to use this field to store the `SyntaxContext` of the interned span instead. When `ctxt_or_zero` and the interned span's `ctxt` don't match, the inlined one takes precedence.
This allows to implement `Span::ctxt` and `Span::with_ctxt` with much less probability to access the interner. Those functions are used a lot for hygiene, so this may be worth it.
On later stages, the feature is already stable.
Result of running:
rg -l "feature.let_else" compiler/ src/librustdoc/ library/ | xargs sed -s -i "s#\\[feature.let_else#\\[cfg_attr\\(bootstrap, feature\\(let_else\\)#"
Debuginfo line information for macro invocations are collapsed by
default - line information are replaced by the line of the outermost
expansion site. Using `-Zdebug-macros` disables this behaviour.
When the `collapse_debuginfo` feature is enabled, the default behaviour
is reversed so that debuginfo is not collapsed by default. In addition,
the `#[collapse_debuginfo]` attribute is available and can be applied to
macro definitions which will then have their line information collapsed.
Signed-off-by: David Wood <david.wood@huawei.com>
Add the diagnostic translation lints to crates that don't emit them
Some of these have a note saying that they should build on a stable compiler, does that mean they shouldn't get these lints? Or can we cfg them out on those?
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).
(This was previously merged as commit 5 in #94732 and then was reverted
in #97905 because of a perf regression caused by commit 4 in #94732.)
This avoids the name clash with `rustc_serialize::Encoder` (a trait),
and allows lots qualifiers to be removed and imports to be simplified
(e.g. fewer `as` imports).
There are two impls of the `Encoder` trait: `opaque::Encoder` and
`opaque::FileEncoder`. The former encodes into memory and is infallible, the
latter writes to file and is fallible.
Currently, standard `Result`/`?`/`unwrap` error handling is used, but this is a
bit verbose and has non-trivial cost, which is annoying given how rare failures
are (especially in the infallible `opaque::Encoder` case).
This commit changes how `Encoder` fallibility is handled. All the `emit_*`
methods are now infallible. `opaque::Encoder` requires no great changes for
this. `opaque::FileEncoder` now implements a delayed error handling strategy.
If a failure occurs, it records this via the `res` field, and all subsequent
encoding operations are skipped if `res` indicates an error has occurred. Once
encoding is complete, the new `finish` method is called, which returns a
`Result`. In other words, there is now a single `Result`-producing method
instead of many of them.
This has very little effect on how any file errors are reported if
`opaque::FileEncoder` has any failures.
Much of this commit is boring mechanical changes, removing `Result` return
values and `?` or `unwrap` from expressions. The more interesting parts are as
follows.
- serialize.rs: The `Encoder` trait gains an `Ok` associated type. The
`into_inner` method is changed into `finish`, which returns
`Result<Vec<u8>, !>`.
- opaque.rs: The `FileEncoder` adopts the delayed error handling
strategy. Its `Ok` type is a `usize`, returning the number of bytes
written, replacing previous uses of `FileEncoder::position`.
- Various methods that take an encoder now consume it, rather than being
passed a mutable reference, e.g. `serialize_query_result_cache`.
`SourceFile::lines` is a big part of metadata. It's stored in a compressed form
(a difference list) to save disk space. Decoding it is a big fraction of
compile time for very small crates/programs.
This commit introduces a new type `SourceFileLines` which has a `Lines`
form and a `Diffs` form. The latter is used when the metadata is first
read, and it is only decoded into the `Lines` form when line data is
actually needed. This avoids the decoding cost for many files,
especially in `std`. It's a performance win of up to 15% for tiny
crates/programs where metadata decoding is a high part of compilation
costs.
A `Lock` is needed because the methods that access lines data (which can
trigger decoding) take `&self` rather than `&mut self`. To allow for this,
`SourceFile::lines` now takes a `FnMut` that operates on the lines slice rather
than returning the lines slice.
Add support for embedding pretty printers via `#[debugger_visualizer]` attribute
Initial support for [RFC 3191](https://github.com/rust-lang/rfcs/pull/3191) in PR https://github.com/rust-lang/rust/pull/91779 was scoped to supporting embedding NatVis files using a new attribute. This PR implements the pretty printer support as stated in the RFC mentioned above.
This change includes embedding pretty printers in the `.debug_gdb_scripts` just as the pretty printers for rustc are embedded today. Also added additional tests for embedded pretty printers. Additionally cleaned up error checking so all error checking is done up front regardless of the current target.
RFC: https://github.com/rust-lang/rfcs/pull/3191
Ensure all error checking for `#[debugger_visualizer]` is done up front and not when the `debugger_visualizer` query is run.
Clean up potential ODR violations when embedding pretty printers into the `__rustc_debug_gdb_scripts_section__` section.
Respond to PR comments and update documentation.
Add a new Rust attribute to support embedding debugger visualizers
Implemented [this RFC](https://github.com/rust-lang/rfcs/pull/3191) to add support for embedding debugger visualizers into a PDB.
Added a new attribute `#[debugger_visualizer]` and updated the `CrateMetadata` to store debugger visualizers for crate dependencies.
RFC: https://github.com/rust-lang/rfcs/pull/3191
Cleanup `DebuggerVisualizerFile` type and other minor cleanup of queries.
Merge the queries for debugger visualizers into a single query.
Revert move of `resolve_path` to `rustc_builtin_macros`. Update dependencies in Cargo.toml for `rustc_passes`.
Respond to PR comments. Load visualizer files into opaque bytes `Vec<u8>`. Debugger visualizers for dynamically linked crates should not be embedded in the current crate.
Update the unstable book with the new feature. Add the tracking issue for the debugger_visualizer feature.
Respond to PR comments and minor cleanups.
The self-profiler's `EventArgRecorder` is general-purpose in its ability to record Strings (and `rustc_span` depends on the crate its defined in, `rustc_data_structure`).
Some generic activities could use recording locations where they happen in the user's code: to allow e.g. to track macro expansions and diagnose performance issues there.
This adds a `SpannedEventArgRecorder` that can record an argument given as a span, rather than a String, since turning spans into Strings can be tricky if you're not happy with its default Debug output. This way the recorder can have a `record_arg_spanned` method which will do that.
Create (unstable) 2024 edition
[On Zulip](https://rust-lang.zulipchat.com/#narrow/stream/213817-t-lang/topic/Deprecating.20macro.20scoping.20shenanigans/near/272860652), there was a small aside regarding creating the 2024 edition now as opposed to later. There was a reasonable amount of support and no stated opposition.
This change creates the 2024 edition in the compiler and creates a prelude for the 2024 edition. There is no current difference between the 2021 and 2024 editions. Cargo and other tools will need to be updated separately, as it's not in the same repository. This change permits the vast majority of work towards the next edition to proceed _now_ instead of waiting until 2024.
For sanity purposes, I've merged the "hello" UI tests into a single file with multiple revisions. Otherwise we'd end up with a file per edition, despite them being essentially identical.
````@rustbot```` label +T-lang +S-waiting-on-review
Not sure on the relevant team, to be honest.
`MultiSpan` contains labels, which are more complicated with the
introduction of diagnostic translation and will use types from
`rustc_errors` - however, `rustc_errors` depends on `rustc_span` so
`rustc_span` cannot use types like `DiagnosticMessage` without
dependency cycles. Introduce a new `rustc_error_messages` crate that can
contain `DiagnosticMessage` and `MultiSpan`.
Signed-off-by: David Wood <david.wood@huawei.com>
The only difference between the default and rustc_interface set version
is that the default accesses the source map from SESSION_GLOBALS while
the rustc_interface version accesses the source map from the global
TyCtxt. SESSION_GLOBALS is always set while running the compiler while
the global TyCtxt is not always set. If the global TyCtxt is set, it's
source map is identical to the one in SESSION_GLOBALS
Incorporate distance limit from `find_best_match_for_name` directly into
Levenshtein distance computation.
Use the string size difference as a lower bound on the distance and exit
early when it exceeds the specified limit.
After finding a candidate within a limit, lower the limit further to
restrict the search space.
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
currently panics on failure (e.g. if the input is too short, or on a
bad `Result` discriminant), and in some places it returns an error
(e.g. on a bad `Option` discriminant). The number of places where
either happens is surprisingly small, just because the binary
representation has very little redundancy and a lot of input reading
can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
`.rlink` file production, and there's a `FIXME` comment suggesting it
should change to a binary format, and (b) in a few tests in
non-fundamental ways. Indeed #85993 is open to remove it entirely.
And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.
Much of this commit is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
optimization for small counts that the impl for `Result<T, E>` has,
because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
`collect`, which is nice; the one for `Vec` uses unsafe code, because
that gave better perf on some benchmarks.
Fixes#92266
In some `HashStable` impls, we use a cache to avoid re-computing
the same `Fingerprint` from the same structure (e.g. an `AdtDef`).
However, the `StableHashingContext` used can be configured to
perform hashing in different ways (e.g. skipping `Span`s). This
configuration information is not included in the cache key,
which will cause an incorrect `Fingerprint` to be used if
we hash the same structure with different `StableHashingContext`
settings.
To fix this, the configuration settings of `StableHashingContext`
are split out into a separate `HashingControls` struct. This
struct is used as part of the cache key, ensuring that our caches
always produce the correct result for the given settings.
With this in place, we now turn off `Span` hashing during the
entire process of computing the hash included in legacy symbols.
This current has no effect, but will matter when a future PR
starts hashing more `Span`s that we currently skip.
std: Stabilize the `thread_local_const_init` feature
This commit is intended to follow the stabilization disposition of the
FCP that has now finished in #84223. This stabilizes the ability to flag
thread local initializers as `const` expressions which enables the macro
to generate more efficient code for accessing it, notably removing
runtime checks for initialization.
More information can also be found in #84223 as well as the tests where
the feature usage was removed in this PR.
Closes#84223
* Annotate `derive`d spans from the user's code with the appropciate context
* Add `Span::can_be_used_for_suggestion` to query if the underlying span
at the users' code
This commit is intended to follow the stabilization disposition of the
FCP that has now finished in #84223. This stabilizes the ability to flag
thread local initializers as `const` expressions which enables the macro
to generate more efficient code for accessing it, notably removing
runtime checks for initialization.
More information can also be found in #84223 as well as the tests where
the feature usage was removed in this PR.
Closes#84223
This also fixes the same suggestion, which was kind of broken, because it just searched for the last occurence of `const` to replace with a `let`. This works great in some cases, but when there is no const and a leading space to the file, it doesn't work and panic with overflow because it thought that it had found a const.
I also changed the suggestion to only trigger if the `const` and the non-constant value are on the same line, because if they aren't, the suggestion is very likely to be wrong.
Also don't trigger the suggestion if the found `const` is on line 0, because that triggers the ICE.
"Fix" an overflow in byte position math
r? `@estebank`
help! I fixed the ICE only to brick the diagnostic.
I mean, it was wrong previously (using an already expanded macro span), but it is really bad now XD
Path remapping: Make behavior of diagnostics output dependent on presence of --remap-path-prefix.
This PR fixes a regression (#87745) with `--remap-path-prefix` where the flag stopped causing diagnostic messages to be remapped as well. The regression was introduced in https://github.com/rust-lang/rust/pull/83813 where we erroneously assumed that remapping of diagnostic messages was not desired anymore (because #70642 partially undid that functionality with nobody objecting).
The issue is fixed by making `--remap-path-prefix` remap diagnostic messages again, including for paths that have been remapped in upstream crates (e.g. the standard library). This means that "sysroot-localization" (implemented in #70642) is also disabled if `rustc` is invoked with `--remap-path-prefix`. The assumption is that once someone starts explicitly remapping paths they also don't want paths to their local Rust installation in their build output.
In the future we might want to give more fine-grained control over this behavior via compiler flags (see https://github.com/rust-lang/rfcs/pull/3127 for a related RFC). For now this PR is intended as a regression fix.
This PR is an alternative to https://github.com/rust-lang/rust/pull/88191, which makes diagnostic messages be remapped unconditionally. That approach, however, would effectively revert #70642.
Fixes https://github.com/rust-lang/rust/issues/87745.
cc `@cbeuw`
r? `@ghost`
These impls were unused, and incorrectly hashed the local
(non-remapped) path for `RealFileName::Remapped` (which would
break reproducible builds if these impls were used).
Fix `--remap-path-prefix` not correctly remapping `rust-src` component paths and unify handling of path mapping with virtualized paths
This PR fixes#73167 ("Binaries end up containing path to the rust-src component despite `--remap-path-prefix`") by preventing real local filesystem paths from reaching compilation output if the path is supposed to be remapped.
`RealFileName::Named` introduced in #72767 is now renamed as `LocalPath`, because this variant wraps a (most likely) valid local filesystem path.
`RealFileName::Devirtualized` is renamed as `Remapped` to be used for remapped path from a real path via `--remap-path-prefix` argument, as well as real path inferred from a virtualized (during compiler bootstrapping) `/rustc/...` path. The `local_path` field is now an `Option<PathBuf>`, as it will be set to `None` before serialisation, so it never reaches any build output. Attempting to serialise a non-`None` `local_path` will cause an assertion faliure.
When a path is remapped, a `RealFileName::Remapped` variant is created. The original path is preserved in `local_path` field and the remapped path is saved in `virtual_name` field. Previously, the `local_path` is directly modified which goes against its purpose of "suitable for reading from the file system on the local host".
`rustc_span::SourceFile`'s fields `unmapped_path` (introduced by #44940) and `name_was_remapped` (introduced by #41508 when `--remap-path-prefix` feature originally added) are removed, as these two pieces of information can be inferred from the `name` field: if it's anything other than a `FileName::Real(_)`, or if it is a `FileName::Real(RealFileName::LocalPath(_))`, then clearly `name_was_remapped` would've been false and `unmapped_path` would've been `None`. If it is a `FileName::Real(RealFileName::Remapped{local_path, virtual_name})`, then `name_was_remapped` would've been true and `unmapped_path` would've been `Some(local_path)`.
cc `@eddyb` who implemented `/rustc/...` path devirtualisation
This PR implements span quoting, allowing proc-macros to produce spans
pointing *into their own crate*. This is used by the unstable
`proc_macro::quote!` macro, allowing us to get error messages like this:
```
error[E0412]: cannot find type `MissingType` in this scope
--> $DIR/auxiliary/span-from-proc-macro.rs:37:20
|
LL | pub fn error_from_attribute(_args: TokenStream, _input: TokenStream) -> TokenStream {
| ----------------------------------------------------------------------------------- in this expansion of procedural macro `#[error_from_attribute]`
...
LL | field: MissingType
| ^^^^^^^^^^^ not found in this scope
|
::: $DIR/span-from-proc-macro.rs:8:1
|
LL | #[error_from_attribute]
| ----------------------- in this macro invocation
```
Here, `MissingType` occurs inside the implementation of the proc-macro
`#[error_from_attribute]`. Previosuly, this would always result in a
span pointing at `#[error_from_attribute]`
This will make many proc-macro-related error message much more useful -
when a proc-macro generates code containing an error, users will get an
error message pointing directly at that code (within the macro
definition), instead of always getting a span pointing at the macro
invocation site.
This is implemented as follows:
* When a proc-macro crate is being *compiled*, it causes the `quote!`
macro to get run. This saves all of the sapns in the input to `quote!`
into the metadata of *the proc-macro-crate* (which we are currently
compiling). The `quote!` macro then expands to a call to
`proc_macro::Span::recover_proc_macro_span(id)`, where `id` is an
opaque identifier for the span in the crate metadata.
* When the same proc-macro crate is *run* (e.g. it is loaded from disk
and invoked by some consumer crate), the call to
`proc_macro::Span::recover_proc_macro_span` causes us to load the span
from the proc-macro crate's metadata. The proc-macro then produces a
`TokenStream` containing a `Span` pointing into the proc-macro crate
itself.
The recursive nature of 'quote!' can be difficult to understand at
first. The file `src/test/ui/proc-macro/quote-debug.stdout` shows
the output of the `quote!` macro, which should make this eaier to
understand.
This PR also supports custom quoting spans in custom quote macros (e.g.
the `quote` crate). All span quoting goes through the
`proc_macro::quote_span` method, which can be called by a custom quote
macro to perform span quoting. An example of this usage is provided in
`src/test/ui/proc-macro/auxiliary/custom-quote.rs`
Custom quoting currently has a few limitations:
In order to quote a span, we need to generate a call to
`proc_macro::Span::recover_proc_macro_span`. However, proc-macros
support renaming the `proc_macro` crate, so we can't simply hardcode
this path. Previously, the `quote_span` method used the path
`crate::Span` - however, this only works when it is called by the
builtin `quote!` macro in the same crate. To support being called from
arbitrary crates, we need access to the name of the `proc_macro` crate
to generate a path. This PR adds an additional argument to `quote_span`
to specify the name of the `proc_macro` crate. Howver, this feels kind
of hacky, and we may want to change this before stabilizing anything
quote-related.
Additionally, using `quote_span` currently requires enabling the
`proc_macro_internals` feature. The builtin `quote!` macro
has an `#[allow_internal_unstable]` attribute, but this won't work for
custom quote implementations. This will likely require some additional
tricks to apply `allow_internal_unstable` to the span of
`proc_macro::Span::recover_proc_macro_span`.
