rust/compiler/rustc_builtin_macros/src/cfg_eval.rs

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

270 lines
10 KiB
Rust
Raw Normal View History

2024-02-24 07:52:59 +00:00
use core::ops::ControlFlow;
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
use rustc_ast as ast;
use rustc_ast::mut_visit::MutVisitor;
use rustc_ast::ptr::P;
use rustc_ast::visit::{AssocCtxt, Visitor};
use rustc_ast::{Attribute, HasAttrs, HasTokens, NodeId, mut_visit, visit};
use rustc_errors::PResult;
use rustc_expand::base::{Annotatable, ExtCtxt};
use rustc_expand::config::StripUnconfigured;
use rustc_expand::configure;
use rustc_feature::Features;
2021-11-28 18:41:04 +00:00
use rustc_parse::parser::{ForceCollect, Parser};
use rustc_session::Session;
use rustc_span::Span;
use rustc_span::symbol::sym;
use smallvec::SmallVec;
use tracing::instrument;
2021-11-24 09:20:23 +00:00
use crate::util::{check_builtin_macro_attribute, warn_on_duplicate_attribute};
pub(crate) fn expand(
ecx: &mut ExtCtxt<'_>,
_span: Span,
meta_item: &ast::MetaItem,
annotatable: Annotatable,
) -> Vec<Annotatable> {
check_builtin_macro_attribute(ecx, meta_item, sym::cfg_eval);
warn_on_duplicate_attribute(ecx, &annotatable, sym::cfg_eval);
vec![cfg_eval(ecx.sess, ecx.ecfg.features, annotatable, ecx.current_expansion.lint_node_id)]
}
pub(crate) fn cfg_eval(
sess: &Session,
features: &Features,
annotatable: Annotatable,
lint_node_id: NodeId,
) -> Annotatable {
let features = Some(features);
CfgEval(StripUnconfigured { sess, features, config_tokens: true, lint_node_id })
.configure_annotatable(annotatable)
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
}
struct CfgEval<'a>(StripUnconfigured<'a>);
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
2024-02-24 07:52:59 +00:00
fn has_cfg_or_cfg_attr(annotatable: &Annotatable) -> bool {
struct CfgFinder;
2024-02-24 07:52:59 +00:00
impl<'ast> visit::Visitor<'ast> for CfgFinder {
type Result = ControlFlow<()>;
fn visit_attribute(&mut self, attr: &'ast Attribute) -> ControlFlow<()> {
if attr
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
.ident()
2024-02-24 07:52:59 +00:00
.is_some_and(|ident| ident.name == sym::cfg || ident.name == sym::cfg_attr)
{
ControlFlow::Break(())
} else {
ControlFlow::Continue(())
}
}
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
}
2024-02-24 07:52:59 +00:00
let res = match annotatable {
Annotatable::Item(item) => CfgFinder.visit_item(item),
Annotatable::AssocItem(item, ctxt) => CfgFinder.visit_assoc_item(item, *ctxt),
2024-02-24 07:52:59 +00:00
Annotatable::ForeignItem(item) => CfgFinder.visit_foreign_item(item),
Annotatable::Stmt(stmt) => CfgFinder.visit_stmt(stmt),
Annotatable::Expr(expr) => CfgFinder.visit_expr(expr),
Annotatable::Arm(arm) => CfgFinder.visit_arm(arm),
Annotatable::ExprField(field) => CfgFinder.visit_expr_field(field),
Annotatable::PatField(field) => CfgFinder.visit_pat_field(field),
Annotatable::GenericParam(param) => CfgFinder.visit_generic_param(param),
Annotatable::Param(param) => CfgFinder.visit_param(param),
Annotatable::FieldDef(field) => CfgFinder.visit_field_def(field),
Annotatable::Variant(variant) => CfgFinder.visit_variant(variant),
Annotatable::Crate(krate) => CfgFinder.visit_crate(krate),
};
res.is_break()
}
impl CfgEval<'_> {
fn configure<T: HasAttrs + HasTokens>(&mut self, node: T) -> Option<T> {
self.0.configure(node)
}
fn configure_annotatable(mut self, mut annotatable: Annotatable) -> Annotatable {
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
// Tokenizing and re-parsing the `Annotatable` can have a significant
// performance impact, so try to avoid it if possible
2024-02-24 07:52:59 +00:00
if !has_cfg_or_cfg_attr(&annotatable) {
return annotatable;
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
}
// The majority of parsed attribute targets will never need to have early cfg-expansion
// run (e.g. they are not part of a `#[derive]` or `#[cfg_eval]` macro input).
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
// Therefore, we normally do not capture the necessary information about `#[cfg]`
// and `#[cfg_attr]` attributes during parsing.
//
// Therefore, when we actually *do* run early cfg-expansion, we need to tokenize
// and re-parse the attribute target, this time capturing information about
// the location of `#[cfg]` and `#[cfg_attr]` in the token stream. The tokenization
// process is lossless, so this process is invisible to proc-macros.
let parse_annotatable_with: for<'a> fn(&mut Parser<'a>) -> PResult<'a, _> =
match annotatable {
Annotatable::Item(_) => {
|parser| Ok(Annotatable::Item(parser.parse_item(ForceCollect::Yes)?.unwrap()))
}
Annotatable::AssocItem(_, AssocCtxt::Trait) => |parser| {
Ok(Annotatable::AssocItem(
parser.parse_trait_item(ForceCollect::Yes)?.unwrap().unwrap(),
AssocCtxt::Trait,
))
},
Annotatable::AssocItem(_, AssocCtxt::Impl) => |parser| {
Ok(Annotatable::AssocItem(
parser.parse_impl_item(ForceCollect::Yes)?.unwrap().unwrap(),
AssocCtxt::Impl,
))
},
Annotatable::ForeignItem(_) => |parser| {
Ok(Annotatable::ForeignItem(
parser.parse_foreign_item(ForceCollect::Yes)?.unwrap().unwrap(),
))
},
Annotatable::Stmt(_) => |parser| {
Ok(Annotatable::Stmt(P(parser
.parse_stmt_without_recovery(false, ForceCollect::Yes)?
.unwrap())))
},
Annotatable::Expr(_) => {
|parser| Ok(Annotatable::Expr(parser.parse_expr_force_collect()?))
}
_ => unreachable!(),
};
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
// 'Flatten' all nonterminals (i.e. `TokenKind::Interpolated`)
// to `None`-delimited groups containing the corresponding tokens. This
// is normally delayed until the proc-macro server actually needs to
// provide a `TokenKind::Interpolated` to a proc-macro. We do this earlier,
// so that we can handle cases like:
//
// ```rust
// #[cfg_eval] #[cfg] $item
//```
//
// where `$item` is `#[cfg_attr] struct Foo {}`. We want to make
// sure to evaluate *all* `#[cfg]` and `#[cfg_attr]` attributes - the simplest
// way to do this is to do a single parse of a stream without any nonterminals.
let orig_tokens = annotatable.to_tokens().flattened();
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
// Re-parse the tokens, setting the `capture_cfg` flag to save extra information
// to the captured `AttrTokenStream` (specifically, we capture
// `AttrTokenTree::AttrsTarget` for all occurrences of `#[cfg]` and `#[cfg_attr]`)
let mut parser = Parser::new(&self.0.sess.psess, orig_tokens, None);
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
parser.capture_cfg = true;
match parse_annotatable_with(&mut parser) {
Ok(a) => annotatable = a,
Make `DiagnosticBuilder::emit` consuming. This works for most of its call sites. This is nice, because `emit` very much makes sense as a consuming operation -- indeed, `DiagnosticBuilderState` exists to ensure no diagnostic is emitted twice, but it uses runtime checks. For the small number of call sites where a consuming emit doesn't work, the commit adds `DiagnosticBuilder::emit_without_consuming`. (This will be removed in subsequent commits.) Likewise, `emit_unless` becomes consuming. And `delay_as_bug` becomes consuming, while `delay_as_bug_without_consuming` is added (which will also be removed in subsequent commits.) All this requires significant changes to `DiagnosticBuilder`'s chaining methods. Currently `DiagnosticBuilder` method chaining uses a non-consuming `&mut self -> &mut Self` style, which allows chaining to be used when the chain ends in `emit()`, like so: ``` struct_err(msg).span(span).emit(); ``` But it doesn't work when producing a `DiagnosticBuilder` value, requiring this: ``` let mut err = self.struct_err(msg); err.span(span); err ``` This style of chaining won't work with consuming `emit` though. For that, we need to use to a `self -> Self` style. That also would allow `DiagnosticBuilder` production to be chained, e.g.: ``` self.struct_err(msg).span(span) ``` However, removing the `&mut self -> &mut Self` style would require that individual modifications of a `DiagnosticBuilder` go from this: ``` err.span(span); ``` to this: ``` err = err.span(span); ``` There are *many* such places. I have a high tolerance for tedious refactorings, but even I gave up after a long time trying to convert them all. Instead, this commit has it both ways: the existing `&mut self -> Self` chaining methods are kept, and new `self -> Self` chaining methods are added, all of which have a `_mv` suffix (short for "move"). Changes to the existing `forward!` macro lets this happen with very little additional boilerplate code. I chose to add the suffix to the new chaining methods rather than the existing ones, because the number of changes required is much smaller that way. This doubled chainging is a bit clumsy, but I think it is worthwhile because it allows a *lot* of good things to subsequently happen. In this commit, there are many `mut` qualifiers removed in places where diagnostics are emitted without being modified. In subsequent commits: - chaining can be used more, making the code more concise; - more use of chaining also permits the removal of redundant diagnostic APIs like `struct_err_with_code`, which can be replaced easily with `struct_err` + `code_mv`; - `emit_without_diagnostic` can be removed, which simplifies a lot of machinery, removing the need for `DiagnosticBuilderState`.
2024-01-03 01:17:35 +00:00
Err(err) => {
err.emit();
return annotatable;
}
}
Implement token-based handling of attributes during expansion This PR modifies the macro expansion infrastructure to handle attributes in a fully token-based manner. As a result: * Derives macros no longer lose spans when their input is modified by eager cfg-expansion. This is accomplished by performing eager cfg-expansion on the token stream that we pass to the derive proc-macro * Inner attributes now preserve spans in all cases, including when we have multiple inner attributes in a row. This is accomplished through the following changes: * New structs `AttrAnnotatedTokenStream` and `AttrAnnotatedTokenTree` are introduced. These are very similar to a normal `TokenTree`, but they also track the position of attributes and attribute targets within the stream. They are built when we collect tokens during parsing. An `AttrAnnotatedTokenStream` is converted to a regular `TokenStream` when we invoke a macro. * Token capturing and `LazyTokenStream` are modified to work with `AttrAnnotatedTokenStream`. A new `ReplaceRange` type is introduced, which is created during the parsing of a nested AST node to make the 'outer' AST node aware of the attributes and attribute target stored deeper in the token stream. * When we need to perform eager cfg-expansion (either due to `#[derive]` or `#[cfg_eval]`), we tokenize and reparse our target, capturing additional information about the locations of `#[cfg]` and `#[cfg_attr]` attributes at any depth within the target. This is a performance optimization, allowing us to perform less work in the typical case where captured tokens never have eager cfg-expansion run.
2020-11-28 23:33:17 +00:00
// Now that we have our re-parsed `AttrTokenStream`, recursively configuring
// our attribute target will correctly configure the tokens as well.
match annotatable {
Annotatable::Item(item) => Annotatable::Item(self.flat_map_item(item).pop().unwrap()),
Annotatable::AssocItem(item, ctxt) => {
Annotatable::AssocItem(self.flat_map_assoc_item(item, ctxt).pop().unwrap(), ctxt)
}
Annotatable::ForeignItem(item) => {
Annotatable::ForeignItem(self.flat_map_foreign_item(item).pop().unwrap())
}
Annotatable::Stmt(stmt) => {
Annotatable::Stmt(P(self.flat_map_stmt(stmt.into_inner()).pop().unwrap()))
}
Annotatable::Expr(mut expr) => {
self.visit_expr(&mut expr);
Annotatable::Expr(expr)
}
_ => unreachable!(),
}
}
}
impl MutVisitor for CfgEval<'_> {
#[instrument(level = "trace", skip(self))]
fn visit_expr(&mut self, expr: &mut P<ast::Expr>) {
self.0.configure_expr(expr, false);
mut_visit::walk_expr(self, expr);
}
#[instrument(level = "trace", skip(self))]
fn visit_method_receiver_expr(&mut self, expr: &mut P<ast::Expr>) {
self.0.configure_expr(expr, true);
mut_visit::walk_expr(self, expr);
}
fn filter_map_expr(&mut self, expr: P<ast::Expr>) -> Option<P<ast::Expr>> {
let mut expr = configure!(self, expr);
mut_visit::walk_expr(self, &mut expr);
Some(expr)
}
fn flat_map_generic_param(
&mut self,
param: ast::GenericParam,
) -> SmallVec<[ast::GenericParam; 1]> {
let param = configure!(self, param);
mut_visit::walk_flat_map_generic_param(self, param)
}
fn flat_map_stmt(&mut self, stmt: ast::Stmt) -> SmallVec<[ast::Stmt; 1]> {
let stmt = configure!(self, stmt);
mut_visit::walk_flat_map_stmt(self, stmt)
}
fn flat_map_item(&mut self, item: P<ast::Item>) -> SmallVec<[P<ast::Item>; 1]> {
let item = configure!(self, item);
mut_visit::walk_flat_map_item(self, item)
}
fn flat_map_assoc_item(
&mut self,
item: P<ast::AssocItem>,
_ctxt: AssocCtxt,
) -> SmallVec<[P<ast::AssocItem>; 1]> {
let item = configure!(self, item);
mut_visit::walk_flat_map_item(self, item)
}
fn flat_map_foreign_item(
&mut self,
foreign_item: P<ast::ForeignItem>,
) -> SmallVec<[P<ast::ForeignItem>; 1]> {
let foreign_item = configure!(self, foreign_item);
mut_visit::walk_flat_map_item(self, foreign_item)
}
fn flat_map_arm(&mut self, arm: ast::Arm) -> SmallVec<[ast::Arm; 1]> {
let arm = configure!(self, arm);
mut_visit::walk_flat_map_arm(self, arm)
}
fn flat_map_expr_field(&mut self, field: ast::ExprField) -> SmallVec<[ast::ExprField; 1]> {
let field = configure!(self, field);
mut_visit::walk_flat_map_expr_field(self, field)
}
fn flat_map_pat_field(&mut self, fp: ast::PatField) -> SmallVec<[ast::PatField; 1]> {
let fp = configure!(self, fp);
mut_visit::walk_flat_map_pat_field(self, fp)
}
fn flat_map_param(&mut self, p: ast::Param) -> SmallVec<[ast::Param; 1]> {
let p = configure!(self, p);
mut_visit::walk_flat_map_param(self, p)
}
fn flat_map_field_def(&mut self, sf: ast::FieldDef) -> SmallVec<[ast::FieldDef; 1]> {
let sf = configure!(self, sf);
mut_visit::walk_flat_map_field_def(self, sf)
}
fn flat_map_variant(&mut self, variant: ast::Variant) -> SmallVec<[ast::Variant; 1]> {
let variant = configure!(self, variant);
mut_visit::walk_flat_map_variant(self, variant)
}
}