Only create a single expansion for each inline integration.
The inlining integrator used to create one expansion for each span from the callee body.
This PR reverses the logic to create a single expansion for the whole call,
which is more consistent with how macro expansions work for macros.
This should remove the large memory regression in #91743.
Fix several asm! related issues
This is a combination of several fixes, each split into a separate commit. Splitting these into PRs is not practical since they conflict with each other.
Fixes#92378Fixes#85247
r? ``@nagisa``
The previous approach of checking for the reserve-r9 target feature
didn't actually work because LLVM only sets this feature very late when
initializing the per-function subtarget.
Adopt let else in more places
Continuation of #89933, #91018, #91481, #93046, #93590, #94011.
I have extended my clippy lint to also recognize tuple passing and match statements. The diff caused by fixing it is way above 1 thousand lines. Thus, I split it up into multiple pull requests to make reviewing easier. This is the biggest of these PRs and handles the changes outside of rustdoc, rustc_typeck, rustc_const_eval, rustc_trait_selection, which were handled in PRs #94139, #94142, #94143, #94144.
Add MemTagSanitizer Support
Add support for the LLVM [MemTagSanitizer](https://llvm.org/docs/MemTagSanitizer.html).
On hardware which supports it (see caveats below), the MemTagSanitizer can catch bugs similar to AddressSanitizer and HardwareAddressSanitizer, but with lower overhead.
On a tag mismatch, a SIGSEGV is signaled with code SEGV_MTESERR / SEGV_MTEAERR.
# Usage
`-Zsanitizer=memtag -C target-feature="+mte"`
# Comments/Caveats
* MemTagSanitizer is only supported on AArch64 targets with hardware support
* Requires `-C target-feature="+mte"`
* LLVM MemTagSanitizer currently only performs stack tagging.
# TODO
* Tests
* Example
Add a stack-`pin!`-ning macro to `core::pin`.
- https://github.com/rust-lang/rust/issues/93178
`pin!` allows pinning a value to the stack. Thanks to being implemented in the stdlib, which gives access to `macro` macros, and to the private `.pointer` field of the `Pin` wrapper, [it was recently discovered](https://rust-lang.zulipchat.com/#narrow/stream/187312-wg-async-foundations/topic/pin!.20.E2.80.94.20the.20.22definitive.22.20edition.20.28a.20rhs-compatible.20pin-nin.2E.2E.2E/near/268731241) ([archive link](https://zulip-archive.rust-lang.org/stream/187312-wg-async-foundations/topic/A.20rhs-compatible.20pin-ning.20macro.html#268731241)), contrary to popular belief, that it is actually possible to implement and feature such a macro:
```rust
let foo: Pin<&mut PhantomPinned> = pin!(PhantomPinned);
stuff(foo);
```
or, directly:
```rust
stuff(pin!(PhantomPinned));
```
- For context, historically, this used to require one of the two following syntaxes:
- ```rust
let foo = PhantomPinned;
pin!(foo);
stuff(foo);
```
- ```rust
pin! {
let foo = PhantomPinned;
}
stuff(foo);
```
This macro thus allows, for instance, doing things like:
```diff
fn block_on<T>(fut: impl Future<Output = T>) -> T {
// Pin the future so it can be polled.
- let mut fut = Box::pin(fut);
+ let mut fut = pin!(fut);
// Create a new context to be passed to the future.
let t = thread::current();
let waker = Arc::new(ThreadWaker(t)).into();
let mut cx = Context::from_waker(&waker);
// Run the future to completion.
loop {
match fut.as_mut().poll(&mut cx) {
Poll::Ready(res) => return res,
Poll::Pending => thread::park(),
}
}
}
```
- _c.f._, https://doc.rust-lang.org/1.58.1/alloc/task/trait.Wake.html
And so on, and so forth.
I don't think such an API can get better than that, barring full featured language support (`&pin` references or something), so I see no reason not to start experimenting with featuring this in the stdlib already 🙂
- cc `@rust-lang/wg-async-foundations` \[EDIT: this doesn't seem to have pinged anybody 😩, thanks `@yoshuawuyts` for the real ping\]
r? `@joshtriplett`
___
# Docs preview
https://user-images.githubusercontent.com/9920355/150605731-1f45c2eb-c9b0-4ce3-b17f-2784fb75786e.mp4
___
# Implementation
The implementation ends up being dead simple (so much it's embarrassing):
```rust
pub macro pin($value:expr $(,)?) {
Pin { pointer: &mut { $value } }
}
```
_and voilà_!
- The key for it working lies in [the rules governing the scope of anonymous temporaries](https://doc.rust-lang.org/1.58.1/reference/destructors.html#temporary-lifetime-extension).
<details><summary>Comments and context</summary>
This is `Pin::new_unchecked(&mut { $value })`, so, for starters, let's
review such a hypothetical macro (that any user-code could define):
```rust
macro_rules! pin {( $value:expr ) => (
match &mut { $value } { at_value => unsafe { // Do not wrap `$value` in an `unsafe` block.
$crate::pin::Pin::<&mut _>::new_unchecked(at_value)
}}
)}
```
Safety:
- `type P = &mut _`. There are thus no pathological `Deref{,Mut}` impls that would break `Pin`'s invariants.
- `{ $value }` is braced, making it a _block expression_, thus **moving** the given `$value`, and making it _become an **anonymous** temporary_.
By virtue of being anonynomous, it can no longer be accessed, thus preventing any attemps to `mem::replace` it or `mem::forget` it, _etc._
This gives us a `pin!` definition that is sound, and which works, but only in certain scenarios:
- If the `pin!(value)` expression is _directly_ fed to a function call:
`let poll = pin!(fut).poll(cx);`
- If the `pin!(value)` expression is part of a scrutinee:
```rust
match pin!(fut) { pinned_fut => {
pinned_fut.as_mut().poll(...);
pinned_fut.as_mut().poll(...);
}} // <- `fut` is dropped here.
```
Alas, it doesn't work for the more straight-forward use-case: `let` bindings.
```rust
let pinned_fut = pin!(fut); // <- temporary value is freed at the end of this statement
pinned_fut.poll(...) // error[E0716]: temporary value dropped while borrowed
// note: consider using a `let` binding to create a longer lived value
```
- Issues such as this one are the ones motivating https://github.com/rust-lang/rfcs/pull/66
This makes such a macro incredibly unergonomic in practice, and the reason most macros out there had to take the path of being a statement/binding macro (_e.g._, `pin!(future);`) instead of featuring the more intuitive ergonomics of an expression macro.
Luckily, there is a way to avoid the problem. Indeed, the problem stems from the fact that a temporary is dropped at the end of its enclosing statement when it is part of the parameters given to function call, which has precisely been the case with our `Pin::new_unchecked()`!
For instance,
```rust
let p = Pin::new_unchecked(&mut <temporary>);
```
becomes:
```rust
let p = { let mut anon = <temporary>; &mut anon };
```
However, when using a literal braced struct to construct the value, references to temporaries can then be taken. This makes Rust change the lifespan of such temporaries so that they are, instead, dropped _at the end of the enscoping block_.
For instance,
```rust
let p = Pin { pointer: &mut <temporary> };
```
becomes:
```rust
let mut anon = <temporary>;
let p = Pin { pointer: &mut anon };
```
which is *exactly* what we want.
Finally, we don't hit problems _w.r.t._ the privacy of the `pointer` field, or the unqualified `Pin` name, thanks to `decl_macro`s being _fully_ hygienic (`def_site` hygiene).
</details>
___
# TODO
- [x] Add compile-fail tests with attempts to break the `Pin` invariants thanks to the macro (_e.g._, try to access the private `.pointer` field, or see what happens if such a pin is used outside its enscoping scope (borrow error));
- [ ] Follow-up stuff:
- [ ] Try to experiment with adding `pin!` to the prelude: this may require to be handled with some extra care, as it may lead to issues reminiscent of those of `assert_matches!`: https://github.com/rust-lang/rust/issues/82913
- [x] Create the tracking issue.
This thus still makes it technically possible to enable the feature, and thus
to trigger UB without `unsafe`, but this is fine since incomplete features are
known to be potentially unsound (labelled "may not be safe").
This follows from the discussion at https://github.com/rust-lang/rust/pull/93176#discussion_r799413561
Couple of driver cleanups
* Remove the `RustcDefaultCalls` struct, which hasn't been necessary since the introduction of `rustc_interface`.
* Move the `setup_callbacks` call around for a tiny code deduplication.
* Remove the `SPAN_DEBUG` global as it isn't actually necessary.
The only difference between the default and rustc_interface set version
is that the default accesses the source map from SESSION_GLOBALS while
the rustc_interface version accesses the source map from the global
TyCtxt. SESSION_GLOBALS is always set while running the compiler while
the global TyCtxt is not always set. If the global TyCtxt is set, it's
source map is identical to the one in SESSION_GLOBALS
Make `span_extend_to_prev_str()` more robust
Fixes#91560. The logic in `span_extend_to_prev_str()` is currently quite brittle and fails if there is extra whitespace or something else in between, and it also should return an `Option` but doesn't currently.
Fix invalid special casing of the unreachable! macro
This pull-request fix an invalid special casing of the `unreachable!` macro in the same way the `panic!` macro was solved, by adding two new internal only macros `unreachable_2015` and `unreachable_2021` edition dependent and turn `unreachable!` into a built-in macro that do dispatching. This logic is stolen from the `panic!` macro.
~~This pull-request also adds an internal feature `format_args_capture_non_literal` that allows capturing arguments from formatted string that expanded from macros. The original RFC #2795 mentioned this as a future possibility. This feature is [required](https://github.com/rust-lang/rust/issues/92137#issuecomment-1018630522) because of concatenation that needs to be done inside the macro:~~
```rust
$crate::concat!("internal error: entered unreachable code: ", $fmt)
```
**In summary** the new behavior for the `unreachable!` macro with this pr is:
Edition 2021:
```rust
let x = 5;
unreachable!("x is {x}");
```
```
internal error: entered unreachable code: x is 5
```
Edition <= 2018:
```rust
let x = 5;
unreachable!("x is {x}");
```
```
internal error: entered unreachable code: x is {x}
```
Also note that the change in this PR are **insta-stable** and **breaking changes** but this a considered as being a [bug](https://github.com/rust-lang/rust/issues/92137#issuecomment-998441613).
If someone could start a perf run and then a crater run this would be appreciated.
Fixes https://github.com/rust-lang/rust/issues/92137
Add `intrinsics::const_deallocate`
Tracking issue: #79597
Related: #91884
This allows deallocation of a memory allocated by `intrinsics::const_allocate`. At the moment, this can be only used to reduce memory usage, but in the future this may be useful to detect memory leaks (If an allocated memory remains after evaluation, raise an error...?).
Introduce a limit to Levenshtein distance computation
Incorporate distance limit from `find_best_match_for_name` directly into
Levenshtein distance computation.
Use the string size difference as a lower bound on the distance and exit
early when it exceeds the specified limit.
After finding a candidate within a limit, lower the limit further to
restrict the search space.
Incorporate distance limit from `find_best_match_for_name` directly into
Levenshtein distance computation.
Use the string size difference as a lower bound on the distance and exit
early when it exceeds the specified limit.
After finding a candidate within a limit, lower the limit further to
restrict the search space.
Make `Decodable` and `Decoder` infallible.
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
currently panics on failure (e.g. if the input is too short, or on a
bad `Result` discriminant), and in some places it returns an error
(e.g. on a bad `Option` discriminant). The number of places where
either happens is surprisingly small, just because the binary
representation has very little redundancy and a lot of input reading
can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
`.rlink` file production, and there's a `FIXME` comment suggesting it
should change to a binary format, and (b) in a few tests in
non-fundamental ways. Indeed #85993 is open to remove it entirely.
And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.
Much of this PR is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
optimization for small counts that the impl for `Result<T, E>` has,
because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
`collect`, which is nice; the one for `Vec` uses unsafe code, because
that gave better perf on some benchmarks.
r? `@bjorn3`
Print a helpful message if unwinding aborts when it reaches a nounwind function
This is implemented by routing `TerminatorKind::Abort` back through the panic handler, but with a special flag in the `PanicInfo` which indicates that the panic handler should *not* attempt to unwind the stack and should instead abort immediately.
This is useful for the planned change in https://github.com/rust-lang/lang-team/issues/97 which would make `Drop` impls `nounwind` by default.
### Code
```rust
#![feature(c_unwind)]
fn panic() {
panic!()
}
extern "C" fn nounwind() {
panic();
}
fn main() {
nounwind();
}
```
### Before
```
$ ./test
thread 'main' panicked at 'explicit panic', test.rs:4:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Illegal instruction (core dumped)
```
### After
```
$ ./test
thread 'main' panicked at 'explicit panic', test.rs:4:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread 'main' panicked at 'panic in a function that cannot unwind', test.rs:7:1
stack backtrace:
0: 0x556f8f86ec9b - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hdccefe11a6ac4396
1: 0x556f8f88ac6c - core::fmt::write::he152b28c41466ebb
2: 0x556f8f85d6e2 - std::io::Write::write_fmt::h0c261480ab86f3d3
3: 0x556f8f8654fa - std::panicking::default_hook::{{closure}}::h5d7346f3ff7f6c1b
4: 0x556f8f86512b - std::panicking::default_hook::hd85803a1376cac7f
5: 0x556f8f865a91 - std::panicking::rust_panic_with_hook::h4dc1c5a3036257ac
6: 0x556f8f86f079 - std::panicking::begin_panic_handler::{{closure}}::hdda1d83c7a9d34d2
7: 0x556f8f86edc4 - std::sys_common::backtrace::__rust_end_short_backtrace::h5b70ed0cce71e95f
8: 0x556f8f865592 - rust_begin_unwind
9: 0x556f8f85a764 - core::panicking::panic_no_unwind::h2606ab3d78c87899
10: 0x556f8f85b910 - test::nounwind::hade6c7ee65050347
11: 0x556f8f85b936 - test::main::hdc6e02cb36343525
12: 0x556f8f85b7e3 - core::ops::function::FnOnce::call_once::h4d02663acfc7597f
13: 0x556f8f85b739 - std::sys_common::backtrace::__rust_begin_short_backtrace::h071d40135adb0101
14: 0x556f8f85c149 - std::rt::lang_start::{{closure}}::h70dbfbf38b685e93
15: 0x556f8f85c791 - std::rt::lang_start_internal::h798f1c0268d525aa
16: 0x556f8f85c131 - std::rt::lang_start::h476a7ee0a0bb663f
17: 0x556f8f85b963 - main
18: 0x7f64c0822b25 - __libc_start_main
19: 0x556f8f85ae8e - _start
20: 0x0 - <unknown>
thread panicked while panicking. aborting.
Aborted (core dumped)
```
`Decoder` has two impls:
- opaque: this impl is already partly infallible, i.e. in some places it
currently panics on failure (e.g. if the input is too short, or on a
bad `Result` discriminant), and in some places it returns an error
(e.g. on a bad `Option` discriminant). The number of places where
either happens is surprisingly small, just because the binary
representation has very little redundancy and a lot of input reading
can occur even on malformed data.
- json: this impl is fully fallible, but it's only used (a) for the
`.rlink` file production, and there's a `FIXME` comment suggesting it
should change to a binary format, and (b) in a few tests in
non-fundamental ways. Indeed #85993 is open to remove it entirely.
And the top-level places in the compiler that call into decoding just
abort on error anyway. So the fallibility is providing little value, and
getting rid of it leads to some non-trivial performance improvements.
Much of this commit is pretty boring and mechanical. Some notes about
a few interesting parts:
- The commit removes `Decoder::{Error,error}`.
- `InternIteratorElement::intern_with`: the impl for `T` now has the same
optimization for small counts that the impl for `Result<T, E>` has,
because it's now much hotter.
- Decodable impls for SmallVec, LinkedList, VecDeque now all use
`collect`, which is nice; the one for `Vec` uses unsafe code, because
that gave better perf on some benchmarks.
Improve SIMD casts
* Allows `simd_cast` intrinsic to take `usize` and `isize`
* Adds `simd_as` intrinsic, which is the same as `simd_cast` except for saturating float-to-int conversions (matching the behavior of `as`).
cc `@workingjubilee`
Avoid unnecessary monomorphization of inline asm related functions
This should reduce build time for codegen backends by avoiding duplicated monomorphization of certain inline asm related functions for each passed in closure type.
