Avoid blessing cargo deps's source code in ui tests
Before this PR, the source code of dependencies was included in UI test error messages whenever possible. Unfortunately, "whenever possible" means in some cases the source code wouldn't be injected, resulting in a test failure.
One such case is when `$CARGO_HOME` is remapped to something that is not present on disk [^1]. As the remapped path doesn't exist on disk, the source code wouldn't be showed in `tests/ui/issues/issue-21763.rs`:
```diff
= note: required for `hashbrown::raw::RawTable<(Rc<()>, Rc<()>)>` to implement `Send`
note: required because it appears within the type `HashMap<Rc<()>, Rc<()>, RandomState>`
--> $HASHBROWN_SRC_LOCATION
- |
-LL | pub struct HashMap<K, V, S = DefaultHashBuilder, A: Allocator + Clone = Global> {
- | ^^^^^^^
note: required because it appears within the type `HashMap<Rc<()>, Rc<()>>`
--> $SRC_DIR/std/src/collections/hash/map.rs:LL:COL
note: required by a bound in `foo`
```
This PR fixes the problem by always hiding dependencies source code in the error messages generated during UI tests. This is implemented with a new internal flag, `-Z ignore-directory-in-diagnostics-source-blocks=$path`, which compiletest passes during UI tests. Once this is merged, remapping the Cargo home will be supported.
This PR is best reviewed commit-by-commit.
[^1]: After being puzzled for a bit, I discovered why this never impacted `rust-lang/rust`: we don't remap `$CARGO_HOME` 😅. Instead, we set `$CARGO_HOME` to `/cargo` in CI, which sort-of-but-not-really achieves the same effect.
Make `.rmeta` file in `dep-info` have correct name (`lib` prefix)
Since `filename_for_metadata()` and
`OutputFilenames::path(OutputType::Metadata)` had different logic for the name of the metadata file, the `.d` file contained a file name different from the actual name used. Share the logic to fix the out-of-sync name.
Without this fix, the `.d` file contained
dash-separated_something-extra.rmeta: dash-separated.rs
instead of
libdash_separated_something-extra.rmeta: dash-separated.rs
which is the name of the file that is actually written by the compiler.
Worth noting: It took me several iterations to get all tests to pass, so I am relatively confident that this PR does not break anything.
Closes#68839
Enable incremental-relative-spans by default.
This was enabled on nightly in https://github.com/rust-lang/rust/pull/84762.
It has been a while, without obvious bugs. It's time to enable it by default for incremental runs.
Use `FreezeLock` for `CStore`
This uses `FreezeLock` to protect the `CStore`. `FreezeReadGuard` and `FreezeWriteGuard` are changed to support a `map` operation.
r? `@oli-obk`
Abort if check nightly options failed on stable
Fixes#115680
Also, if there are multiple unstable options passing on stable compiler, printing multiple same `note` and `help` seems noisy.
LLVM already supports emitting compressed debuginfo. In debuginfo=full
builds, the debug section is often a large amount of data, and it
typically compresses very well (3x is not unreasonable.) We add a new
knob to allow debuginfo to be compressed when the matching LLVM
functionality is present. Like clang, if a known-but-disabled
compression mechanism is requested, we disable compression and emit
uncompressed debuginfo sections.
The API is different enough on older LLVMs we just pretend the support
is missing on LLVM older than 16.
Add CL and CMD into to pdb debug info
Partial fix for https://github.com/rust-lang/rust/issues/96475
The Arg0 and CommandLineArgs of the MCTargetOptions cpp class are not set within bb548f9645/compiler/rustc_llvm/llvm-wrapper/PassWrapper.cpp (L378)
This causes LLVM to not neither output any compiler path (cl) nor the arguments that were used when invoking it (cmd) in the PDB file.
This fix adds the missing information to the target machine so LLVM can use it.
Stabilize `PATH` option for `--print KIND=PATH`
This PR propose stabilizing the `PATH` option for `--print KIND=PATH`. This option was previously added in https://github.com/rust-lang/rust/pull/113780 (as insta-stable before being un-stablized in https://github.com/rust-lang/rust/pull/114139).
Description of the `PATH` option:
> A filepath may optionally be specified for each requested information kind, in the format `--print KIND=PATH`, just like for `--emit`. When a path is specified, information will be written there instead of to stdout.
------
Description of the original PR [\[link\]](https://github.com/rust-lang/rust/pull/113780#issue-1807080607):
> **Support --print KIND=PATH command line syntax**
>
> As is already done for `--emit KIND=PATH` and `-L KIND=PATH`.
>
> In the discussion of https://github.com/rust-lang/rust/pull/110785, it was pointed out that `--print KIND=PATH` is nicer than trying to apply the single global `-o path` to `--print`'s output, because in general there can be multiple print requests within a single rustc invocation, and anyway `-o` would already be used for a different meaning in the case of `link-args` and `native-static-libs`.
>
> I am interested in using `--print cfg=PATH` in Buck2. Currently Buck2 works around the lack of support for `--print KIND=PATH` by [indirecting through a Python wrapper script](d43cf3a51a/prelude/rust/tools/get_rustc_cfg.py) to redirect rustc's stdout into the location dictated by the build system.
>
> From skimming Cargo's usages of `--print`, it definitely seems like it would benefit from `--print KIND=PATH` too. Currently it is working around the lack of this by inserting `--crate-name=___ --print=crate-name` so that it can look for a line containing `___` as a delimiter between the 2 other `--print` informations it actually cares about. This is commented as a "HACK" and "abuse". 31eda6f7c3/src/cargo/core/compiler/build_context/target_info.rs (L242)
-----
cc `@dtolnay`
r? `@jackh726`
Description of the `PATH` option:
> A filepath may optionally be specified for each requested information
> kind, in the format `--print KIND=PATH`, just like for `--emit`. When
> a path is specified, information will be written there instead of to
> stdout.
Since `filename_for_metadata()` and
`OutputFilenames::path(OutputType::Metadata)` had different logic for
the name of the metadata file, the `.d` file contained a file name
different from the actual name used. Share the logic to fix the
out-of-sync name.
Closes 68839.
Add an (perma-)unstable option to disable vtable vptr
This flag is intended for evaluation of trait upcasting space cost for embedded use cases.
Compared to the approach in #112355, this option provides a way to evaluate end-to-end cost of trait upcasting. Rationale: https://github.com/rust-lang/rust/issues/112355#issuecomment-1658207769
## How this flag should be used (after merge)
Build your project with and without `-Zno-trait-vptr` flag. If you are using cargo, set `RUSTFLAGS="-Zno-trait-vptr"` in the environment variable. You probably also want to use `-Zbuild-std` or the binary built may be broken. Save both binaries somewhere.
### Evaluate the space cost
The option has a direct and indirect impact on vtable space usage. Directly, it gets rid of the trait vptr entry needed to store a pointer to a vtable of a supertrait. (IMO) this is a small saving usually. The larger saving usually comes with the indirect saving by eliminating the vtable of the supertrait (and its parent).
Both impacts only affects vtables (notably the number of functions monomorphized should , however where vtable reside can depend on your relocation model. If the relocation model is static, then vtable is rodata (usually stored in Flash/ROM together with text in embedded scenario). If the binary is relocatable, however, the vtable will live in `.data` (more specifically, `.data.rel.ro`), and this will need to reside in RAM (which may be a more scarce resource in some cases), together with dynamic relocation info living in readonly segment.
For evaluation, you should run `size` on both binaries, with and without the flag. `size` would output three columns, `text`, `data`, `bss` and the sum `dec` (and it's hex version). As explained above, both `text` and `data` may change. `bss` shouldn't usually change. It'll be useful to see:
* Percentage change in text + data (indicating required flash/ROM size)
* Percentage change in data + bss (indicating required RAM size)
CFI: Fix error compiling core with LLVM CFI enabled
Fix#90546 by filtering out global value function pointer types from the type tests, and adding the LowerTypeTests pass to the rustc LTO optimization pipelines.
Fix#90546 by filtering out global value function pointer types from the
type tests, and adding the LowerTypeTests pass to the rustc LTO
optimization pipelines.
Lots of tiny incremental simplifications of `EmitterWriter` internals
ignore the first commit, it's https://github.com/rust-lang/rust/pull/114088 squashed and rebased, but it's needed to use to use `derive_setters`, as they need a newer `syn` version.
Then this PR starts out with removing many arguments that are almost always defaulted to `None` or `false` and replace them with builder methods that can set these fields in the few cases that want to set them.
After that it's one commit after the other that removes or merges things until everything becomes some very simple trait objects
It lints against features that are inteded to be internal to the
compiler and standard library. Implements MCP #596.
We allow `internal_features` in the standard library and compiler as those
use many features and this _is_ the standard library from the "internal to the compiler and
standard library" after all.
Marking some features as internal wasn't exactly the most scientific approach, I just marked some
mostly obvious features. While there is a categorization in the macro,
it's not very well upheld (should probably be fixed in another PR).
We always pass `-Ainternal_features` in the testsuite
About 400 UI tests and several other tests use internal features.
Instead of throwing the attribute on each one, just always allow them.
There's nothing wrong with testing internal features^^
Remove -Z diagnostic-width
This removes the `-Z diagnostic-width` option since it is ignored and does nothing. `-Z diagnostic-width` was stabilized as `--diagnostic-width` in #95635. It is not entirely clear why the `-Z` flag was kept, but in part its final use was removed in #102216, but the `-Z` flag itself was not removed.
Split some functions with many arguments into builder pattern functions
r? `@estebank`
This doesn't resolve all of the ones in rustc, mostly because I need to do other cleanups in order to be able to use some builder derives from crates.io
Works around https://github.com/rust-lang/rust/issues/90672 by making `x test rustfmt --bless` format itself instead of testing that it is formatted
new unstable option: -Zwrite-long-types-to-disk
This option guards the logic of writing long type names in files and instead using short forms in error messages in rustc_middle/ty/error behind a flag. The main motivation for this change is to disable this behaviour when running ui tests.
This logic can be triggered by running tests in a directory that has a long enough path, e.g. /my/very-long-path/where/rust-codebase/exists/
This means ui tests can fail depending on how long the path to their file is.
Some ui tests actually rely on this behaviour for their assertions, so for those we enable the flag manually.
This option guards the logic of writing long type names in files and
instead using short forms in error messages in rustc_middle/ty/error
behind a flag. The main motivation for this change is to disable this
behaviour when running ui tests.
This logic can be triggered by running tests in a directory that has a
long enough path, e.g. /my/very-long-path/where/rust-codebase/exists/
This means ui tests can fail depending on how long the path to their
file is.
Some ui tests actually rely on this behaviour for their assertions,
so for those we enable the flag manually.
Prototype: Add unstable `-Z reference-niches` option
MCP: rust-lang/compiler-team#641
Relevant RFC: rust-lang/rfcs#3204
This prototype adds a new `-Z reference-niches` option, controlling the range of valid bit-patterns for reference types (`&T` and `&mut T`), thereby enabling new enum niching opportunities. Like `-Z randomize-layout`, this setting is crate-local; as such, references to built-in types (primitives, tuples, ...) are not affected.
The possible settings are (here, `MAX` denotes the all-1 bit-pattern):
| `-Z reference-niches=` | Valid range |
|:---:|:---:|
| `null` (the default) | `1..=MAX` |
| `size` | `1..=(MAX- size)` |
| `align` | `align..=MAX.align_down_to(align)` |
| `size,align` | `align..=(MAX-size).align_down_to(align)` |
------
This is very WIP, and I'm not sure the approach I've taken here is the best one, but stage 1 tests pass locally; I believe this is in a good enough state to unleash this upon unsuspecting 3rd-party code, and see what breaks.
Use SHA256 source file checksums by default when targeting MSVC
Currently, when targeting Windows (more specifically, the MSVC toolchain), Rust will use SHA1 source file checksums by default. SHA1 has been superseded by SHA256, and Microsoft recommends migrating to SHA256.
As of Visual Studio 2022, MSVC defaults to SHA256. This change aligns Rust and MSVC.
LLVM can already use SHA256 checksums, so this does not require any change to LLVM.
MSVC docs on source file checksums: https://learn.microsoft.com/en-us/cpp/build/reference/zh?view=msvc-170
Support `--print KIND=PATH` command line syntax
As is already done for `--emit KIND=PATH` and `-L KIND=PATH`.
In the discussion of #110785, it was pointed out that `--print KIND=PATH` is nicer than trying to apply the single global `-o` path to `--print`'s output, because in general there can be multiple print requests within a single rustc invocation, and anyway `-o` would already be used for a different meaning in the case of `link-args` and `native-static-libs`.
I am interested in using `--print cfg=PATH` in Buck2. Currently Buck2 works around the lack of support for `--print KIND=PATH` by [indirecting through a Python wrapper script](d43cf3a51a/prelude/rust/tools/get_rustc_cfg.py) to redirect rustc's stdout into the location dictated by the build system.
From skimming Cargo's usages of `--print`, it definitely seems like it would benefit from `--print KIND=PATH` too. Currently it is working around the lack of this by inserting `--crate-name=___ --print=crate-name` so that it can look for a line containing `___` as a delimiter between the 2 other `--print` informations it actually cares about. This is commented as a "HACK" and "abuse". 31eda6f7c3/src/cargo/core/compiler/build_context/target_info.rs (L242) (FYI `@weihanglo` as you dealt with this recently in https://github.com/rust-lang/cargo/pull/11633.)
Mentioning reviewers active in #110785: `@fee1-dead` `@jyn514` `@bjorn3`
Resurrect: rustc_llvm: Add a -Z `print-codegen-stats` option to expose LLVM statistics.
This resurrects PR https://github.com/rust-lang/rust/pull/104000, which has sat idle for a while. And I want to see the effect of stack-move optimizations on LLVM (like https://reviews.llvm.org/D153453) :).
I have applied the changes requested by `@oli-obk` and `@nagisa` https://github.com/rust-lang/rust/pull/104000#discussion_r1014625377 and https://github.com/rust-lang/rust/pull/104000#discussion_r1014642482 in the latest commits.
r? `@oli-obk`
-----
LLVM has a neat [statistics](https://llvm.org/docs/ProgrammersManual.html#the-statistic-class-stats-option) feature that tracks how often optimizations kick in. It's very handy for optimization work. Since we expose the LLVM pass timings, I thought it made sense to expose the LLVM statistics too.
-----
(Edit: fix broken link
(Edit2: fix segmentation fault and use malloc
If `rustc` is built with
```toml
[llvm]
assertions = true
```
Then you can see like
```
rustc +stage1 -Z print-codegen-stats -C opt-level=3 tmp.rs
===-------------------------------------------------------------------------===
... Statistics Collected ...
===-------------------------------------------------------------------------===
3 aa - Number of MayAlias results
193 aa - Number of MustAlias results
531 aa - Number of NoAlias results
...
```
And the current default build emits only
```
$ rustc +stage1 -Z print-codegen-stats -C opt-level=3 tmp.rs
===-------------------------------------------------------------------------===
... Statistics Collected ...
===-------------------------------------------------------------------------===
$
```
This might be better to emit the message to tell assertion flag necessity, but now I can't find how to do that...
On nightly, dump ICE backtraces to disk
Implement rust-lang/compiler-team#578.
When an ICE is encountered on nightly releases, the new rustc panic handler will also write the contents of the backtrace to disk. If any `delay_span_bug`s are encountered, their backtrace is also added to the file. The platform and rustc version will also be collected.
<img width="1032" alt="Screenshot 2023-03-03 at 2 13 25 PM" src="https://user-images.githubusercontent.com/1606434/222842420-8e039740-4042-4563-b31d-599677171acf.png">
The current behavior will *always* write to disk on nightly builds, regardless of whether the backtrace is printed to the terminal, unless the environment variable `RUSTC_ICE_DISK_DUMP` is set to `0`. This is a compromise and can be changed.
Make it clearer that edition functions are `>=`, not `==`
r? `@Nilstrieb`
We could also perhaps derive `Ord` on `Edition` and use comparison operators.
Implement rust-lang/compiler-team#578.
When an ICE is encountered on nightly releases, the new rustc panic
handler will also write the contents of the backtrace to disk. If any
`delay_span_bug`s are encountered, their backtrace is also added to the
file. The platform and rustc version will also be collected.
LLVM has a neat [statistics] feature that tracks how often optimizations kick
in. It's very handy for optimization work. Since we expose the LLVM pass
timings, I thought it made sense to expose the LLVM statistics too.
[statistics]: https://llvm.org/docs/ProgrammersManual.html#the-statistic-class-stats-option
Replace RPITIT current impl with new strategy that lowers as a GAT
This PR replaces the current implementation of RPITITs with the new implementation that we had under -Zlower-impl-trait-in-trait-to-assoc-ty flag that lowers the RPIT as a GAT on the trait and on the impls that implement that trait.
Opening this PR as a draft because this goes after #112682, ~#112981~ and ~#112983~.
As soon as those are merged, I can rebase and we should run perf, crater and test a lot.
r? `@compiler-errors`
Add simple markdown formatting to `rustc --explain` output
This is a second attempt at #104540, which is #63128 without dependencies.
This PR adds basic markdown formatting to `rustc --explain` output when available. Currently, the output just displays raw markdown: this works of course, but it really doesn't look very elegant. (output is `rustc --explain E0038`)
<img width="583" alt="image" src="https://github.com/rust-lang/rust/assets/13724985/ea418117-47af-455b-83c0-6fc59276efee">
After this patch, sample output from the same file:
<img width="693" alt="image" src="https://github.com/rust-lang/rust/assets/13724985/12f7bf9b-a3fe-4104-b74b-c3e5227f3de9">
This also obeys the `--color always/auto/never` command option. Behavior:
- If pager is available and supports color, print with formatting to the pager
- If pager is not available or fails print with formatting to stdout - otherwise without formatting
- Follow `--color always/never` if suppied
- If everything fails, just print plain text to stdout
r? `@oli-obk`
cc `@estebank`
(since the two of you were involved in the previous discussion)
add flag for enabling global cache usage for proof trees and printing proof trees on error
This adds a few new things:
- `-Zdump-solver-proof-tree=always/never/on-error`
- `always`/`never` were previosuly specifiable by whether the flag exists or not, th new flag is `on_error` which reruns obligations of fulfillment and selection errors with proof tree generation enabled and prints them out
- `-Zdump-solver-proof-tree-uses-cache`
- allows forcing global cache to be used or unused for all generated proof trees, global cache is enabled by default for `always` so that it accurately represents what happend. This flag currently would affect misc uses of `GenerateProofTree::Yes` which will be added in the future for things like diagnostics logic and rustdoc's auto_trait file. We can fix this when we start using proof tree generation for those use cases if it's desirable.
I also changed the output to go straight to stdout instead of going through `debug!` so that `-Zdump-solver-proof-tree` can be adequately used on `nightly` not just a locally built toolchain.
The idea for `on-error` is that it should hopefully make it easier to quickly figure out "why doesnt this code compile"- you just pass in `-Zdump-solver-proof-tree=on-error` and you'll only get proof trees you care about.
---
r? `@lcnr` `@compiler-errors`
Currently, the output of `rustc --explain foo` displays the raw markdown in a
pager. This is acceptable, but using actual formatting makes it easier to
understand.
This patch consists of three major components:
1. A markdown parser. This is an extremely simple non-backtracking recursive
implementation that requires normalization of the final token stream
2. A utility to write the token stream to an output buffer
3. Configuration within rustc_driver_impl to invoke this combination for
`--explain`. Like the current implementation, it first attempts to print to
a pager with a fallback colorized terminal, and standard print as a last
resort.
If color is disabled, or if the output does not support it, or if printing
with color fails, it will write the raw markdown (which matches current
behavior).
Pagers known to support color are: `less` (with `-r`), `bat` (aka `catbat`),
and `delta`.
The markdown parser does not support the entire markdown specification, but
should support the following with reasonable accuracy:
- Headings, including formatting
- Comments
- Code, inline and fenced block (no indented block)
- Strong, emphasis, and strikethrough formatted text
- Links, anchor, inline, and reference-style
- Horizontal rules
- Unordered and ordered list items, including formatting
This parser and writer should be reusable by other systems if ever needed.
Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML
This PR adds an option for `rustc` to emit LLVM optimization remarks to a set of YAML files, which can then be digested by existing tools, like https://github.com/OfekShilon/optview2. When `-Cremark-dir` is passed, and remarks are enabled (`-Cremark=all`), the remarks will be now written to the specified directory, **instead** of being printed to standard error output. The files are named based on the CGU from which they are being generated.
Currently, the remarks are written using the LLVM streaming machinery, directly in the diagnostics handler. It seemed easier than going back to Rust and then form there back to C++ to use the streamer from the diagnostics handler. But there are many ways to implement this, of course, so I'm open to suggestions :)
I included some comments with questions into the code. Also, I'm not sure how to test this.
r? `@tmiasko`
linker flavors
- only the stable values for `-Clink-self-contained` can be used on stable until we
have more feedback on the interface
- `-Zunstable-options` is required to use unstable linker flavors
