LLVM already supports emitting compressed debuginfo. In debuginfo=full
builds, the debug section is often a large amount of data, and it
typically compresses very well (3x is not unreasonable.) We add a new
knob to allow debuginfo to be compressed when the matching LLVM
functionality is present. Like clang, if a known-but-disabled
compression mechanism is requested, we disable compression and emit
uncompressed debuginfo sections.
The API is different enough on older LLVMs we just pretend the support
is missing on LLVM older than 16.
Remove -Z diagnostic-width
This removes the `-Z diagnostic-width` option since it is ignored and does nothing. `-Z diagnostic-width` was stabilized as `--diagnostic-width` in #95635. It is not entirely clear why the `-Z` flag was kept, but in part its final use was removed in #102216, but the `-Z` flag itself was not removed.
new unstable option: -Zwrite-long-types-to-disk
This option guards the logic of writing long type names in files and instead using short forms in error messages in rustc_middle/ty/error behind a flag. The main motivation for this change is to disable this behaviour when running ui tests.
This logic can be triggered by running tests in a directory that has a long enough path, e.g. /my/very-long-path/where/rust-codebase/exists/
This means ui tests can fail depending on how long the path to their file is.
Some ui tests actually rely on this behaviour for their assertions, so for those we enable the flag manually.
This option guards the logic of writing long type names in files and
instead using short forms in error messages in rustc_middle/ty/error
behind a flag. The main motivation for this change is to disable this
behaviour when running ui tests.
This logic can be triggered by running tests in a directory that has a
long enough path, e.g. /my/very-long-path/where/rust-codebase/exists/
This means ui tests can fail depending on how long the path to their
file is.
Some ui tests actually rely on this behaviour for their assertions,
so for those we enable the flag manually.
Prototype: Add unstable `-Z reference-niches` option
MCP: rust-lang/compiler-team#641
Relevant RFC: rust-lang/rfcs#3204
This prototype adds a new `-Z reference-niches` option, controlling the range of valid bit-patterns for reference types (`&T` and `&mut T`), thereby enabling new enum niching opportunities. Like `-Z randomize-layout`, this setting is crate-local; as such, references to built-in types (primitives, tuples, ...) are not affected.
The possible settings are (here, `MAX` denotes the all-1 bit-pattern):
| `-Z reference-niches=` | Valid range |
|:---:|:---:|
| `null` (the default) | `1..=MAX` |
| `size` | `1..=(MAX- size)` |
| `align` | `align..=MAX.align_down_to(align)` |
| `size,align` | `align..=(MAX-size).align_down_to(align)` |
------
This is very WIP, and I'm not sure the approach I've taken here is the best one, but stage 1 tests pass locally; I believe this is in a good enough state to unleash this upon unsuspecting 3rd-party code, and see what breaks.
LLVM has a neat [statistics] feature that tracks how often optimizations kick
in. It's very handy for optimization work. Since we expose the LLVM pass
timings, I thought it made sense to expose the LLVM statistics too.
[statistics]: https://llvm.org/docs/ProgrammersManual.html#the-statistic-class-stats-option
Add simple markdown formatting to `rustc --explain` output
This is a second attempt at #104540, which is #63128 without dependencies.
This PR adds basic markdown formatting to `rustc --explain` output when available. Currently, the output just displays raw markdown: this works of course, but it really doesn't look very elegant. (output is `rustc --explain E0038`)
<img width="583" alt="image" src="https://github.com/rust-lang/rust/assets/13724985/ea418117-47af-455b-83c0-6fc59276efee">
After this patch, sample output from the same file:
<img width="693" alt="image" src="https://github.com/rust-lang/rust/assets/13724985/12f7bf9b-a3fe-4104-b74b-c3e5227f3de9">
This also obeys the `--color always/auto/never` command option. Behavior:
- If pager is available and supports color, print with formatting to the pager
- If pager is not available or fails print with formatting to stdout - otherwise without formatting
- Follow `--color always/never` if suppied
- If everything fails, just print plain text to stdout
r? `@oli-obk`
cc `@estebank`
(since the two of you were involved in the previous discussion)
add flag for enabling global cache usage for proof trees and printing proof trees on error
This adds a few new things:
- `-Zdump-solver-proof-tree=always/never/on-error`
- `always`/`never` were previosuly specifiable by whether the flag exists or not, th new flag is `on_error` which reruns obligations of fulfillment and selection errors with proof tree generation enabled and prints them out
- `-Zdump-solver-proof-tree-uses-cache`
- allows forcing global cache to be used or unused for all generated proof trees, global cache is enabled by default for `always` so that it accurately represents what happend. This flag currently would affect misc uses of `GenerateProofTree::Yes` which will be added in the future for things like diagnostics logic and rustdoc's auto_trait file. We can fix this when we start using proof tree generation for those use cases if it's desirable.
I also changed the output to go straight to stdout instead of going through `debug!` so that `-Zdump-solver-proof-tree` can be adequately used on `nightly` not just a locally built toolchain.
The idea for `on-error` is that it should hopefully make it easier to quickly figure out "why doesnt this code compile"- you just pass in `-Zdump-solver-proof-tree=on-error` and you'll only get proof trees you care about.
---
r? `@lcnr` `@compiler-errors`
Currently, the output of `rustc --explain foo` displays the raw markdown in a
pager. This is acceptable, but using actual formatting makes it easier to
understand.
This patch consists of three major components:
1. A markdown parser. This is an extremely simple non-backtracking recursive
implementation that requires normalization of the final token stream
2. A utility to write the token stream to an output buffer
3. Configuration within rustc_driver_impl to invoke this combination for
`--explain`. Like the current implementation, it first attempts to print to
a pager with a fallback colorized terminal, and standard print as a last
resort.
If color is disabled, or if the output does not support it, or if printing
with color fails, it will write the raw markdown (which matches current
behavior).
Pagers known to support color are: `less` (with `-r`), `bat` (aka `catbat`),
and `delta`.
The markdown parser does not support the entire markdown specification, but
should support the following with reasonable accuracy:
- Headings, including formatting
- Comments
- Code, inline and fenced block (no indented block)
- Strong, emphasis, and strikethrough formatted text
- Links, anchor, inline, and reference-style
- Horizontal rules
- Unordered and ordered list items, including formatting
This parser and writer should be reusable by other systems if ever needed.
Add `-Zremark-dir` unstable flag to write LLVM optimization remarks to YAML
This PR adds an option for `rustc` to emit LLVM optimization remarks to a set of YAML files, which can then be digested by existing tools, like https://github.com/OfekShilon/optview2. When `-Cremark-dir` is passed, and remarks are enabled (`-Cremark=all`), the remarks will be now written to the specified directory, **instead** of being printed to standard error output. The files are named based on the CGU from which they are being generated.
Currently, the remarks are written using the LLVM streaming machinery, directly in the diagnostics handler. It seemed easier than going back to Rust and then form there back to C++ to use the streamer from the diagnostics handler. But there are many ways to implement this, of course, so I'm open to suggestions :)
I included some comments with questions into the code. Also, I'm not sure how to test this.
r? `@tmiasko`
Per the discussion in #106380 plt=no isn't a great default, and
rust-lang/compiler-team#581 decided that the default should be PLT=yes
for everything except x86_64. Not everyone agrees about the x86_64 part
of this change, but this at least is an improvement in the state of
things without changing the x86_64 situation, so I've attempted making
this change in the name of not letting the perfect be the enemy of the
good.
This option was introduced three years ago, but it's never been
meaningfully used, and `default` is the only acceptable value.
Also, I think the `Partition` trait presents an interface that is too
closely tied to the existing strategy and would probably be wrong for
other strategies. (My rule of thumb is to not make something generic
until there are at least two instances of it, to avoid this kind of
problem.)
Also, I don't think providing multiple partitioning strategies to the
user is a good idea, because the compiler already has enough obscure
knobs.
This commit removes the option, along with the `Partition` trait, and
the `Partitioner` and `DefaultPartitioning` types. I left the existing
code in `compiler/rustc_monomorphize/src/partitioning/default.rs`,
though I could be persuaded that moving it into
`compiler/rustc_monomorphize/src/partitioning/mod.rs` is better.
Adds support for LLVM [SafeStack] which provides backward edge control
flow protection by separating the stack into two parts: data which is
only accessed in provable safe ways is allocated on the normal stack
(the "safe stack") and all other data is placed in a separate allocation
(the "unsafe stack").
SafeStack support is enabled by passing `-Zsanitizer=safestack`.
[SafeStack]: https://clang.llvm.org/docs/SafeStack.html
Error message all end up passing into a function as an `impl
Into<{D,Subd}iagnosticMessage>`. If an error message is creatd as
`&format("...")` that means we allocate a string (in the `format!`
call), then take a reference, and then clone (allocating again) the
reference to produce the `{D,Subd}iagnosticMessage`, which is silly.
This commit removes the leading `&` from a lot of these cases. This
means the original `String` is moved into the
`{D,Subd}iagnosticMessage`, avoiding the double allocations. This
requires changing some function argument types from `&str` to `String`
(when all arguments are `String`) or `impl
Into<{D,Subd}iagnosticMessage>` (when some arguments are `String` and
some are `&str`).
Support linking to rust dylib with --crate-type staticlib
This allows for example dynamically linking libstd, while statically linking the user crate into an executable or C dynamic library. For this two unstable flags (`-Z staticlib-allow-rdylib-deps` and `-Z staticlib-prefer-dynamic`) are introduced. Without the former you get an error. The latter is the equivalent to `-C prefer-dynamic` for the staticlib crate type to indicate that dynamically linking is preferred when both options are available, like for libstd. Care must be taken to ensure that no crate ends up being merged into two distinct staticlibs that are linked together. Doing so will cause a linker error at best and undefined behavior at worst. In addition two distinct staticlibs compiled by different rustc may not be combined under any circumstances due to some rustc private symbols not being mangled.
To successfully link a staticlib, `--print native-static-libs` can be used while compiling to ask rustc for the linker flags necessary when linking the staticlib. This is an existing flag which previously only listed native libraries. It has been extended to list rust dylibs too. Trying to locate libstd yourself to link against it is not supported and may break if for example the libstd of multiple rustc versions are put in the same directory.
For an example on how to use this see the `src/test/run-make-fulldeps/staticlib-dylib-linkage/` test.
Stabilize raw-dylib, link_ordinal, import_name_type and -Cdlltool
This stabilizes the `raw-dylib` feature (#58713) for all architectures (i.e., `x86` as it is already stable for all other architectures).
Changes:
* Permit the use of the `raw-dylib` link kind for x86, the `link_ordinal` attribute and the `import_name_type` key for the `link` attribute.
* Mark the `raw_dylib` feature as stable.
* Stabilized the `-Zdlltool` argument as `-Cdlltool`.
* Note the path to `dlltool` if invoking it failed (we don't need to do this if `dlltool` returns an error since it prints its path in the error message).
* Adds tests for `-Cdlltool`.
* Adds tests for being unable to find the dlltool executable, and dlltool failing.
* Fixes a bug where we were checking the exit code of dlltool to see if it failed, but dlltool always returns 0 (indicating success), so instead we need to check if anything was written to `stderr`.
NOTE: As previously noted (https://github.com/rust-lang/rust/pull/104218#issuecomment-1315895618) using dlltool within rustc is temporary, but this is not the first time that Rust has added a temporary tool use and argument: https://github.com/rust-lang/rust/pull/104218#issuecomment-1318720482
Big thanks to ``````@tbu-`````` for the first version of this PR (#104218)
Add cross-language LLVM CFI support to the Rust compiler
This PR adds cross-language LLVM Control Flow Integrity (CFI) support to the Rust compiler by adding the `-Zsanitizer-cfi-normalize-integers` option to be used with Clang `-fsanitize-cfi-icall-normalize-integers` for normalizing integer types (see https://reviews.llvm.org/D139395).
It provides forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space). For more information about LLVM CFI and cross-language LLVM CFI support for the Rust compiler, see design document in the tracking issue #89653.
Cross-language LLVM CFI can be enabled with -Zsanitizer=cfi and -Zsanitizer-cfi-normalize-integers, and requires proper (i.e., non-rustc) LTO (i.e., -Clinker-plugin-lto).
Thank you again, ``@bjorn3,`` ``@nikic,`` ``@samitolvanen,`` and the Rust community for all the help!
This commit adds cross-language LLVM Control Flow Integrity (CFI)
support to the Rust compiler by adding the
`-Zsanitizer-cfi-normalize-integers` option to be used with Clang
`-fsanitize-cfi-icall-normalize-integers` for normalizing integer types
(see https://reviews.llvm.org/D139395).
It provides forward-edge control flow protection for C or C++ and Rust
-compiled code "mixed binaries" (i.e., for when C or C++ and Rust
-compiled code share the same virtual address space). For more
information about LLVM CFI and cross-language LLVM CFI support for the
Rust compiler, see design document in the tracking issue #89653.
Cross-language LLVM CFI can be enabled with -Zsanitizer=cfi and
-Zsanitizer-cfi-normalize-integers, and requires proper (i.e.,
non-rustc) LTO (i.e., -Clinker-plugin-lto).
Report allocation errors as panics
OOM is now reported as a panic but with a custom payload type (`AllocErrorPanicPayload`) which holds the layout that was passed to `handle_alloc_error`.
This should be review one commit at a time:
- The first commit adds `AllocErrorPanicPayload` and changes allocation errors to always be reported as panics.
- The second commit removes `#[alloc_error_handler]` and the `alloc_error_hook` API.
ACP: https://github.com/rust-lang/libs-team/issues/192Closes#51540Closes#51245
