Upgrade indexmap and thorin-dwp to use hashbrown 0.12
This removes the last dependencies on hashbrown 0.11.
This also upgrades to hashbrown 0.12.3 to fix a double-free (#99372).
Add fine-grained LLVM CFI support to the Rust compiler
This PR improves the LLVM Control Flow Integrity (CFI) support in the Rust compiler by providing forward-edge control flow protection for Rust-compiled code only by aggregating function pointers in groups identified by their return and parameter types.
Forward-edge control flow protection for C or C++ and Rust -compiled code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code share the same virtual address space) will be provided in later work as part of this project by identifying C char and integer type uses at the time types are encoded (see Type metadata in the design document in the tracking issue https://github.com/rust-lang/rust/issues/89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e., -Clto).
Thank you again, `@eddyb,` `@nagisa,` `@pcc,` and `@tmiasko` for all the help!
This commit improves the LLVM Control Flow Integrity (CFI) support in
the Rust compiler by providing forward-edge control flow protection for
Rust-compiled code only by aggregating function pointers in groups
identified by their return and parameter types.
Forward-edge control flow protection for C or C++ and Rust -compiled
code "mixed binaries" (i.e., for when C or C++ and Rust -compiled code
share the same virtual address space) will be provided in later work as
part of this project by identifying C char and integer type uses at the
time types are encoded (see Type metadata in the design document in the
tracking issue #89653).
LLVM CFI can be enabled with -Zsanitizer=cfi and requires LTO (i.e.,
-Clto).
Use constant eval to do strict mem::uninit/zeroed validity checks
I'm not sure about the code organisation here, I just dumped the check in rustc_const_eval at the root. Not hard to move it elsewhere, in any case.
Also, this means cranelift codegen intrinsics lose the strict checks, since they don't seem to depend on rustc_const_eval, and I didn't see a point in keeping around two copies.
I also left comments in the is_zero_valid methods about "uhhh help how do i do this", those apply to both methods equally.
Also rustc_codegen_ssa now depends on rustc_const_eval... is this okay?
Pinging `@RalfJung` since you were the one who mentioned this to me, so I'm assuming you're interested.
Haven't had a chance to run full tests on this since it's really warm, and it's 1AM, I'll check out any failures/comments in the morning :)
Stop keeping metadata in memory before writing it to disk
Fixes#96358
I created this PR according with the instruction given in the issue except for the following points:
- While the issue says "Write metadata into the temporary file in `encode_and_write_metadata` even if `!need_metadata_file`", I could not do that. That is because though I tried to do that and run `x.py test`, I got a lot of test failures as follows.
<details>
<summary>List of failed tests</summary>
<pre>
<code>
failures:
[ui] src/test/ui/json-multiple.rs
[ui] src/test/ui/json-options.rs
[ui] src/test/ui/rmeta/rmeta-rpass.rs
[ui] src/test/ui/save-analysis/emit-notifications.rs
[ui] src/test/ui/svh/changing-crates.rs
[ui] src/test/ui/svh/svh-change-lit.rs
[ui] src/test/ui/svh/svh-change-significant-cfg.rs
[ui] src/test/ui/svh/svh-change-trait-bound.rs
[ui] src/test/ui/svh/svh-change-type-arg.rs
[ui] src/test/ui/svh/svh-change-type-ret.rs
[ui] src/test/ui/svh/svh-change-type-static.rs
[ui] src/test/ui/svh/svh-use-trait.rs
test result: FAILED. 12915 passed; 12 failed; 100 ignored; 0 measured; 0 filtered out; finished in 71.41s
Some tests failed in compiletest suite=ui mode=ui host=x86_64-unknown-linux-gnu target=x86_64-unknown-linux-gnu
Build completed unsuccessfully in 0:01:58
</code>
</pre>
</details>
- I could not resolve the extra tasks about `create_rmeta_file` and `create_compressed_metadata_file` for my lack of ability.
macros: `LintDiagnostic` derive
- Move `LintDiagnosticBuilder` into `rustc_errors` so that a diagnostic derive can refer to it.
- Introduce a `DecorateLint` trait, which is equivalent to `SessionDiagnostic` or `AddToDiagnostic` but for lints. Necessary without making more changes to the lint infrastructure as `DecorateLint` takes a `LintDiagnosticBuilder` and re-uses all of the existing logic for determining what type of diagnostic a lint should be emitted as (e.g. error/warning).
- Various refactorings of the diagnostic derive machinery (extracting `build_field_mapping` helper and moving `sess` field out of the `DiagnosticDeriveBuilder`).
- Introduce a `LintDiagnostic` derive macro that works almost exactly like the `SessionDiagnostic` derive macro except that it derives a `DecorateLint` implementation instead. A new derive is necessary for this because `SessionDiagnostic` is intended for when the generated code creates the diagnostic. `AddToDiagnostic` could have been used but it would have required more changes to the lint machinery.
~~At time of opening this pull request, ignore all of the commits from #98624, it's just the last few commits that are new.~~
r? `@oli-obk`
`SessionDiagnostic` isn't suitable for use on lints as whether or not it
creates an error or a warning is decided at compile-time by the macro,
whereas lints decide this at runtime based on the location of the lint
being reported (as it will depend on the user's `allow`/`deny`
attributes, etc). Re-using most of the machinery for
`SessionDiagnostic`, this macro introduces a `LintDiagnostic` derive
which implements a `DecorateLint` trait, taking a
`LintDiagnosticBuilder` and adding to the lint according to the
diagnostic struct.
Bump RLS to latest master on rust-lang/rls
Of primary interest, this merges
rust-lang/rls@ece09b88c0 into rust-lang/rust,
which brings in the changes that fix RLS tests broken by #97853. #97853 already
introduced that commit's changes (under
rust-lang/rls@27f4044df0) but without putting those changes on
rust-lang/rls as a branch, so we ended up with an orphan commit that caused
trouble when updating submodules in rust-lang/rust.
This commit, once merged into rust-lang/rust, should continue to let RLS tests
to pass on rust-lang/rust's side and move us back into a healthy state where tip
of the submodule points to a valid master commit in the rust-lang/rls
repository.
cc https://github.com/rust-lang/rust/issues/98451, but not marking as fixed as I believe we need to add verification to prevent future oversights.
Fully remove submodule handling from bootstrap.py
These submodules were previously updated in python because Cargo gives a hard error if toml files
are missing from the workspace:
```
error: failed to load manifest for workspace member `/home/jnelson/rust-lang/rust/src/tools/rls`
Caused by:
failed to read `/home/jnelson/rust-lang/rust/src/tools/rls/Cargo.toml`
Caused by:
No such file or directory (os error 2)
failed to run: /home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /home/jnelson/rust-lang/rust/src/bootstrap/Cargo.toml
```
However, bootstrap doesn't actually need to be part of the workspace.
Remove it so we can move submodule handling fully to Rust, avoiding duplicate code between Rust and Python.
Note that this does break `cargo run`; it has to be `cd src/bootstrap && cargo run` now.
Given that we're planning to make the main entrypoint a shell script (or rust binary),
I think this is a good tradeoff for reduced complexity in bootstrap.py.
To get this working, I also had to remove support for vendoring when using the git sources, because `cargo vendor` requires all submodules to be checked out. I think this is ok; people who care about this are likely already using the pre-vendored `rustc-src` tarball.
Fixes https://github.com/rust-lang/rust/issues/90764. Helps with #94829
Of primary interest, this merges
rust-lang/rls@ece09b88c0 into rust-lang/rust,
which brings in the changes that fix RLS tests broken by #97853. #97853 already
introduced that commit's changes (under
27f4044df03d15c7c38a483c3e4635cf4f51807d) but without putting those changes on
rust-lang/rls as a branch, so we ended up with an orphan commit that caused
trouble when updating submodules in rust-lang/rust.
This commit, once merged into rust-lang/rust, should continue to let RLS tests
to pass on rust-lang/rust's side and move us back into a healthy state where tip
of the submodule points to a valid master commit in the rust-lang/rls
repository.
Migrate two diagnostics from the `rustc_builtin_macros` crate
Migrate two diagnostics to use the struct derive and be translatable.
r? ```@davidtwco```
These submodules were previously updated in python because Cargo gives a hard error if toml files
are missing from the workspace:
```
error: failed to load manifest for workspace member `/home/jnelson/rust-lang/rust/src/tools/rls`
Caused by:
failed to read `/home/jnelson/rust-lang/rust/src/tools/rls/Cargo.toml`
Caused by:
No such file or directory (os error 2)
failed to run: /home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --manifest-path /home/jnelson/rust-lang/rust/src/bootstrap/Cargo.toml
```
However, bootstrap doesn't actually need to be part of the workspace.
Remove it so we can move submodule handling fully to Rust, avoiding duplicate code between Rust and Python.
Note that this does break `cargo run`; it has to be `cd src/bootstrap && cargo run` now.
Given that we're planning to make the main entrypoint a shell script (or rust binary),
I think this is a good tradeoff for reduced complexity in bootstrap.py.
Make "Assemble stage1 compiler" orders of magnitude faster (take 2)
This used to take upwards of 5 seconds for me locally. I found that the culprit was copying the downloaded LLVM shared object:
```
[22:28:03] Install "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/ci-llvm/lib/libLLVM-14-rust-1.62.0-nightly.so" to "/home/jnelson/rust-lang/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libLLVM-14-rust-1.62.0-nightly.so"
[22:28:09] c Sysroot { compiler: Compiler { stage: 1, host: x86_64-unknown-linux-gnu(x86_64-unknown-linux-gnu) } }
```
It turned out that `install()` used full copies unconditionally. Change it to try using a hard-link before falling back to copying.
- Panic if we generate a symbolic link in a tarball
- Change install to use copy internally, like in my previous PR
- Change copy to dereference symbolic links, which avoids the previous regression in #96803.
I also took the liberty of fixing `x dist llvm-tools` to work even if you don't call `x build` previously.
