Compile some CGUs in parallel at the start of codegen
This brings the compilation time for `syntex_syntax` from 11.542s to 10.453s with 6 threads in non-incremental debug mode. Just compiling `n` CGUs in parallel at the beginning of codegen seems sufficient to get rid of the staircase effect, at least for `syntex_syntax`.
Based on https://github.com/rust-lang/rust/pull/67777.
r? @michaelwoerister
cc @alexcrichton @Mark-Simulacrum
Rollup of 8 pull requests
Successful merges:
- #67666 (make use of pointer::is_null)
- #67806 (Extract `rustc_ast_passes`, move gating, & refactor linting)
- #68043 (Add some missing timers)
- #68074 (Add `llvm-skip-rebuild` flag to `x.py`)
- #68079 (Clarify suggestion for E0013)
- #68084 (Do not ICE on unicode next point)
- #68102 (Inline some conversion methods around OsStr)
- #68106 (Fix issue with using `self` module via indirection)
Failed merges:
r? @ghost
Inline some conversion methods around OsStr
Diff on the assembly of this snippet before and after this PR: https://www.diffchecker.com/NeGMjaJ2
```rust
use std::env;
use std::io;
use std::path::{Path, PathBuf};
pub fn cargo_home_with_cwd(cwd: &Path) -> io::Result<PathBuf> {
match env::var_os("CARGO_HOME").filter(|h| !h.is_empty()) {
Some(home) => {
let home = PathBuf::from(home);
if home.is_absolute() {
Ok(home)
} else {
Ok(cwd.join(&home))
}
}
_ => env::home_dir()
.map(|p| p.join(".cargo"))
.ok_or_else(|| io::Error::new(io::ErrorKind::Other, "could not find cargo home dir")),
}
}
```
Add `llvm-skip-rebuild` flag to `x.py`
This PR follows on from #67437 to complete the feature request from #65612.
Specifically it adds a new command-line flag, `--llvm-skip-rebuild`, which overrides both any value set in `config.toml` and the default value (`false`).
I'm not 100% confident that I've implemented the override in the "best" way, but I've checked it locally and it seems to work at least.
This option isn't currently mentioned in the Guide to Rustc Development. I'd be happy to write something on it if folk think that's worthwhile.
Extract `rustc_ast_passes`, move gating, & refactor linting
Based on https://github.com/rust-lang/rust/pull/67770.
This PR extracts a crate `rustc_ast_passes`:
- `ast_validation.rs`, which is contributed by `rustc_passes` (now only has HIR based passes) -- the goal here is to improve recompilation of the parser,
- `feature_gate.rs`, which is contributed by `syntax` and performs post-expansion-gating & final erroring for pre-expansion gating,
- `show_span`, which is contributed by `syntax`.
To facilitate this, we first have to also:
- Move `{leveled_}feature_err{_err}` from `syntax::feature_gate::check` into `rustc_session::parse`.
- Move `get_features` into `rustc_parse::config`, the only place it is used.
- Move some some lint datatypes and traits, e.g. `LintBuffer`, `BufferedEarlyLint`, `BuiltinLintDiagnostics`, `LintPass`, and `LintArray` into `rustc_session::lint`.
- Move all the hard-wired lint `static`s into `rustc_session::lint::builtin`.
build-std compatible sanitizer support
### Motivation
When using `-Z sanitizer=*` feature it is essential that both user code and
standard library is instrumented. Otherwise the utility of sanitizer will be
limited, or its use will be impractical like in the case of memory sanitizer.
The recently introduced cargo feature build-std makes it possible to rebuild
standard library with arbitrary rustc flags. Unfortunately, those changes alone
do not make it easy to rebuild standard library with sanitizers, since runtimes
are dependencies of std that have to be build in specific environment,
generally not available outside rustbuild process. Additionally rebuilding them
requires presence of llvm-config and compiler-rt sources.
The goal of changes proposed here is to make it possible to avoid rebuilding
sanitizer runtimes when rebuilding the std, thus making it possible to
instrument standard library for use with sanitizer with simple, although
verbose command:
```
env CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS=-Zsanitizer=thread cargo test -Zbuild-std --target x86_64-unknown-linux-gnu
```
### Implementation
* Sanitizer runtimes are no long packed into crates. Instead, libraries build
from compiler-rt are used as is, after renaming them into `librusc_rt.*`.
* rustc obtains runtimes from target libdir for default sysroot, so that
they are not required in custom build sysroots created with build-std.
* The runtimes are only linked-in into executables to address issue #64629.
(in previous design it was hard to avoid linking runtimes into static
libraries produced by rustc as demonstrated by sanitizer-staticlib-link
test, which still passes despite changes made in #64780).
cc @kennytm, @japaric, @firstyear, @choller
remove explicit strip-hidden pass from compiler doc generation
`strip-hidden` is now implied by `--document-private-items` with #67875, so there's no need to specify it anymore.
Build compiletest with in-tree libtest
This updates compiletest to build in `Mode::ToolStd`, using the locally-built crates for `std` and especially `test`. This way we're immune to unstable differences in the bootstrap compiler crates, whether that's a prior-release stage0 or a current release local rebuild. Fixes#59264.
As a minor cleanup, this also removes the unused `llvm_tools` flag.
