nordic-dev.net/rust
nordic-dev.net/rust
2
0
Fork 0
mirror of https://github.com/rust-lang/rust.git synced 2024-11-26 16:54:01 +00:00
Code Issues Packages Projects Releases Wiki Activity

Auto merge of #94704 - Kobzol:llvm-pgo-update-suite, r=Mark-Simulacrum

Browse Source 
Gather LLVM PGO profiles from `rustc-perf` suite on real-world crates

This PR expands the benchmark suite used to gather LLVM PGO profiles in CI from `libcore` to several real-world crates. I hand-picked a few crates, but the list is up for debate.

Previous results that we got from running `syn,cargo,serde` looked pretty [good](https://perf.rust-lang.org/compare.html?start=2631aeef823a9e16d31f999d3f07001e5fcc4b3d&end=abf097afa10dde1aa3d8a1d422724a46aab79bf3).

Running `libcore` + `rustc-perf` with some number of crates is repeated now (and for BOLT it will also be needed), so maybe we can extract it to a bash function?

r? `@Mark-Simulacrum`
This commit is contained in:
bors 2022-03-13 15:53:14 +00:00
commit ebed06fcba
1 changed files with 49 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

81
src/ci/pgo.sh
View File

@ -2,6 +2,40 @@
set -euxo pipefail
# Compile several crates to gather execution PGO profiles.
# Arg0 => builds (Debug, Opt)
# Arg1 => runs (Full, IncrFull, All)
# Arg2 => crates (syn, cargo, ...)
gather_profiles () {
cd /checkout/obj
# Compile libcore, both in opt-level=0 and opt-level=3
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc \
--edition=2021 --crate-type=lib ../library/core/src/lib.rs
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc \
--edition=2021 --crate-type=lib -Copt-level=3 ../library/core/src/lib.rs
cd rustc-perf
# Run rustc-perf benchmarks
# Benchmark using profile_local with eprintln, which essentially just means
# don't actually benchmark -- just make sure we run rustc a bunch of times.
RUST_LOG=collector=debug \
RUSTC=/checkout/obj/build/$PGO_HOST/stage0/bin/rustc \
RUSTC_BOOTSTRAP=1 \
/checkout/obj/build/$PGO_HOST/stage0/bin/cargo run -p collector --bin collector -- \
profile_local \
eprintln \
/checkout/obj/build/$PGO_HOST/stage2/bin/rustc \
Test \
--builds $1 \
--cargo /checkout/obj/build/$PGO_HOST/stage0/bin/cargo \
--runs $2 \
--include $3
cd /checkout/obj
}
rm -rf /tmp/rustc-pgo
# We collect LLVM profiling information and rustc profiling information in
@ -19,6 +53,19 @@ RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc \
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc \
--edition=2021 --crate-type=lib -Copt-level=3 ../library/core/src/lib.rs
# Compile rustc perf
cp -r /tmp/rustc-perf ./
chown -R $(whoami): ./rustc-perf
cd rustc-perf
# Build the collector ahead of time, which is needed to make sure the rustc-fake
# binary used by the collector is present.
RUSTC=/checkout/obj/build/$PGO_HOST/stage0/bin/rustc \
RUSTC_BOOTSTRAP=1 \
/checkout/obj/build/$PGO_HOST/stage0/bin/cargo build -p collector
gather_profiles "Debug,Opt" "Full" "syn,cargo,serde,ripgrep,regex,clap-rs,hyper-2"
# Merge the profile data we gathered for LLVM
# Note that this uses the profdata from the clang we used to build LLVM,
# which likely has a different version than our in-tree clang.
@ -36,38 +83,8 @@ python3 ../x.py build --target=$PGO_HOST --host=$PGO_HOST \
--stage 2 library/std \
--rust-profile-generate=/tmp/rustc-pgo
# Profile libcore compilation in opt-level=0 and opt-level=3
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc \
--edition=2021 --crate-type=lib ../library/core/src/lib.rs
RUSTC_BOOTSTRAP=1 ./build/$PGO_HOST/stage2/bin/rustc \
--edition=2021 --crate-type=lib -Copt-level=3 ../library/core/src/lib.rs
cp -r /tmp/rustc-perf ./
chown -R $(whoami): ./rustc-perf
cd rustc-perf
# Build the collector ahead of time, which is needed to make sure the rustc-fake
# binary used by the collector is present.
RUSTC=/checkout/obj/build/$PGO_HOST/stage0/bin/rustc \
RUSTC_BOOTSTRAP=1 \
/checkout/obj/build/$PGO_HOST/stage0/bin/cargo build -p collector
# benchmark using profile_local with eprintln, which essentially just means
# don't actually benchmark -- just make sure we run rustc a bunch of times.
RUST_LOG=collector=debug \
RUSTC=/checkout/obj/build/$PGO_HOST/stage0/bin/rustc \
RUSTC_BOOTSTRAP=1 \
/checkout/obj/build/$PGO_HOST/stage0/bin/cargo run -p collector --bin collector -- \
profile_local \
eprintln \
/checkout/obj/build/$PGO_HOST/stage2/bin/rustc \
Test \
--builds Check,Debug,Opt \
--cargo /checkout/obj/build/$PGO_HOST/stage0/bin/cargo \
--runs All \
--include externs,ctfe-stress-4,inflate,cargo,token-stream-stress,match-stress-enum
cd /checkout/obj
gather_profiles "Check,Debug,Opt" "All" \
"externs,ctfe-stress-4,inflate,cargo,token-stream-stress,match-stress-enum"
# Merge the profile data we gathered
./build/$PGO_HOST/llvm/bin/llvm-profdata \