Skip to content

Add reproducible profile-guided builds and benchmarks - #3983

Open
Matei02355 wants to merge 2 commits into
sharkdp:masterfrom
Matei02355:profile-guided-build-2701-20260907
Open

Add reproducible profile-guided builds and benchmarks#3983
Matei02355 wants to merge 2 commits into
sharkdp:masterfrom
Matei02355:profile-guided-build-2701-20260907

Conversation

@Matei02355

Copy link
Copy Markdown
Contributor

Provide an optional native-host PGO build workflow so developers can train and measure bat on their own files.

The Python 3 script builds with the matching toolchain's LLVM tools, gathers fresh profiles, merges them, and rebuilds with profile-use. It preserves compiler flags, isolates each run, supports paths containing spaces, and verifies output hashes before reporting success. An optional baseline build measures alternating, warmed runs and records raw timings, medians, compiler details and output hashes in JSON. Documentation explains setup, reproducibility and workload-dependent results. Normal release builds remain unchanged; BOLT and release-binary training are outside this change.

Addresses #2701.

Validation: completed all three release builds with Rust 1.91.1 / LLVM 21.1.2 on x86_64 Linux, offline, in a work directory containing spaces. Used two training repetitions and ten benchmark repetitions. Baseline, instrumented and optimized output hashes matched for all six training file/mode combinations. Separate large JSON and Python files not used for training also produced identical baseline/optimized output in both modes.

Measured medians (milliseconds; this machine/run only):

Input Mode Baseline PGO Baseline/PGO
src/printer.rs highlighted 47.450 44.364 1.070
Cargo.toml highlighted 10.259 8.891 1.154
README.md highlighted 93.237 86.983 1.072
src/printer.rs plain 2.076 1.936 1.072
Cargo.toml plain 1.705 1.594 1.070
README.md plain 2.174 2.252 0.965
JSON holdout highlighted 1100.787 982.980 1.120
Python holdout highlighted 1052.670 990.963 1.062
JSON holdout plain 15.375 14.838 1.036
Python holdout plain 7.780 7.436 1.046

Small timing differences can be noise; the plain README case regressed slightly. The holdout JSON contains 10,000 indented objects with id/name/enabled/values fields; Python repeats a three-line function 15,000 times. Compiler-flag inheritance and invalid inputs were also checked.

@Matei02355
Matei02355 marked this pull request as ready for review September 7, 2026 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant