Add reproducible profile-guided builds and benchmarks - #3983
Open
Matei02355 wants to merge 2 commits into
Open
Conversation
Matei02355
marked this pull request as ready for review
September 7, 2026 15:43
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Provide an optional native-host PGO build workflow so developers can train and measure bat on their own files.
The Python 3 script builds with the matching toolchain's LLVM tools, gathers fresh profiles, merges them, and rebuilds with profile-use. It preserves compiler flags, isolates each run, supports paths containing spaces, and verifies output hashes before reporting success. An optional baseline build measures alternating, warmed runs and records raw timings, medians, compiler details and output hashes in JSON. Documentation explains setup, reproducibility and workload-dependent results. Normal release builds remain unchanged; BOLT and release-binary training are outside this change.
Addresses #2701.
Validation: completed all three release builds with Rust 1.91.1 / LLVM 21.1.2 on x86_64 Linux, offline, in a work directory containing spaces. Used two training repetitions and ten benchmark repetitions. Baseline, instrumented and optimized output hashes matched for all six training file/mode combinations. Separate large JSON and Python files not used for training also produced identical baseline/optimized output in both modes.
Measured medians (milliseconds; this machine/run only):
Small timing differences can be noise; the plain README case regressed slightly. The holdout JSON contains 10,000 indented objects with id/name/enabled/values fields; Python repeats a three-line function 15,000 times. Compiler-flag inheritance and invalid inputs were also checked.