StrictPRMBench: A Replayable Evaluation Benchmark for Auditing Fixed-Score PRM Aggregation

Abstract:Process reward models (PRMs) score reasoning steps, but best-of-N selection also requires a rule that aggregates those scores. We study that rule as a separate evaluation object. StrictPRMBench fixes mathematical reasoning pools, scorer evidence, and the source trace of a deletion intervention, so that different reducers face the same recorded selection problem. Across seven scoring checkpoints and two benchmarks, the stored N=32 outcomes give higher deletion-induced selection instability for product than geometric mean (GM) in all 14 cells. The equal-cell difference is +0.0965; a shared-problem bootstrap, conditional on these checkpoints and frozen pools, gives a 95% interval of [0.0846,0.1086]. A pre-injection-defined sensitivity excludes replacement slots that are either reducer's clean winner and retains 4,459 of 4,622 cell-problem records. The retained difference is +0.0986 [0.0875,0.1098]. Effects are heterogeneous: the two Qwen cells have positive percentile-bootstrap intervals on retained support but two-sided exact paired p-values of 0.0625 and 0.0703. Clean-pool Qwen accuracy does not detect a product advantage, but this is not an equivalence result. We provide versioned outcome records, CPU reanalysis, and an explicit evidence-availability contract. The contribution is a diagnostic evaluation protocol and a reproducible audit of recorded selector behavior, not a new PRM, a universally optimal reducer, or evidence of deployment-wide harm. Raw-score replay and outcome-record reanalysis are distinguished: missing primary edited-score vectors limit independent evaluation of new reducers under the same stress.

Machine Learning (cs.LG)

arXiv:2610.41333 [cs.LG]

(or arXiv:2610.41333v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.41333 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

FIGURE_EDIT_LOG.md

RIGHTS_AND_PROVENANCE.md

apply_original_hero_labels.py

derive_ni_records.py

figure2_data.json

figure_label_corrections.json

input_manifest.json

inputs/historical_ni_summary.csv

inputs/human_natural_summary.csv

inputs/human_three_choice_summary.csv

inputs/math500_level_map.csv

inputs/math_level_records.csv

inputs/ni_pair_records.csv

inputs/qwen72b_records.csv

inputs/qwen_natural_records.csv

inputs/selection_records.csv

outputs/analysis_summary.json

outputs/cell_results.csv

outputs/environment.json

outputs/math_levels.csv

outputs/ni_cluster.csv

outputs/pooled_results.csv

outputs/qwen_natural.csv

provenance/expert_input_audit.txt

provenance/qwen_natural_records.csv.upstream_provenance.json

provenance/selection_records.csv.upstream_provenance.json

release_manifest.json

requirements.txt

test_reanalysis.py

(30 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.