What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings

Abstract:Routing benchmarks often compare a policy that commits to one model before seeing its response with a hindsight oracle that credits any correct recorded output. Under stochastic decoding these are different decision classes. For success marginals $p_{im}$ under a frozen query--model--decoder--scorer protocol, we distinguish the clairvoyant single-commit ceiling $R_i=\max_m p_{im}$, product-coupling union $U_i^\perp=1-\prod_m(1-p_{im})$, and premium $\Delta_i^\perp=U_i^\perp-R_i$. The marginals alone identify exactly the union interval $[R_i,\min\{1,\sum_m p_{im}\}]$: zero premium is attainable over compatible couplings, not established for an actual deployment. If generation is independent across models, product is the specified protocol's union probability. We audit frozen correctness tensors from 11 open models and 30 archived responses per query--model cell on GSM8K, MATH-500, and GPQA-Diamond. Full-pool display-channel product premiums are 0.371, 3.542, and 5.100 percentage points; the premium intervals from empirical marginals are $[0,0.473]$, $[0,4.787]$, and $[0,7.744]$. Their zero lower endpoints are algebraic. All retained scorers, eight finite-draw paths, and all 2,047 nonempty subpools expose scorer, estimator, and pool sensitivity, not confidence intervals. The GSM8K and GPQA display scorers were developed after limited output inspection. A separate retrospective held-out policy illustration instantiates the policy-specific gap decomposition without establishing new-data generalization. A limited reference-based human check supports scorer agreement only on definite-consensus subsets. Hash-bound evidence supports number checks, not end-to-end reproduction. The contribution is a measurement contract for interpreting oracle gaps conditional on coupling, decision class, scorer, pool, and finite draws, not a population effect or an equal-cost routing gain.

Substantially revised and retitled; supersedes empirical summaries and interpretations in earlier versions. Coupling-aware fixed-archive audit, retrospective policy illustration, and limited reference-based human check. 19 pages, 8 figures. Source and evidence: this https URL

Machine Learning (cs.LG)

arXiv:2607.03436 [cs.LG]

(or arXiv:2607.03436v3 [cs.LG] for this version)