aiqbench analysis
This research identifies "odds-shift slippage" as a critical failure mode in one-vs-rest rankers handling rare labels, where standard positive-class weighting distorts log-odds and degrades precision@K. The authors demonstrate that finite learners fail to realize the theoretical shift due to saturation effects, causing significant performance loss that simple analytic inversion cannot fully recover. A new calibration recipe is proposed to mitigate these errors without harming models on scarce positive classes, offering a practical fix for imbalanced recommendation systems.
Odds-Shift Slippage in One-vs-Rest Rankers: Diagnosing and Repairing Reweighting-Induced Top-K Errors
arXiv:2609.13810v1 Announce Type: cross Abstract: One-vs-rest rankers that show each user the top-$K$ of many rare labels usually counter imbalance with a per-label positive-class weight, scale_pos_weight $= n_-/n_+$. Elkan's identity says such a weight shifts label $j$'s log-odds by $\ln w_j$, so the model ranks by weighted odds rather than by the marginal that is Bayes-optimal for precision@$K$, and suggests inverting the shift afterwards; what a finite learner does with a weight in the thousands, and which repair then works, has not been measured. We call the gap between the promised and the realized shift odds-shift slippage and measure it on matched pairs of LightGBM and MLP models that differ only in the weights. On Santander the weight takes MAP@7 from 0.808 to 0.117; for the boosted pairs the ideal odds shift accounts for 23% of that loss (32% on Instacart; 98% for an MLP pair on the same rows) and slippage for the rest. We prove that a booster whose leaf steps are capped at $c$ realizes at most $T\eta c$ nat of shift in $T$ rounds at rate $\eta$, which a cap sweep confirms, and show that without a cap saturated cells tie at exactly 1.0, beyond the reach of any separable map. The analytic inversion therefore pays only where the shift was realized and nothing saturated, whereas per-label isotonic regression returns the Santander model to 0.784 (0.780 with the calibrator fitted on the validation period), but only if labels without calibration positives are mapped to their prior rather than passed through. On 11 public MULAN benchmarks and 5 learners the weighted model loses more than half of its MAP@$K$ in 8 of 55 cells, and on delicious and Corel5k the same repair returns it to the unweighted level; per-label calibration hurts where positives are scarce, a harm that a cross-validated rule removes. The recipe is released as oddslip.
Abstract:One-vs-rest rankers that show each user the top-$K$ of many rare labels usually counter imbalance with a per-label positive-class weight, scale_pos_weight $= n_-/n_+$. Elkan's identity says such a weight shifts label $j$'s log-odds by $\ln w_j$, so the model ranks by weighted odds rather than by the marginal that is Bayes-optimal for precision@$K$, and suggests inverting the shift afterwards; what a finite learner does with a weight in the thousands, and which repair then works, has not been measured. We call the gap between the promised and the realized shift odds-shift slippage and measure it on matched pairs of LightGBM and MLP models that differ only in the weights. On Santander the weight takes MAP@7 from 0.808 to 0.117; for the boosted pairs the ideal odds shift accounts for 23% of that loss (32% on Instacart; 98% for an MLP pair on the same rows) and slippage for the rest. We prove that a booster whose leaf steps are capped at $c$ realizes at most $T\eta c$ nat of shift in $T$ rounds at rate $\eta$, which a cap sweep confirms, and show that without a cap saturated cells tie at exactly 1.0, beyond the reach of any separable map. The analytic inversion therefore pays only where the shift was realized and nothing saturated, whereas per-label isotonic regression returns the Santander model to 0.784 (0.780 with the calibrator fitted on the validation period), but only if labels without calibration positives are mapped to their prior rather than passed through. On 11 public MULAN benchmarks and 5 learners the weighted model loses more than half of its MAP@$K$ in 8 of 55 cells, and on delicious and Corel5k the same repair returns it to the unweighted level; per-label calibration hurts where positives are scarce, a harm that a cross-validated rule removes. The recipe is released as oddslip.
10 pages, 4 figures, 2 tables; 9-page supplement as ancillary file. Code, result files and paper sources: this https URL (Zenodo DOI https://doi.org/10.5281/zenodo.22719148)
Information Retrieval (cs.IR); Machine Learning (cs.LG)
arXiv:2609.13810 [cs.IR]
(or arXiv:2609.13810v1 [cs.IR] for this version)
https://doi.org/10.48550/arXiv.2609.13810 arXiv-issued DOI via DataCite (pending registration)
Full text under the publisher's open licence · 350 words
Related coverage
Anthropic reasoning without duplicates is just standard Bayesian updating
LessWrong · Jul 30, 2026
I Did This Without Your Knowledge
LessWrong · Oct 17, 2026
LiveLaw Supreme Court Half-Yearly Digest: January - June 2026 [Part 2]
livelaw.in · Sep 13, 2026
We’re Squandering LEDs’ Potential to Save Our Night Skies
IEEE Spectrum · Jul 20, 2026