Proper Calibeating

Abstract:The classic concept of "calibrated forecasts" and its more recent refinement, "calibeating," are defined with respect to the standard quadratic scoring rule. We extend these notions to the class of proper scoring rules (for which the true distribution is an optimal forecast) and define \textit{proper calibration} and \textit{proper calibeating} by requiring the corresponding guarantees to hold uniformly over all bounded proper scoring rules. We first establish that calibration always implies proper calibration, whereas calibeating need not imply proper calibeating. Second, we show how to guarantee proper calibeating and proper multicalibeating; in particular, \textit{complete calibeating}---a strong form of calibeating that calibeats the joint binning---is always proper. Finally, we consider \textit{decision-making under uncertainty}, where one best replies to the forecasts. We establish that (proper) calibration is equivalent to universal no regret, and (proper) complete calibeating is equivalent to universal no regret together with universal subsuming of the reference sequence, where \textit{universal} refers to all bounded utility functions.

Theoretical Economics (econ.TH); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG); Machine Learning (stat.ML)

arXiv:2605.26703 [econ.TH]

(or arXiv:2605.26703v3 [econ.TH] for this version)

https://doi.org/10.48550/arXiv.2605.26703 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.