analiza aiqbench
This research addresses a fundamental challenge in AI agent oversight by proving that welfare-maximizing approval rules are inherently non-affine when coupled with scoring mechanisms. The authors demonstrate that truthful reporting can be restored not by enforcing smoothness, but by designing rules with sufficient steepness to prevent strategic manipulation. These findings provide a rigorous mathematical framework for building more robust marketplaces and oversight systems for autonomous AI agents.
The Endogeneity of Miscalibration: Impossibility and Escape in Scored Reporting
arXiv:2605.07671v2 Announce Type: replace-cross Abstract: An agent's probability report is paid for twice: by a strictly proper scoring rule, and by an approval rule for the decision it triggers. In this classical decision-coupled setting, non-affine approval is known to defeat truthful reporting. We show the conflict is endogenous: when feasible, the welfare-maximizing approval rule is never affine. The distortion, however, is predictable and can be designed around. There is a reserve report at which pretending to be the marginal type costs exactly the approval prize. Approving at or above the reserve screens types perfectly under every strictly proper score, and the reserve does not depend on the type distribution. A Lipschitz rule with a single kink attains first-best exactly; under strict feasibility no continuously differentiable rule does. The binding constraint is steepness, not smoothness. First-best is attainable within a slope budget if and only if the budget is at least the critical slope: the steepest chord of the pretending cost up to the reserve. Below it the welfare loss is cubic in the shortfall. Where the pretending cost is convex up to the reserve, as for Brier, log and power scores, the critical slope is closed-form. The instances are AI-agent oversight and marketplace operation.
Abstract:An agent's probability report is paid for twice: by a strictly proper scoring rule, and by an approval rule for the decision it triggers. In this classical decision-coupled setting, non-affine approval is known to defeat truthful reporting. We show the conflict is endogenous: when feasible, the welfare-maximizing approval rule is never affine. The distortion, however, is predictable and can be designed around. There is a reserve report at which pretending to be the marginal type costs exactly the approval prize. Approving at or above the reserve screens types perfectly under every strictly proper score, and the reserve does not depend on the type distribution. A Lipschitz rule with a single kink attains first-best exactly; under strict feasibility no continuously differentiable rule does. The binding constraint is steepness, not smoothness. First-best is attainable within a slope budget if and only if the budget is at least the critical slope: the steepest chord of the pretending cost up to the reserve. Below it the welfare loss is cubic in the shortfall. Where the pretending cost is convex up to the reserve, as for Brier, log and power scores, the critical slope is closed-form. The instances are AI-agent oversight and marketplace operation.
38 pages, no figures. Targeting ACM Transactions on Economics and Computation (TEAC); preprint
Computer Science and Game Theory (cs.GT); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA); Theoretical Economics (econ.TH); Optimization and Control (math.OC)
arXiv:2605.07671 [cs.GT]
(or arXiv:2605.07671v2 [cs.GT] for this version)
Text integral sub licența deschisă a publicației · 239 cuvinte
Subiecte conexe
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · 3 aug. 2026
The Behavioral Credibility Trilemma: When Calibrated Autonomy Becomes Impossible
arXiv · 21 iul. 2026
Inside the suddenly explosive world of AI safety
The Verge · 17 sept. 2026
What We're Up Against: An AI Safety Crash Course
LessWrong · 24 sept. 2026