aiqbench analysis
The paper gives a formal test for whether a controller that reads only a few learned features, such as sparse autoencoder activations, loses decision-relevant information compared with full model state. On SmolLM2 it finds that two on/off features raise minimum loss by 0.0345 to 0.1166, so some safety-relevant decisions need different controller inputs, not just better training or architecture. For readers, the useful takeaway is that poor controller performance does not show whether the features lack information or the decision rule is weak, and this framework helps separate the two.
Decision Sufficiency of Learned Representation Interfaces for Safe Generative Control
arXiv:2610.27446v1 Announce Type: new Abstract: A controller can guide a language model toward safe and useful responses by choosing when to assist, ask for clarification, or refuse. When the controller uses selected internal features, poor performance alone cannot distinguish missing information from limitations of its decision rules or training. We study decision sufficiency: whether the best controller using a representation can match the minimum average loss attainable with full state information, under the same strategies and evaluation criteria. For interfaces with finitely many outputs, we compute the optimal loss exactly and separate the three sources of excess loss. Pairwise checks can miss information loss: states with the same controller input may share an optimal strategy in every pair but not across the whole group. We bound the number of states needed to reveal a conflict and characterize the fewest outputs an unrestricted interface needs to avoid all conflicts. We audit selected features from sparse autoencoders, which encode model activations using relatively few active features, in Qwen2.5, SmolLM2, and Phi. On SmolLM2 responses scored by two human raters, restricting the controller to two selected on/off features increases its minimum average loss by 0.0345--0.1166 relative to full state information, even with the governing policy supplied. Both independently trained autoencoders show the increase on both evaluation sets. The audit identifies decision failures that require changing the controller's input, not merely its architecture or training.
Abstract:A controller can guide a language model toward safe and useful responses by choosing when to assist, ask for clarification, or refuse. When the controller uses selected internal features, poor performance alone cannot distinguish missing information from limitations of its decision rules or training. We study decision sufficiency: whether the best controller using a representation can match the minimum average loss attainable with full state information, under the same strategies and evaluation criteria. For interfaces with finitely many outputs, we compute the optimal loss exactly and separate the three sources of excess loss. Pairwise checks can miss information loss: states with the same controller input may share an optimal strategy in every pair but not across the whole group. We bound the number of states needed to reveal a conflict and characterize the fewest outputs an unrestricted interface needs to avoid all conflicts. We audit selected features from sparse autoencoders, which encode model activations using relatively few active features, in Qwen2.5, SmolLM2, and Phi. On SmolLM2 responses scored by two human raters, restricting the controller to two selected on/off features increases its minimum average loss by 0.0345--0.1166 relative to full state information, even with the governing policy supplied. Both independently trained autoencoders show the increase on both evaluation sets. The audit identifies decision failures that require changing the controller's input, not merely its architecture or training.
Machine Learning (cs.LG)
arXiv:2610.27446 [cs.LG]
(or arXiv:2610.27446v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.27446 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 320 words
Related coverage
Action-Sufficient Goal Representations
arXiv · Jul 28, 2026
LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models
arXiv · Aug 19, 2026
Bayes-Sufficient Representations in Supervised Learning
arXiv · Jun 5, 2026
Degree Bounds for Separating Polynomial and Neural Invariants for Point Clouds
arXiv · Oct 21, 2026