analiza aiqbench
SAEExplainer introduces a closed-loop training framework that uses activation scores as a reward signal to iteratively refine feature explanations in Sparse Autoencoders. By leveraging mechanistic feedback for self-correction, the method significantly reduces hallucinations and strengthens causal triggering patterns compared to open-loop baselines. This approach offers a practical pathway for improving the interpretability and reliability of LLM internal representations through automated bootstrapping.
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
arXiv:2606.08496v1 Announce Type: cross Abstract: Although Sparse Autoencoders (SAEs) have mitigated the opacity of large language models (LLMs) by decomposing dense representations into sparse features, explaining these features still remains a central challenge. Current explanation methods, however, typically operate within an open-loop paradigm, failing to leverage mechanistic feedback for further refinement. In this paper, we propose SAEExplainer, a training framework utilizes activation scores as an objective reward signal to train the model for self-correction and iterative bootstrapping. By iteratively verifying and correcting foundational explanations through a two-round optimization process, SAEExplainer achieves continuous improvement in its explanatory capabilities. This mechanism significantly reduces explanation hallucinations and reinforces causal triggering patterns. Extensive experiments demonstrate our approach improves upon established baselines across most metrics, especially in causal triggering and discriminative activation.
Abstract:Although Sparse Autoencoders (SAEs) have mitigated the opacity of large language models (LLMs) by decomposing dense representations into sparse features, explaining these features still remains a central challenge. Current explanation methods, however, typically operate within an open-loop paradigm, failing to leverage mechanistic feedback for further refinement. In this paper, we propose SAEExplainer, a training framework utilizes activation scores as an objective reward signal to train the model for self-correction and iterative bootstrapping. By iteratively verifying and correcting foundational explanations through a two-round optimization process, SAEExplainer achieves continuous improvement in its explanatory capabilities. This mechanism significantly reduces explanation hallucinations and reinforces causal triggering patterns. Extensive experiments demonstrate our approach improves upon established baselines across most metrics, especially in causal triggering and discriminative activation.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2606.08496 [cs.CL]
(or arXiv:2606.08496v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.08496 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 220 cuvinte
Subiecte conexe
SAEExplainer: Interpreting SAE Features with Activation-Guided Preference Optimization
arXiv · 14 sept. 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · 18 sept. 2026
YFPO: Yoked Feature Preference Optimization with Neuron-Guided Rewards
arXiv · 16 sept. 2026
Introducing the Third Generation of Apple’s Foundation Models
Apple · 8 iun. 2026