analiza aiqbench
This research introduces SAEVerbalizer, a framework that enables large language models to generate direct natural-language explanations for sparse autoencoder features without relying on external behavioral observations. By fine-tuning downstream layers with injected decoder directions, the method achieves generalization across unseen features and different model architectures while offering insights into how feature meanings combine or shift during intervention.
SAEVerbalizer: Generating Explanations for Sparse Autoencoder Features via Representation Verbalization
arXiv:2608.13538v2 Announce Type: replace Abstract: Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale. We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations. Our experiments show that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries, and, with a lightweight adapter, extends to SAE features from different LLMs. Intervention experiments show that injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.
Abstract:Sparse autoencoders (SAEs) are proposed to extract numerous features from large language model (LLM) representations, yet explaining these features still relies primarily on external observation. This reliance leads to superficial explanations inferred from observed model behavior and computational inefficiency from collecting such behavioral evidence at scale. We introduce SAEVerbalizer, a framework that injects SAE decoder directions into an LLM's representations and fine-tunes the LLM's downstream layers to generate natural-language explanations of the injected features. Once trained, the resulting verbalizer explains SAE features directly from decoder directions, addressing both limitations. Our experiments show that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries, and, with a lightweight adapter, extends to SAE features from different LLMs. Intervention experiments show that injecting multiple directions yields an explanation combining their meanings, while reversing individual directions produces corresponding meaning shifts.
Computation and Language (cs.CL)
arXiv:2608.13538 [cs.CL]
(or arXiv:2608.13538v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.13538 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 233 cuvinte
Subiecte conexe
J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses
LessWrong · 8 oct. 2026
DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing
arXiv · 16 iun. 2026
Thousand-dimensional structure
AI Alignment Forum · 30 iul. 2026
Where Do Apparent LLM Clinical Triage Failures Arise? Localizing the Multiple-Choice Format Effect
arXiv · 6 oct. 2026