analiza aiqbench
This paper proposes SMDA, a method that links individual fine-tuning examples to a symbolic, interpretable model of refusal behavior in Llama-3.2-3B-Instruct, using sparse autoencoder features and a closed-form Ridge fit. The evidence is a single model and behavior, with a reported accuracy gain of up to 4.5 points from retraining without flagged pairs, so the scalability and generality claims need testing on other models and behaviors. Its most practical contribution is the diagnostic workflow for spotting training pairs that push unintended features, which matters to teams doing SFT data curation and safety evaluation.
Symbolic Mechanistic Data Attribution: Tracing Training Influence to Learned Behavioral Policies
arXiv:2606.29171v2 Announce Type: replace-cross Abstract: While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the behavioral decisions a model learns. We introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes training pairs to the interpretable symbolic policies governing model behavior. SMDA fits a closed-form Ridge regression over sparse autoencoder (SAE) features to model a behavior, then decomposes how each supervised fine-tuning (SFT) example shifts that policy through feature-activation ($\Delta X$) and output-probability ($\Delta Y$) pathways. We distill a symbolic policy for refusal in Llama-3.2-3B-Instruct and analyze 200 SFT pairs. Our analysis reveals (1)~the symbolic policy's coefficients expose systematic gaps in the base model's safety behavior, such as religious stereotyping; (2)~per-feature $\Delta X/\Delta Y$ decomposition mechanistically explains why harmful and harmless pairs exert qualitatively different influences on certain features; and (3)~individual pairs routinely exhibit cross-feature interference, allowing SMDA to identify pairs whose dominant effect falls on unintended features. Retraining without such pairs improved accuracy by up to 4.5 percentage points over full fine-tuning. These results show that combining mechanistic interpretability with data attribution yields a diagnostic tool both more fine-grained than black-box influence functions and more scalable than manual circuit analysis.
Abstract:While existing data attribution methods can identify which training examples build specific mechanistic circuits, they cannot explain how training data shapes the behavioral decisions a model learns. We introduce Symbolic Mechanistic Data Attribution (SMDA), a framework that attributes training pairs to the interpretable symbolic policies governing model behavior. SMDA fits a closed-form Ridge regression over sparse autoencoder (SAE) features to model a behavior, then decomposes how each supervised fine-tuning (SFT) example shifts that policy through feature-activation ($\Delta X$) and output-probability ($\Delta Y$) pathways. We distill a symbolic policy for refusal in Llama-3.2-3B-Instruct and analyze 200 SFT pairs. Our analysis reveals (1)~the symbolic policy's coefficients expose systematic gaps in the base model's safety behavior, such as religious stereotyping; (2)~per-feature $\Delta X/\Delta Y$ decomposition mechanistically explains why harmful and harmless pairs exert qualitatively different influences on certain features; and (3)~individual pairs routinely exhibit cross-feature interference, allowing SMDA to identify pairs whose dominant effect falls on unintended features. Retraining without such pairs improved accuracy by up to 4.5 percentage points over full fine-tuning. These results show that combining mechanistic interpretability with data attribution yields a diagnostic tool both more fine-grained than black-box influence functions and more scalable than manual circuit analysis.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
arXiv:2606.29171 [cs.LG]
(or arXiv:2606.29171v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.29171 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 297 cuvinte
Subiecte conexe
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · 18 sept. 2026
Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units
arXiv · 9 iun. 2026
Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation
arXiv · 6 iun. 2026
STRIDE: Training Data Attribution via Sparse Recovery from Subset Perturbations
arXiv · 5 iun. 2026