analiza aiqbench
This research challenges the static assumptions of current training data attribution by introducing a framework that models influence as dynamic across distinct learning stages. Grounded in singular learning theory, the study demonstrates that influence patterns can exhibit non-monotonic shifts, such as sign flips, which directly correspond to a model's progression through semantic hierarchies. Validated both in toy models and at scale in language models, these findings offer a more nuanced understanding of how specific data points shape model development over time.
Influence Dynamics and Stagewise Data Attribution
arXiv:2510.12071v2 Announce Type: replace Abstract: Current training data attribution (TDA) methods treat the influence one sample has on another as static, but neural networks learn in distinct stages that exhibit changing patterns of influence. In this work, we introduce a framework for stagewise data attribution grounded in singular learning theory. We predict that influence can change non-monotonically, including sign flips and sharp peaks at developmental transitions. We first validate these predictions analytically and empirically in a toy model, showing that dynamic shifts in influence directly map to the model's progressive learning of a semantic hierarchy. Finally, we demonstrate these phenomena at scale in language models, where token-level influence changes align with known developmental stages.
Abstract:Current training data attribution (TDA) methods treat the influence one sample has on another as static, but neural networks learn in distinct stages that exhibit changing patterns of influence. In this work, we introduce a framework for stagewise data attribution grounded in singular learning theory. We predict that influence can change non-monotonically, including sign flips and sharp peaks at developmental transitions. We first validate these predictions analytically and empirically in a toy model, showing that dynamic shifts in influence directly map to the model's progressive learning of a semantic hierarchy. Finally, we demonstrate these phenomena at scale in language models, where token-level influence changes align with known developmental stages.
Machine Learning (cs.LG)
arXiv:2510.12071 [cs.LG]
(or arXiv:2510.12071v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2510.12071 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 201 cuvinte
Subiecte conexe
Which Influence Are We Estimating? The Role of Counterfactual Specifications in Data Attribution
arXiv · 28 sept. 2026
Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation
arXiv · 29 sept. 2026
Tracing Target Answers in Poisoned Retrieval Corpora via Token Influence Attribution
arXiv · 27 aug. 2026
Traceable Spectral Inference via Influence Functions: Efficient Data Attribution and Error Proxies for the Ariel Mission
arXiv · 25 aug. 2026