Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning

arXiv:2606.07560v3 Announce Type: replace Abstract: Function-vector (FV) heads (Todd et al., ICLR 2024) are identified by the magnitude of their causal contribution to in-context rule tasks, and the resulting top set is treated as a single functional class. We show that it holds two. Under a sign-preserving criterion (refined direct logit attribution, validated head by head with path patching) the FV population splits into writers, which push the rule-correct logit up, and cancellers, which push it down. The two groups are mechanistically distinct: writers place a third to a half of their attention on the demonstration labels, cancellers shift 9 to 17 points of that mass off the labels and mostly onto the format tokens, and the two groups' write directions are more anti-aligned than same-layer controls. Their oppositely signed effects combine additively at the readout, so they partly cancel and the FV set understates what its writers do. Magnitude-only ranking surfaces whichever group locally dominates and misses the other, so any function vector or ablation built that way averages a promoting and a suppressing mechanism. The signed split holds in all fifteen (model, task) cells we test, spanning six Pythia scales and three architectures, and a sign-shuffle null rejects a chance split in five of the six cells that carry the full statistical pipeline. Zero-ablating the cancellers recovers +0.13 to +0.29 nats on the correct label in all six and shifts accuracy by +2 to +7 pp.

Abstract:Function-vector (FV) heads (Todd et al., ICLR 2024) are identified by the magnitude of their causal contribution to in-context rule tasks, and the resulting top set is treated as a single functional class. We show that it holds two. Under a sign-preserving criterion (refined direct logit attribution, validated head by head with path patching) the FV population splits into writers, which push the rule-correct logit up, and cancellers, which push it down. The two groups are mechanistically distinct: writers place a third to a half of their attention on the demonstration labels, cancellers shift 9 to 17 points of that mass off the labels and mostly onto the format tokens, and the two groups' write directions are more anti-aligned than same-layer controls. Their oppositely signed effects combine additively at the readout, so they partly cancel and the FV set understates what its writers do. Magnitude-only ranking surfaces whichever group locally dominates and misses the other, so any function vector or ablation built that way averages a promoting and a suppressing mechanism. The signed split holds in all fifteen (model, task) cells we test, spanning six Pythia scales and three architectures, and a sign-shuffle null rejects a chance split in five of the six cells that carry the full statistical pipeline. Zero-ablating the cancellers recovers +0.13 to +0.29 nats on the correct label in all six and shifts accuracy by +2 to +7 pp.

Computation and Language (cs.CL); Machine Learning (cs.LG)

arXiv:2606.07560 [cs.CL]

(or arXiv:2606.07560v3 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2606.07560 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

aiqbench analysis

Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning

Related coverage