Reading the Gate, Not the Interference: Output-Side Interference Measurement Does Not Track Merge Collapse

Abstract:Model merging by task arithmetic works until it doesn't, and the field diagnoses why by measuring interference inside the merged model. We take the most direct such measure - the exact layerwise activation cross-term of a factorial ledger - establish its causal anatomy, and then ask what it tracks. The anatomy is clean: each block mostly transports and amplifies the cross-term rather than generating it anew; erased, it is regenerated by the untouched marginal paths to 99% of its norm unless removed near the output; its output effect varies monotonically with the displacement's angle, orthogonal displacements making interference worse; and a two-assumption model derives the angle law, retro-dicts the dose curve (R^2 = 0.99-1.00), and passes a preregistered scaling test. What the measure tracks is not what the field assumes. Behavioural expert-likeness is decoupled from it across four instruments. Its cross-condition behaviour is denominator-dominated: an instruction template pins the main effect within 1% while the absolute interaction grows 111x from two to six merged tasks. And where merging actually collapses, the cross-term is a bystander, not the carrier: across two collapse parameterizations at two scales - even erased persistently, at every position - removing it entirely repairs none of the collapse. There the output-side ratio carries no method information under a common counterfactual, while two state-space measures the field already uses rank methods correctly at both scales, on GSM8K and HumanEval. All 81 predictions were frozen before their data; falsifications, ours included, are reported as such. Output-side interference measurement reads the gate, the denominator, and the displacement budget - not the interference. What fails a merge is the carrier-bystander split: collapse rides in the marginal displacements, and only state space sees the carrier.

Machine Learning (cs.LG)

arXiv:2608.11797 [cs.LG]

(or arXiv:2608.11797v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2608.11797 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.