aiqbench analysis
This study reveals that language models do not fully replace lexical priors when instructed to override word meanings, but instead allow the original semantic associations to persist alongside local rule adjustments. Through activation patching, researchers identified a specific neural triplet responsible for this binding mechanism, demonstrating that interference effects are driven by the strength of these enduring priors rather than simple frequency or tokenization artifacts. For model developers, this implies that prompt engineering strategies relying on semantic remapping must account for the resilience of underlying lexical biases that continue to influence generation even after explicit instruction.
Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override
arXiv:2606.07555v1 Announce Type: cross Abstract: Glossaries, technical specifications, and system prompts routinely ask language models to use familiar words in unfamiliar ways. When this works, the lexical prior persists through override rather than being replaced: it continues to operate after the local rule applies, with the rule lowering its logit rather than installing the new meaning on top. We test this with a Stroop-style paradigm: a remapping rule ("doctor" means "forest") pitted against the query word's lexical-prior distractor ("hospital"), with matched neutral controls. Across 11 open-weight models spanning four families and 1B--9B parameters, lexical-prior strength predicts interference even after item-level controls for answer prior, frequency, tokenization, and prompt wording. Activation patching on five aligned models locates a source-position triplet (definition subject, definition target, query word) that nearly fully recovers the conflict effect (aggregate $R \in [0.92, 1.06]$). A definition-target swap shows the triplet performs binding rather than identity matching. Dissociation experiments isolate target preservation as the binding-specific signature: distractor suppression occurs under matched, swap, and item-mismatched conditions alike, whereas target logit collapse occurs only when the definition-target position is corrupted. Behavior and mechanism converge on the same channel: the lexical prior is where both interference originates and where override leaves its mark.
Abstract:Glossaries, technical specifications, and system prompts routinely ask language models to use familiar words in unfamiliar ways. When this works, the lexical prior persists through override rather than being replaced: it continues to operate after the local rule applies, with the rule lowering its logit rather than installing the new meaning on top. We test this with a Stroop-style paradigm: a remapping rule ("doctor" means "forest") pitted against the query word's lexical-prior distractor ("hospital"), with matched neutral controls. Across 11 open-weight models spanning four families and 1B--9B parameters, lexical-prior strength predicts interference even after item-level controls for answer prior, frequency, tokenization, and prompt wording. Activation patching on five aligned models locates a source-position triplet (definition subject, definition target, query word) that nearly fully recovers the conflict effect (aggregate $R \in [0.92, 1.06]$). A definition-target swap shows the triplet performs binding rather than identity matching. Dissociation experiments isolate target preservation as the binding-specific signature: distractor suppression occurs under matched, swap, and item-mismatched conditions alike, whereas target logit collapse occurs only when the definition-target position is corrupted. Behavior and mechanism converge on the same channel: the lexical prior is where both interference originates and where override leaves its mark.
Computation and Language (cs.CL); Machine Learning (cs.LG)
arXiv:2606.07555 [cs.CL]
(or arXiv:2606.07555v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.07555 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 293 words
Related coverage
Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override
arXiv · Jun 17, 2026
Priors Persist Through Suppression: A Stroop Paradigm for Lexical Override
arXiv · Aug 4, 2026
Reflections on unlearning and inoculation
LessWrong · Sep 20, 2026
MemVenom: Triggered Poisoning of Multimodal Memories in Web Agents
arXiv · Jun 10, 2026