Causal Localization of Evidence Effects in Transformers

Abstract:Can controlled symbolic inputs reveal attention heads that influence a language model's decisions from natural language evidence? We introduce controlled evidence aggregation (CEA), which varies the counts and order of two symbols in otherwise fixed prompts and measures changes in a frozen Transformer's preference between two answer tokens. The preference is measured by their logit difference; the prompts prescribe no correct answer. Balanced utility causal selection (BUC) identifies compact head sets by jointly evaluating how much ablating their outputs removes the count effect and how closely patching mean outputs between count conditions recovers the effect in both directions. We derive conditional bounds linking head attribution to intervention effects and specifying when the attribution shortlist can be recovered from finite samples. In Pythia-410M, ablating sets of 1-12 heads selected separately for four ordering regimes removes 65.5-91.6% of the count effect. The sets retain their influence on count comparisons not used for selection. To test relevance beyond symbolic inputs, we evaluate a natural language task asking which state is supported by more reports. Ablating eight heads selected only on symbolic yes/no prompts lowers mean accuracy by 8.2 percentage points, versus 0.8 for random sets matched by size and layer. Our results demonstrate that controlled symbolic probes can identify compact attention head sets whose causal influence extends to natural language decisions.

Computation and Language (cs.CL); Machine Learning (cs.LG)

arXiv:2610.36524 [cs.CL]

(or arXiv:2610.36524v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.36524 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

MANIFEST_SHA256.json

SUPPLEMENTARY_README.md

code/CEA_CONTINUE_REFERENCE_V9_1.ipynb

code/cea_colab_support.py

code/cea_fast_v9.py

code/cea_native_suite.py

code/cea_numeric_v9.py

code/requirements-colab.txt

code/setup_cea_env.py

code/test_numeric_v9.py

code/test_prefix_replay.py

code/test_runtime_resume_v91.py

code/test_v9_lifecycle.py

protocol/CODE_SHA256.json

protocol/README.md

protocol/SOURCE_TO_METHOD_MAP.md

protocol/engine_config.json

protocol/nl_state_registry.json

protocol/recorded_environment.json

protocol/scientific_config.json

results/reported_aggregate_results.csv

results/reported_aggregate_results.json

results/transfer_availability.csv

results/transfer_availability.json

tools/check_manuscript_math.py

tools/verify_package.py

(23 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.