aiqbench analysis
This study investigates how large language models geometrically organize and causally process relational information across transformer layers during compositional tasks. The findings reveal a consistent progression where intermediate layers infer pairwise relationships while later layers integrate joint dependencies, with restricting models to these causally relevant representations actually improving prediction accuracy. This work advances the understanding of internal model mechanisms required for handling complex relational reasoning in next-token prediction.
Causal and Interpretable Structures in LLM Compositional Tasks
arXiv:2609.35970v1 Announce Type: new Abstract: Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept (months, hours, weekdays, and musical notes) to correctly predict the next token. Across model families (Llama, Qwen, Gemma, and Mistral) and cyclic concepts, we find a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used: intermediate layers use a joint representation based on the inferred relationship between two tokens, while later layers use a joint representation associated with all three tokens to correctly complete the task. We also find other relationships between tokens that are geometrically structured but remain causally inert in the next-token prediction. Crucially, when taken together, these geometric and causal investigations reveal the representation-level mechanism that progressively organizes and composes the relational information to form the answer. More surprisingly, restricting the models to such causally relevant joint representations improves next-token prediction accuracy.
Abstract:Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept (months, hours, weekdays, and musical notes) to correctly predict the next token. Across model families (Llama, Qwen, Gemma, and Mistral) and cyclic concepts, we find a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used: intermediate layers use a joint representation based on the inferred relationship between two tokens, while later layers use a joint representation associated with all three tokens to correctly complete the task. We also find other relationships between tokens that are geometrically structured but remain causally inert in the next-token prediction. Crucially, when taken together, these geometric and causal investigations reveal the representation-level mechanism that progressively organizes and composes the relational information to form the answer. More surprisingly, restricting the models to such causally relevant joint representations improves next-token prediction accuracy.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2609.35970 [cs.CL]
(or arXiv:2609.35970v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2609.35970 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 287 words
Related coverage
PIRAMID: Progress and Plans
LessWrong · Jul 27, 2026
Mech Interp is a Verifiable Task
LessWrong · Sep 21, 2026
Introducing PIRAMID: Physics-Informed Research for Ambitious Mechanistic Interpretability
LessWrong · Jul 25, 2026
Bridging Linguistic Structure and Mechanistic Interpretability for Conceptual Interpretation in Language Models
arXiv · Aug 25, 2026