analiza aiqbench
This research identifies implicit multi-hop reasoning failures in Large Language Models as a fundamental limitation of pretraining exposure rather than a lack of factual knowledge. The study demonstrates that specific compositional contexts must be present during the pretraining phase for models to successfully chain related facts, establishing a necessary condition for advanced reasoning capabilities. This finding offers critical insights into the architectural and data requirements needed to overcome current bottlenecks in complex query resolution.
Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
arXiv:2606.09338v2 Announce Type: replace Abstract: Large Language Models fail at implicit multi-hop reasoning: a model answers "When was $X$ born?" and "Who is $Y$'s closest friend?" correctly but fails on "When was $Y$'s closest friend born?" in a single forward pass, even when both facts are perfectly memorized and individually retrievable. We study this failure in a controlled natural language setting with a strict separation between individuals exposed to compositional contexts during pretraining and those that never appear in any such context. We confirm that compositional failure persists even at 97% 1-hop accuracy, establishing the gap as a pretraining failure rather than a knowledge absence. We propose and test nine data-centric augmentation formats and find that compositional pretraining transfers to unseen questions for exposed individuals, but never to individuals absent from compositional pretraining, suggesting that exposure to compositional contexts during pretraining is a necessary condition for implicit multi-hop reasoning. Code is available at https://github.com/ykrmm/composition-bound .
Abstract:Large Language Models fail at implicit multi-hop reasoning: a model answers "When was $X$ born?" and "Who is $Y$'s closest friend?" correctly but fails on "When was $Y$'s closest friend born?" in a single forward pass, even when both facts are perfectly memorized and individually retrievable. We study this failure in a controlled natural language setting with a strict separation between individuals exposed to compositional contexts during pretraining and those that never appear in any such context. We confirm that compositional failure persists even at 97% 1-hop accuracy, establishing the gap as a pretraining failure rather than a knowledge absence. We propose and test nine data-centric augmentation formats and find that compositional pretraining transfers to unseen questions for exposed individuals, but never to individuals absent from compositional pretraining, suggesting that exposure to compositional contexts during pretraining is a necessary condition for implicit multi-hop reasoning. Code is available at this https URL .
Accepted at EMNLP 2026. Camera-ready version
Computation and Language (cs.CL)
arXiv:2606.09338 [cs.CL]
(or arXiv:2606.09338v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2606.09338 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 251 cuvinte
Subiecte conexe
Multi-Hop Knowledge Composition is Bound by Pretraining Exposure
arXiv · 9 iun. 2026
What's Continual Learning, and Why Might We Expect To See It In Advanced LLM Agents?
LessWrong · 12 iun. 2026
Reflection’s Beam model hits 501 billion parameters
Quantum Zeitgeist · 5 oct. 2026
To Learn is to Wander: Learning Across Graphs and Tasks with Random Walks
arXiv · 6 oct. 2026