What Shapes Brain Alignment of Large Language Models during Dialogue? Insights from Language Production and Comprehension

Title:What Shapes Brain Alignment of Large Language Models during Dialogue? Insights from Language Production and Comprehension

Abstract:Brain alignment of large language models (LLMs) offers a way to investigate representational similarities between LLMs and the human brain during language processing. While previous studies have mainly focused on passive language comprehension, little is known about such alignment during dialogue, in which speakers engage in both language production and comprehension. In this study, using fMRI data collected during dialogue, we examine brain alignment during language production and comprehension across LLM layers and brain regions. We assess its association with the intrinsic dimension (ID) of LLM representations and the contributions of several interpretable linguistic and communicative features. Our results reveal distinct patterns of brain alignment between production and comprehension, as well as strong associations with ID in brain regions associated with each processing mode. Furthermore, we show that the contributions of linguistic and communicative features also vary across processing modes, LLM layers, and brain regions.

Computation and Language (cs.CL); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC)

arXiv:2610.22961 [cs.CL]

(or arXiv:2610.22961v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.22961 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.