analiza aiqbench
This research challenges the assumption that increasingly sophisticated transformer models are always superior at predicting human reading dynamics, suggesting instead that simpler n-gram statistics may better capture naturalistic eye-tracking metrics. The study highlights a critical nuance in AI evaluation where model complexity can sometimes degrade correlation with specific cognitive benchmarks compared to baseline statistical methods. For researchers modeling language processing and cognition, this implies that next-word prediction accuracy does not automatically translate to improved predictions of reading time.
N-gram-like Language Models Predict Naturalistic Reading Time Best
arXiv:2603.09872v2 Announce Type: replace Abstract: Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting naturalistic reading time. In this paper, we propose that this can be explained by reading time being shaped by simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neural language models whose predictions are most correlated with n-gram probability are also those that calculate probabilities that are the most correlated with eye-tracking-based metrics of reading time on naturalistic text.
Abstract:Recent work has found that contemporary language models such as transformers can become so good at next-word prediction that the probabilities they calculate become worse for predicting naturalistic reading time. In this paper, we propose that this can be explained by reading time being shaped by simple n-gram statistics rather than the more complex statistics learned by state-of-the-art transformer language models. We demonstrate that the neural language models whose predictions are most correlated with n-gram probability are also those that calculate probabilities that are the most correlated with eye-tracking-based metrics of reading time on naturalistic text.
Computation and Language (cs.CL)
arXiv:2603.09872 [cs.CL]
(or arXiv:2603.09872v2 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2603.09872 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 189 cuvinte
Subiecte conexe
[AINews] Meta Connect 2026: Muse glasses, voice, video, and Charm
Latent Space · 24 sept. 2026
Six Principles of Voice Design: Designing for the Ear and Encoding It into Agent Script
Salesforce Blog · 8 oct. 2026
Reasons to believe current AI models are conscious
LessWrong · 17 iul. 2026
Links #5: 2026/07
LessWrong · 31 iul. 2026