analiza aiqbench
This study presents a novel deep learning framework that decodes fMRI signals directly into textual descriptions without relying on visual input, achieving state-of-the-art semantic performance. By identifying the critical roles of higher-level visual cortices like the MT+ complex and ventral stream, the work offers an interpretable methodology for probing the neural basis of complex semantic processing. This approach bridges a significant gap between current decoding models and established neuroscientific theories, potentially informing the development of brain-inspired language models.
Brain2Text Decoding Model Reveals the Neural Mechanisms of Visual Semantic Processing
arXiv:2503.22697v3 Announce Type: replace-cross Abstract: Decoding sensory experiences from neural activity to reconstruct human-perceived visual stimuli and semantic content remains a challenge in neuroscience and artificial intelligence. Despite notable progress in current brain decoding models, a critical gap still persists in their systematic integration with established neuroscientific theories and the exploration of underlying neural mechanisms. Here, we present a novel framework that directly decodes fMRI signals into textual descriptions of viewed natural images. Our novel deep learning model, trained without visual information, achieves state-of-the-art semantic decoding performance, generating meaningful captions that capture the core semantic content of complex scenes. Neuroanatomical analysis reveals the critical role of higher-level visual cortices, including MT+ complex, ventral stream visual cortex, and inferior parietal cortex, in visual semantic processing. Furthermore, category-specific analysis demonstrates nuanced neural representations for semantic dimensions like animacy and motion. This work provides a more direct and interpretable framework to the brain's semantic decoding, offering a powerful new methodology for probing the neural basis of complex semantic processing, refining the understanding of the distributed semantic network, and potentially developing brain-inspired language models.
Abstract:Decoding sensory experiences from neural activity to reconstruct human-perceived visual stimuli and semantic content remains a challenge in neuroscience and artificial intelligence. Despite notable progress in current brain decoding models, a critical gap still persists in their systematic integration with established neuroscientific theories and the exploration of underlying neural mechanisms. Here, we present a novel framework that directly decodes fMRI signals into textual descriptions of viewed natural images. Our novel deep learning model, trained without visual information, achieves state-of-the-art semantic decoding performance, generating meaningful captions that capture the core semantic content of complex scenes. Neuroanatomical analysis reveals the critical role of higher-level visual cortices, including MT+ complex, ventral stream visual cortex, and inferior parietal cortex, in visual semantic processing. Furthermore, category-specific analysis demonstrates nuanced neural representations for semantic dimensions like animacy and motion. This work provides a more direct and interpretable framework to the brain's semantic decoding, offering a powerful new methodology for probing the neural basis of complex semantic processing, refining the understanding of the distributed semantic network, and potentially developing brain-inspired language models.
Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2503.22697 [q-bio.NC]
(or arXiv:2503.22697v3 [q-bio.NC] for this version)
Text integral sub licența deschisă a publicației · 196 cuvinte
Subiecte conexe
Google Research Shares 5 Visuals of Male Fruit Fly’s Neural Network
Quantum Zeitgeist · 15 sept. 2026
[AINews] not much happened today
Latent Space · 18 sept. 2026
Diagnosing Visual Ignorance in Vision-Language Models
arXiv · 8 iun. 2026
Does preservation make sense before we know how to revive?
LessWrong · 15 iun. 2026