aiqbench analysis
This paper tests whether machine-learning decoders of imagined speech from scalp EEG can work on people they weren't trained on, using three public datasets. Across 482 participant pairs, cross-subject transfer was indistinguishable from chance, while within-subject decoding was above chance. The authors also show that one apparently successful imagery contrast was actually picking up a post-stimulus state, which is a useful caution about evaluation artifacts in neural decoding.
Cross-subject decoding of imagined speech from scalp EEG: a leakage-controlled evaluation on three public corpora
arXiv:2610.20954v1 Announce Type: cross Abstract: Objective. A decoder of imagined speech that works for a new user without calibration would make a non-invasive speech interface practical. We tested whether imagined-speech content recorded with scalp electroencephalography (EEG) transfers across people, and examined why accuracy within a single person remains low. Approach. We analyzed three public corpora, ds003626, KaraOne and Track 3 of the 2020 International Brain-Computer Interface Competition, under a protocol that enforces split integrity by assertion and reports exact binomial thresholds, group tests and bootstrap confidence intervals. We trained decoders on each participant and tested them on every other, compared representational geometry across participants without a classifier, applied subject-level alignment, and tested five explanations for the within-subject ceiling. Main results. Across 482 ordered pairs of participants, no pair survived correction for multiple comparisons; mean pair-transfer accuracy lay within 0.006 of chance on every corpus, while every within-subject reference lay above it. Imagery against pre-stimulus rest generalized at about 0.61 after artifact and elapsed-time controls, but its decoder labelled the interval preceding imagery as imagery as often as imagery itself, so the contrast reflects a post-stimulus state, not imagery. Within subject, only the competition corpus decoded above threshold for every participant (0.3147; chance 0.200); syllable count explained little of this, but its frequency profile failed an artifact audit. Significance. In these corpora, imagined-speech content did not transfer between people through any representation we examined. Trial count per class was the only variable that ordered the corpora, which favors larger single-condition datasets over further model development.
Abstract:Objective. A decoder of imagined speech that works for a new user without calibration would make a non-invasive speech interface practical. We tested whether imagined-speech content recorded with scalp electroencephalography (EEG) transfers across people, and examined why accuracy within a single person remains low. Approach. We analyzed three public corpora, ds003626, KaraOne and Track 3 of the 2020 International Brain-Computer Interface Competition, under a protocol that enforces split integrity by assertion and reports exact binomial thresholds, group tests and bootstrap confidence intervals. We trained decoders on each participant and tested them on every other, compared representational geometry across participants without a classifier, applied subject-level alignment, and tested five explanations for the within-subject ceiling. Main results. Across 482 ordered pairs of participants, no pair survived correction for multiple comparisons; mean pair-transfer accuracy lay within 0.006 of chance on every corpus, while every within-subject reference lay above it. Imagery against pre-stimulus rest generalized at about 0.61 after artifact and elapsed-time controls, but its decoder labelled the interval preceding imagery as imagery as often as imagery itself, so the contrast reflects a post-stimulus state, not imagery. Within subject, only the competition corpus decoded above threshold for every participant (0.3147; chance 0.200); syllable count explained little of this, but its frequency profile failed an artifact audit. Significance. In these corpora, imagined-speech content did not transfer between people through any representation we examined. Trial count per class was the only variable that ordered the corpora, which favors larger single-condition datasets over further model development.
23 pages, 5 figures, 10 tables plus supplementary tables. To be submitted to Journal of Neural Engineering
Signal Processing (eess.SP); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC)
arXiv:2610.20954 [eess.SP]
(or arXiv:2610.20954v1 [eess.SP] for this version)
https://doi.org/10.48550/arXiv.2610.20954 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 367 words
Related coverage
Decoding Imagined Speech: A Strictly Subject-Independent Approach Using EEG
arXiv · Sep 25, 2026
Integrating Language Models into Listened and Imagined Speech Decoding from MEG
arXiv · Sep 30, 2026
Cross-Subject Generalization in Decoding Perceived Speech from Non-Invasive Brain Recordings
arXiv · Aug 25, 2026
Subject-Invariant Cross-Modal Decoding of Perceived Speech from Brain Recordings
arXiv · Sep 28, 2026