analiza aiqbench
This arXiv preprint studies whether scaling data and model size improves EEG foundation models, using up to 1.97 billion parameters and tens of thousands of hours of recordings. Its main finding is that gains are uneven: larger encoders help full fine-tuning more consistently than linear probing, and lower pretraining loss can coincide with worse downstream performance. Readers working on scaling laws or transfer learning should note that the benchmark gains are modest (a few percentage points) and the results are still preprint-stage, not yet peer-reviewed.
LaBraM at Scale: Scaling Data and Models for EEG Pretraining
arXiv:2611.03165v1 Announce Type: new Abstract: LaBraM, an early EEG foundation model, demonstrated that large-scale EEG pretraining can yield strong representations for diverse downstream tasks. Building on this foundation, we expand pretraining data and model capacity while retaining its core encoder and objective. We train ten encoders with 5.8 million to 1.97 billion parameters on three EEG collections spanning 25,067 to 79,910 hours, adapting signal preparation and training to heterogeneous recordings. Across 23 tasks, Atom-L* reaches 59.05% mean balanced accuracy at its best evaluated full fine-tuning (Full-FT) checkpoint, 3.61 percentage points above the strongest baseline EEG foundation model. Quark-G leads linear probing (LP) at 54.33%, a 1.24-point margin. Both also lead average task rank. At the last evaluated checkpoints, larger encoders improve Full-FT more consistently than LP, and stronger LP does not guarantee stronger Full-FT. Preserving patch structure improves LP over average pooling. Data-fraction experiments suggest capacity-dependent saturation, while checkpoint trajectories show that downstream performance can decline as pretraining loss falls. Joint classification with a shared Qwen3 readout reaches 50.50% with Molecule-H and Qwen3-8B; increasing language-model size provides no consistent gain. These results demonstrate the value of scaling LaBraM and the importance of matching pretraining scale to downstream adaptation.
Abstract:LaBraM, an early EEG foundation model, demonstrated that large-scale EEG pretraining can yield strong representations for diverse downstream tasks. Building on this foundation, we expand pretraining data and model capacity while retaining its core encoder and objective. We train ten encoders with 5.8 million to 1.97 billion parameters on three EEG collections spanning 25,067 to 79,910 hours, adapting signal preparation and training to heterogeneous recordings. Across 23 tasks, Atom-L* reaches 59.05% mean balanced accuracy at its best evaluated full fine-tuning (Full-FT) checkpoint, 3.61 percentage points above the strongest baseline EEG foundation model. Quark-G leads linear probing (LP) at 54.33%, a 1.24-point margin. Both also lead average task rank. At the last evaluated checkpoints, larger encoders improve Full-FT more consistently than LP, and stronger LP does not guarantee stronger Full-FT. Preserving patch structure improves LP over average pooling. Data-fraction experiments suggest capacity-dependent saturation, while checkpoint trajectories show that downstream performance can decline as pretraining loss falls. Joint classification with a shared Qwen3 readout reaches 50.50% with Molecule-H and Qwen3-8B; increasing language-model size provides no consistent gain. These results demonstrate the value of scaling LaBraM and the importance of matching pretraining scale to downstream adaptation.
Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
arXiv:2611.03165 [cs.LG]
(or arXiv:2611.03165v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2611.03165 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 290 cuvinte
Subiecte conexe
Benchmarking EEG Foundation Models at Scale: Lessons from 20,000 Evaluations
arXiv · 30 sept. 2026
Memory Decoder at Scale: A Pretrained, Parametric Long-Term Memory
arXiv · 31 iul. 2026
Training and Scaling Compute-Optimal Physiological Waveform Foundation Models
arXiv · 6 oct. 2026
NeuroPB: Scaling Neural Decoding with Pretrained Behavioral Representations
arXiv · 6 aug. 2026