Meta AI Open-Sources NeuralBench: A Standardized Benchmark for EEG Foundation Models

Meta AI Releases NeuralBench: A Unified Open-Source Framework to Benchmark NeuroAI Models Across 36 EEG Tasks and 94 Datasets

Meta AI has launched NeuralBench-EEG v1.0, the largest open-source benchmark for evaluating brain activity AI models. The framework encompasses 13,603 hours of EEG data from 9,478 subjects to address the fragmented landscape of NeuroAI evaluation.

Why This Matters

Current NeuroAI research suffers from fragmented evaluation where research groups use inconsistent preprocessing pipelines and cherry-picked datasets, making it impossible to verify the ‘foundational’ nature of brain models. NeuralBench enforces technical standardization by providing a unified interface for 14 deep learning architectures, ensuring that performance gains are attributed to model design rather than optimization tricks like layer-wise learning rate decay or LoRA.

The framework reveals a significant technical plateau: massive foundation models like REVE (69.2M parameters) only marginally outperform lightweight task-specific models like CTNet (150K parameters). This suggests that scaling parameters in brain models does not yet yield the same exponential returns seen in LLMs, particularly for complex cognitive decoding tasks that remain near dummy-level performance across all tested architectures.

Key Insights

• NeuralBench-EEG v1.0 covers 36 downstream tasks across 8 categories, including clinical seizure detection, BCI, and cognitive decoding from 94 datasets.
• The CTNet architecture with 150K parameters achieves a mean normalized rank of 0.32, nearly matching the LUNA foundation model which has 40.4M parameters.
• Foundation models like REVE and LaBraM were evaluated using a shared training recipe including AdamW optimizer, 10^-4 learning rate, and cosine-annealing with 10% warmup.
• Cognitive decoding tasks for speech, sentence, and video remain largely unsolved, with even top-tier models frequently yielding performance close to dummy levels.
• The framework utilizes three modular Python packages: NeuralFetch for acquisition, NeuralSet for PyTorch-ready dataloaders, and NeuralTrain for modular execution.
• Cross-modality transfer was observed where the REVE model, pretrained only on EEG, achieved the highest performance on MEG typing decoding tasks.

Working Examples

CLI commands to install NeuralBench and execute a sample audiovisual stimulus classification task using the standardized pipeline.

pip install neuralbench
# Download, prepare, and run a specific EEG task
neuralbench eeg audiovisual_stimulus --download
neuralbench eeg audiovisual_stimulus --prepare
neuralbench eeg audiovisual_stimulus

Practical Applications

• Clinical Seizure Detection: Implementing standardized pipelines for pathology detection across 9,478 subjects. Pitfall: Failure to account for data leakage from pretraining sets can lead to inflated performance claims in medical applications.
• Brain-Computer Interfacing (BCI): Developing motor imagery decoding systems using cross-subject splits to ensure real-world generalization. Pitfall: Relying on within-subject splits often results in models that fail when deployed to new users.
• Cognitive Decoding: Attempting to recover speech or image representations from brain activity. Pitfall: High parameter models may overfit to noise in EEG signals without providing meaningful gains over simple convolutional baselines.

References:

• https://www.marktechpost.com/2026/05/07/meta-ai-releases-neuralbench-a-unified-open-source-framework-to-benchmark-neuroai-models-across-36-eeg-tasks-and-94-datasets/