Implementing End-to-End Brain Decoding from MEG Signals with NeuralSet and CNNs

A Coding Implementation of End-to-End Brain Decoding from MEG Signals Using NeuralSet and Deep Learning for Predicting Linguistic Features

This tutorial introduces a modern neuroAI pipeline designed to transform raw neural activity from MEG data into linguistic predictions. The system utilizes a convolutional neural network to estimate word length by learning temporal and spatial patterns in neural responses.

Why This Matters

Brain decoding often struggles with the high dimensionality and noise inherent in neural signals, making the transition from theoretical models to functional pipelines difficult. By employing structured frameworks like NeuralSet, researchers can manage data complexity through modular chains and segmenters, ensuring that preprocessing steps like 100 Hz signal extraction are reproducible and scalable for real-world neuroAI research.

Key Insights

• NeuralSet provides a structured data pipeline for neuroAI, allowing researchers to catalog and select specific studies such as Fake2025Meg or Test2023Meg for standardized experimentation.
• The MegExtractor tool enables high-frequency signal processing at 100.0 Hz to capture the rapid temporal dynamics of linguistic processing in the brain.
• A custom CharCount extractor, subclassed from BaseStatic, demonstrates how to map linguistic metadata directly to neural segments for supervised learning tasks.
• The MEGDecoder architecture employs a 1x1 spatial convolution followed by two temporal convolutions to decode features from multi-channel MEG data.
• Evaluation metrics like Pearson correlation (r) and MSE are critical for assessing how well synthetic or real brain signals correlate with linguistic features.

Working Examples

Defining a custom linguistic feature extractor and segmenting MEG data using NeuralSet.

import neuralset as ns
from neuralset import extractors as ext_mod
import torch
import torch.nn as nn

class CharCount(ext_mod.BaseStatic):
    event_types: str = "Word"
    def get_static(self, event) -> torch.Tensor:
        return torch.tensor([float(len(event.text))], dtype=torch.float32)

segmenter = ns.dataloader.Segmenter(
    extractors={
        "meg": {"name": "MegExtractor", "frequency": 100.0},
        "char_count": CharCount(aggregation="trigger"),
    },
    trigger_query="type == 'Word'",
    start=-0.2, duration=0.8,
    drop_incomplete=True,
)
dataset = segmenter.apply(events)

A CNN-based MEG decoder utilizing spatial and temporal convolutions for linguistic feature prediction.

class MEGDecoder(nn.Module):
    def __init__(self, n_channels: int, mid: int = 64):
        super().__init__()
        self.spatial = nn.Conv1d(n_channels, mid, 1)
        self.bn0 = nn.BatchNorm1d(mid)
        self.temporal1 = nn.Conv1d(mid, mid, 7, padding=3)
        self.bn1 = nn.BatchNorm1d(mid)
        self.temporal2 = nn.Conv1d(mid, mid//2, 7, padding=3)
        self.bn2 = nn.BatchNorm1d(mid//2)
        self.pool = nn.AdaptiveAvgPool1d(1)
        self.head = nn.Linear(mid//2, 1)
    
    def forward(self, x):
        x = torch.nn.functional.gelu(self.bn0(self.spatial(x)))
        x = torch.nn.functional.gelu(self.bn1(self.temporal1(x)))
        x = torch.nn.functional.gelu(self.bn2(self.temporal2(x)))
        return self.head(self.pool(x).squeeze(-1)).squeeze(-1)

Practical Applications

• Use Case: Building real-time word length estimation in neural interfaces for assistive communication devices. Pitfall: Using unnormalized MEG signals which can lead to training instability due to high variance across brain regions.
• Use Case: Modular brain signal preprocessing using NeuralSet’s Segmenter for rapid prototyping of different neuroimaging tasks. Pitfall: Failing to use drop_incomplete=True, resulting in mismatched tensor shapes during batch processing in the DataLoader.

References:

• https://www.marktechpost.com/2026/05/01/a-coding-implementation-of-end-to-end-brain-decoding-from-meg-signals-using-neuralset-and-deep-learning-for-predicting-linguistic-features/