Group-level Brain Decoding with Deep Learning

• Main Authors: Csaky, Richard, Van Es, Mats, Jones, Oiwi Parker, Woolrich, Mark
• Format: Journal Article
• Language: English
• Published: 19-01-2024
• Subjects: Computer Science - Learning; Quantitative Biology - Neurons and Cognition
• DOI: 10.48550/arxiv.2205.14102

Decoding brain imaging data are gaining popularity, with applications in
brain-computer interfaces and the study of neural representations. Decoding is
typicallysubject-specific and does not generalise well over subjects, due to
high amounts ofbetween subject variability. Techniques that overcome this will
not only providericher neuroscientific insights but also make it possible for
group-level models to out-perform subject-specific models. Here, we propose a
method that uses subjectembedding, analogous to word embedding in natural
language processing, to learnand exploit the structure in between-subject
variability as part of a decoding model,our adaptation of the WaveNet
architecture for classification. We apply this to mag-netoencephalography data,
where 15 subjects viewed 118 different images, with30 examples per image; to
classify images using the entire 1 s window followingimage presentation. We
show that the combination of deep learning and subjectembedding is crucial to
closing the performance gap between subject- and group-level decoding models.
Importantly, group models outperform subject models onlow-accuracy subjects
(although slightly impair high-accuracy subjects) and can behelpful for
initialising subject models. While we have not generally found
group-levelmodels to perform better than subject-level models, the performance
of groupmodelling is expected to be even higher with bigger datasets. In order
to providephysiological interpretation at the group level, we make use of
permutation featureimportance. This provides insights into the spatiotemporal
and spectral informationencoded in the models. All code is available on GitHub
(https://github.com/ricsinaruto/MEG-group-decode).