Deep learning approaches for neural decoding: from CNNs to LSTMs and spikes to fMRI

• Main Authors: Livezey, Jesse A, Glaser, Joshua I
• Format: Journal Article
• Language: English
• Published: 19-05-2020
• Subjects: Computer Science - Learning; Quantitative Biology - Neurons and Cognition
• DOI: 10.48550/arxiv.2005.09687

Decoding behavior, perception, or cognitive state directly from neural
signals has applications in brain-computer interface research as well as
implications for systems neuroscience. In the last decade, deep learning has
become the state-of-the-art method in many machine learning tasks ranging from
speech recognition to image segmentation. The success of deep networks in other
domains has led to a new wave of applications in neuroscience. In this article,
we review deep learning approaches to neural decoding. We describe the
architectures used for extracting useful features from neural recording
modalities ranging from spikes to EEG. Furthermore, we explore how deep
learning has been leveraged to predict common outputs including movement,
speech, and vision, with a focus on how pretrained deep networks can be
incorporated as priors for complex decoding targets like acoustic speech or
images. Deep learning has been shown to be a useful tool for improving the
accuracy and flexibility of neural decoding across a wide range of tasks, and
we point out areas for future scientific development.