Feature-space selection with banded ridge regression

• Published in: NeuroImage (Orlando, Fla.) Vol. 264; p. 119728
• Main Authors: Dupré la Tour, Tom, Eickenberg, Michael, Nunez-Elizalde, Anwar O., Gallant, Jack L.
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01-12-2022; Elsevier Limited; Elsevier
• Subjects: Complementarity; Computational neuroscience; Decomposition; Encoding models; Group sparsity; Humans; Hyperparameter optimization; Hypotheses; Linear Models; Mathematical models; Medical imaging; Methods; Neuroimaging; Regression Analysis; Regularized regression; Semantics; Variance decomposition; Neuroimaging; Group sparsity; Regularized regression; Hyperparameter optimization; Variance decomposition; Encoding models
• ISSN: 1053-8119; 1095-9572
• DOI: 10.1016/j.neuroimage.2022.119728

•Using multiple feature spaces in a joint encoding model improves prediction accuracy.•The variance explained by the joint model can be decomposed over feature spaces.•Banded ridge regression optimizes the regularization for each feature space.•Banded ridge regression contains an implicit feature-space selection mechanism.•Banded ridge regression can be solved with random search or gradient descent.
Encoding models provide a powerful framework to identify the information represented in brain recordings. In this framework, a stimulus representation is expressed within a feature space and is used in a regularized linear regression to predict brain activity. To account for a potential complementarity of different feature spaces, a joint model is fit on multiple feature spaces simultaneously. To adapt regularization strength to each feature space, ridge regression is extended to banded ridge regression, which optimizes a different regularization hyperparameter per feature space. The present paper proposes a method to decompose over feature spaces the variance explained by a banded ridge regression model. It also describes how banded ridge regression performs a feature-space selection, effectively ignoring non-predictive and redundant feature spaces. This feature-space selection leads to better prediction accuracy and to better interpretability. Banded ridge regression is then mathematically linked to a number of other regression methods with similar feature-space selection mechanisms. Finally, several methods are proposed to address the computational challenge of fitting banded ridge regressions on large numbers of voxels and feature spaces. All implementations are released in an open-source Python package called Himalaya.