LassoLayer: Nonlinear Feature Selection by Switching One-to-one Links

• Main Authors: Sudo, Akihito, Hou, Teng Teck, Yamaguchi, Masaki, Tone, Yoshinori
• Format: Journal Article
• Language: English
• Published: 27-08-2021
• Subjects: Computer Science - Computer Vision and Pattern Recognition; Computer Science - Learning
• DOI: 10.48550/arxiv.2108.12165

Along with the desire to address more complex problems, feature selection
methods have gained in importance. Feature selection methods can be classified
into wrapper method, filter method, and embedded method. Being a powerful
embedded feature selection method, Lasso has attracted the attention of many
researchers. However, as a linear approach, the applicability of Lasso has been
limited. In this work, we propose LassoLayer that is one-to-one connected and
trained by L1 optimization, which work to drop out unnecessary units for
prediction. For nonlinear feature selections, we build LassoMLP: the network
equipped with LassoLayer as its first layer. Because we can insert LassoLayer
in any network structure, it can harness the strength of neural network
suitable for tasks where feature selection is needed. We evaluate LassoMLP in
feature selection with regression and classification tasks. LassoMLP receives
features including considerable numbers of noisy factors that is harmful for
overfitting. In the experiments using MNIST dataset, we confirm that LassoMLP
outperforms the state-of-the-art method.