Linear Regression With a Sparse Parameter Vector

• Published in: IEEE transactions on signal processing Vol. 55; no. 2; pp. 451 - 460
• Main Authors: Larsson, E.G., Selen, Y.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01-02-2007; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Approximation; Basis selection; Bayesian inference; Bayesian methods; Computational efficiency; Detection, estimation, filtering, equalization, prediction; Estimation error; Estimators; Exact sciences and technology; Information, signal and communications theory; Input variables; Lasso; Lasso linear regression; Linear regression; Mathematical analysis; Mathematical models; Maximum likelihood estimation; minimum mean-square error (MMSE) estimation; MMSE estimation; model averaging; model selection; Parameter estimation; Regression; Robustness; Signal and communications theory; Signal, noise; Sparse matrices; sparse models; TECHNOLOGY; TEKNIKVETENSKAP; Telecommunications and information theory; variable selection; Vectors; Vectors (mathematics); White noise; Bayes estimation; Performance evaluation; Averaging method; model averaging; Parameter estimation; Linear regression; Model selection; Lasso; minimum mean-square error (MMSE) estimation; Regression analysis; Mean square error; Regression model; A priori estimation; Numerical simulation; Robustness; variable selection; Basis selection; Bayesian inference; sparse models
• ISSN: 1053-587X; 1941-0476; 1941-0476
• DOI: 10.1109/TSP.2006.887109

We consider linear regression under a model where the parameter vector is known to be sparse. Using a Bayesian framework, we derive the minimum mean-square error (MMSE) estimate of the parameter vector and a computationally efficient approximation of it. We also derive an empirical-Bayesian version of the estimator, which does not need any a priori information, nor does it need the selection of any user parameters. As a byproduct, we obtain a powerful model ("basis") selection tool for sparse models. The performance and robustness of our new estimators are illustrated via numerical examples