Shrinkage of Covariance Matrices for Linear Signal Estimation Using Cross-Validation

• Published in: IEEE transactions on signal processing Vol. 64; no. 11; pp. 2965 - 2975
• Main Authors: Jun Tong, Schreier, Peter J., Qinghua Guo, Sheng Tong, Jiangtao Xi, Yanguang Yu
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-06-2016; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computational modeling; Covariance matrices; Covariance matrix; cross-validation (CV); Data models; Distributed databases; Estimation; Feature extraction; Genotype & phenotype; shrinkage; signal estimation; Training; Training data
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2016.2535392

Linear estimation of signals is often based on covariance matrices estimated from training, which can perform poorly if the training data are limited and the estimated covariance matrices are ill-conditioned. Shrinking the covariance matrix toward a scaled identity matrix can improve the robustness against the model uncertainty provided the shrinkage factor is appropriately chosen. This paper introduces several cross-validation schemes for choosing the shrinkage factors in applications where the covariance matrices are replaced with sample covariance matrices or constructed from least squares estimates of the linear model parameters. For cases where the training and out-of-training data are identically distributed, we derive leave-one-out cross-validation (LOOCV) schemes that repeatedly split the training data with respect to time to determine the optimal shrinkage factors for either model fitting or signal estimation. For cases where they are distributed differently, we develop alternative LOOCV schemes that repeatedly split the out-of-training observations with respect to space. We derive computationally efficient implementations of those schemes and provide examples to demonstrate their performance.