Covariance matrix testing in high dimension using random projections

• Published in: Computational statistics Vol. 37; no. 3; pp. 1111 - 1141
• Main Authors: Ayyala, Deepak Nag, Ghosh, Santu, Linder, Daniel F.
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01-07-2022; Springer Nature B.V
• Subjects: Asymptotic properties; Covariance matrix; Economic Theory/Quantitative Economics/Mathematical Methods; Eigenvalues; Gene expression; Hypotheses; Hypothesis testing; Mathematics and Statistics; Multivariate analysis; Original Paper; Probability and Statistics in Computer Science; Probability Theory and Stochastic Processes; Random variables; Regression analysis; Statistics; Subspaces; Test procedures; Hypothesis testing; Random projections; High dimension; Covariance matrix
• ISSN: 0943-4062; 1613-9658
• DOI: 10.1007/s00180-021-01166-4

Estimation and hypothesis tests for the covariance matrix in high dimensions is a challenging problem as the traditional multivariate asymptotic theory is no longer valid. When the dimension is larger than or increasing with the sample size, standard likelihood based tests for the covariance matrix have poor performance. Existing high dimensional tests are either computationally expensive or have very weak control of type I error. In this paper, we propose a test procedure,
CRAMP
(
c
ovariance testing using
ra
ndom
m
atrix
p
rojections), for testing hypotheses involving one or more covariance matrices using random projections. Projecting the high dimensional data randomly into lower dimensional subspaces alleviates of the
curse
of dimensionality, allowing for the use of traditional multivariate tests. An extensive simulation study is performed to compare CRAMP against asymptotics-based high dimensional test procedures. An application of the proposed method to two gene expression data sets is presented.