On the Consistency of Maximum Likelihood Estimation of Probabilistic Principal Component Analysis

• Main Authors: Datta, Arghya, Chakrabarty, Sayak
• Format: Journal Article
• Language: English
• Published: 08-11-2023
• Subjects: Computer Science - Learning; Statistics - Machine Learning
• DOI: 10.48550/arxiv.2311.05046

Probabilistic principal component analysis (PPCA) is currently one of the
most used statistical tools to reduce the ambient dimension of the data. From
multidimensional scaling to the imputation of missing data, PPCA has a broad
spectrum of applications ranging from science and engineering to quantitative
finance.
Despite this wide applicability in various fields, hardly any theoretical
guarantees exist to justify the soundness of the maximal likelihood (ML)
solution for this model. In fact, it is well known that the maximum likelihood
estimation (MLE) can only recover the true model parameters up to a rotation.
The main obstruction is posed by the inherent identifiability nature of the
PPCA model resulting from the rotational symmetry of the parameterization. To
resolve this ambiguity, we propose a novel approach using quotient topological
spaces and in particular, we show that the maximum likelihood solution is
consistent in an appropriate quotient Euclidean space. Furthermore, our
consistency results encompass a more general class of estimators beyond the
MLE. Strong consistency of the ML estimate and consequently strong covariance
estimation of the PPCA model have also been established under a compactness
assumption.