The Consistency of MDL for Linear Regression Models With Increasing Signal-to-Noise Ratio

• Published in: IEEE transactions on signal processing Vol. 60; no. 3; pp. 1508 - 1510
• Main Authors: Schmidt, D. F., Makalic, E.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01-03-2012; Institute of Electrical and Electronics Engineers
• Subjects: Applied sciences; Bayesian methods; Computational modeling; Consistency; Data models; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Information theory; Information, signal and communications theory; linear models; Linear regression; Maximum likelihood estimation; minimum description length; model selection; Noise; Random variables; Signal and communications theory; Signal, noise; Telecommunications and information theory; Linear regression; Model selection; Consistency; Regression analysis; linear models; Linear model; Regression model; MDL criterion; Signal processing; Maximum likelihood; minimum description length; Signal to noise ratio; Information theory
• ISSN: 1053-587X; 1941-0476
• DOI: 10.1109/TSP.2011.2177833

Recent work by Ding and Kay has demonstrated that the Bayesian information criterion (BIC) is an inconsistent estimator of model order in nested model selection as the noise variance τ*→ 0. Unfortunately, Ding and Kay have erroneously concluded that the minimum description length (MDL) principle also leads to inconsistent estimates of model order in this setting by equating BIC with MDL. This correspondence shows that only the earlier MDL criterion based on asymptotic assumptions has this problem, and proves that the new MDL linear regression criteria based on normalized maximum likelihood and Bayesian mixture codes satisfy the notion of consistency as τ*→ 0. The main result may be used as a basis to easily establish similar consistency results for other closely related information theoretic regression criteria.