Universal Models for the Exponential Distribution

• Published in: IEEE transactions on information theory Vol. 55; no. 7; pp. 3087 - 3090
• Main Authors: Schmidt, D.F., Makalic, E.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01-07-2009; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Australia; Bayesian analysis; Bayesian methods; Bulk molding compounds; Codes; Construction; Distributed computing; Exact sciences and technology; Exponential distribution; Information; Information theory; Information, signal and communications theory; Integral equations; Maximum likelihood estimation; Maximum likelihood method; Messages; Minimax techniques; Minimum description length (MDL); minimum message length (MML); Parametric statistics; Predictive models; Statistical distributions; Telecommunications and information theory; Theory; universal models; Exponential distribution; minimum message length (MML); universal models; MDL criterion; Maximum likelihood; Minimum description length (MDL); Information theory
• ISSN: 0018-9448; 1557-9654
• DOI: 10.1109/TIT.2009.2018331

This paper considers the problem of constructing information theoretic universal models for data distributed according to the exponential distribution. The universal models examined include the sequential normalized maximum likelihood (SNML) code, conditional normalized maximum likelihood (CNML) code, the minimum message length (MML) code, and the Bayes mixture code (BMC). The CNML code yields a codelength identical to the Bayesian mixture code, and within O (1) of the MML codelength, with suitable data driven priors.