Prediction and Generation of Binary Markov Processes: Can a Finite-State Fox Catch a Markov Mouse?

• Main Authors: Ruebeck, J, James, R. G, Mahoney, J. R, Crutchfield, J. P
• Format: Journal Article
• Language: English
• Published: 01-08-2017
• Subjects: Computer Science - Computational Complexity; Computer Science - Information Theory; Mathematics - Information Theory; Physics - Chaotic Dynamics; Physics - Statistical Mechanics
• DOI: 10.48550/arxiv.1708.00113

Understanding the generative mechanism of a natural system is a vital
component of the scientific method. Here, we investigate one of the fundamental
steps toward this goal by presenting the minimal generator of an arbitrary
binary Markov process. This is a class of processes whose predictive model is
well known. Surprisingly, the generative model requires three distinct
topologies for different regions of parameter space. We show that a previously
proposed generator for a particular set of binary Markov processes is, in fact,
not minimal. Our results shed the first quantitative light on the relative
(minimal) costs of prediction and generation. We find, for instance, that the
difference between prediction and generation is maximized when the process is
approximately independently, identically distributed.