Methods for Scalable Markov Chain Monte Carlo Inference

• Main Author: Li, Dangna
• Format: Dissertation
• Language: English
• Published: ProQuest Dissertations & Theses 01-01-2018
• Subjects: Applied Mathematics; Computer Engineering
• ISBN: 9798662555525

This dissertation contains three methods aiming at improving the scalability of Markov Chain Monte Carlo (MCMC) sampling. The first method addresses the computation bottleneck of Metropolis-Hastings in Bayesian models with a large number of observations. As an extension to the first method, we have developed a new MCMC algorithm to handle multimodal high-dimensional posterior distributions with a large number of observations. Our objective with the third method is to improve the efficiency of MH in high dimension, by designing a novel local MH move to make an effective use of the gradient information of the log target density. In the first part, we propose a general framework of performing MCMC with only a mini-batch of data. We show that by estimating the MH ratio with only a subset of data, one can sample from the true posterior raised to a known temperature. We show by experiments that our algorithm, Mini-batch Tempered MCMC, can efficiently explore the landscape of a multimodal posterior distribution. In addition, based on the Equi-Energy sampler, we propose a new MCMC algorithm, which enables exact sampling from high-dimensional multimodal posteriors with well-separated modes. In the second part, we introduce a novel local MH move which we shall refer to as the Cone Move. Our goal is to address a crucial limitation of Langevin Dynamics, namely its inherent inefficiency in utilizing the gradient information in exploring the target distribution. Compared with Langevin Dynamics, Cone Move makes better use of the gradient information and thus generates more targeted moves along the shape of the target distribution. We propose a new MCMC algorithm, the Cone Sampler, which adapts to the geometry of the target density by alternating between Cone Move and Langevin Dynamics. We demonstrate the effectiveness of Cone Sampler by testing its performance on challenging sampling problems in high dimensions.