Books

Markov Chain Monte Carlo Methods

Nawaf Bou-Rabee and Andreas Eberle

A No-U-Turn Sampler on a seven-well potential: the points are the chain’s draws, and the red paths are three NUTS transitions, each from its current state (open circle) to the next draw (gold dot).

Markov Chain Monte Carlo methods produce samples from a probability distribution that is known only up to a normalizing constant, by simulating a Markov process whose invariant distribution is the target. They are a basic tool of computational statistics and machine learning. This book gives a self-contained introduction to their design and to their quantitative analysis. The first part presents the main classes of methods: Metropolis–Hastings algorithms, Gibbs samplers, overdamped Langevin dynamics and its discretizations, auxiliary variable methods, Hamiltonian Monte Carlo and the No-U-turn sampler, and piecewise deterministic Markov processes. The second part develops the mathematical foundations of convergence to equilibrium: mixing and relaxation times, functional inequalities, couplings and transportation metrics, and quantitative bounds for ergodic averages. A third part treats advanced topics, including non-reversible lifts, MCMC methods on function spaces, MCMC methods for sequences of probability measures and their connections to diffusion modelling, and links to Riemannian geometry. The text grew out of graduate courses taught at Bonn and at Rutgers, and is intended for graduate students and researchers in mathematics, statistics, and machine learning.

The book will be published by Birkhäuser in the series Compact Textbooks in Mathematics. Materials and updates related to the project will be posted on this page.