Book

Markov Chain Monte Carlo Methods

Nawaf Bou-Rabee and Andreas Eberle

A No-U-Turn Sampler chain on a triple-well potential: the mint points are the chain’s draws, and the gold path is a single NUTS transition, from the current state (white circle) to the next draw (gold dot). Real simulation.

Markov Chain Monte Carlo methods produce samples from a probability distribution that is known only up to a normalizing constant, by simulating a Markov process whose invariant distribution is the target. They are a basic tool of computational statistics, machine learning, and the physical sciences. This book gives a self-contained introduction to their design and to their quantitative analysis. The first part presents the main classes of methods: Metropolis–Hastings algorithms, Gibbs samplers, overdamped Langevin dynamics and its discretizations, auxiliary variable methods, Hamiltonian Monte Carlo and the No-U-turn sampler, and piecewise deterministic Markov processes. The second part develops the mathematical foundations of convergence to equilibrium: mixing and relaxation times, functional inequalities, couplings and transportation metrics, and quantitative bounds for ergodic averages. A third part treats advanced topics, including non-reversible lifts, MCMC methods on function spaces, and connections to geometry. The text grew out of graduate courses taught at Bonn and at Rutgers, and is intended for graduate students and researchers in mathematics, statistics, and machine learning.

The book will be published by Birkhäuser in the series Compact Textbooks in Mathematics. Materials and updates related to the project will be posted on this page.