Adaptive Markov Chain Monte Carlo samplers

April 28, 2020 — April 30, 2020

Bayes
estimator distribution
Markov processes
Monte Carlo
probabilistic algorithms
probability
Figure 1: In adaptive MCMC, the trajectories of the simulator are perturbed by external forces (bottom right, centre) to change how they approach the target (top right)

Designing MCMC transition density by online optimisation for optimal mixing. Also called controlled MCMC.

Here we are no longer truly using a Markov chain because the transition parameters depend upon the entire history of the chain (for example because you are dynamically updating the transition parameters to improve mixing etc). Tutorials: Atchadé et al. (2011) and Andrieu and Thoms (2008).

With a Markov chain it is more complicated; if we perturb the transition density infinitely often we do not know in general that we will still converge to the target stationary distribution. However, we could do a “pilot” run to estimate optimal mixing kernels then use the adapted mixing kernels, discarding the samples from the pilot run as suspect and using the ones that remained. This is then a tuned MCMC rather than an adaptive MCMC.

Here I will keep notes, if any, on the perturbation problem. How do we guarantee that the proposal density is not changing too much by some criterion? Solutions to this seem to be sampler-specific.

1 References

Andrieu, and Robert. 2001. “Controlled MCMC for Optimal Sampling.” Working Paper 2001-33.
Andrieu, and Thoms. 2008. “A Tutorial on Adaptive MCMC.” Statistics and Computing.
Atchadé, Fort, Moulines, et al. 2011. “Adaptive Markov Chain Monte Carlo: Theory and Methods.” In Bayesian Time Series Models.
Gilks, Roberts, and Sahu. 1998. “Adaptive Markov Chain Monte Carlo Through Regeneration.” Journal of the American Statistical Association.
Griffin, Latuszynski, and Steel. 2019. “In Search of Lost (Mixing) Time: Adaptive Markov Chain Monte Carlo Schemes for Bayesian Variable Selection with Very Large p.” arXiv:1708.05678 [Stat].
Maire, Friel, Mira, et al. 2019. “Adaptive Incremental Mixture Markov Chain Monte Carlo.” Journal of Computational and Graphical Statistics.
Mathew, Bauer, Koistinen, et al. 2012. “Bayesian Adaptive Markov Chain Monte Carlo Estimation of Genetic Parameters.” Heredity.
Roberts, and Rosenthal. 2001. “Optimal Scaling for Various Metropolis-Hastings Algorithms.” Statistical Science.
———. 2009. “Examples of Adaptive MCMC.” Journal of Computational and Graphical Statistics.
Rosenthal. 2011. “Optimal Proposal Distributions and Adaptive MCMC.” In Handbook of Markov Chain Monte Carlo.
Sejdinovic, Strathmann, Garcia, et al. 2014. “Kernel Adaptive Metropolis-Hastings.” In International Conference on Machine Learning.
Strathmann, Sejdinovic, Livingstone, et al. 2015. “Gradient-Free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families.” In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1. NIPS’15.