Approximate Bayesian Computation

Posterior updates without likelihood

August 25, 2020 — September 20, 2021

Bayes
feature construction
likelihood free
machine learning
Monte Carlo
probabilistic algorithms
probability
signal processing
state space models
statistics
time series
Figure 1

Approximate Bayesian Computation is a terribly underspecified description. There are many ways that inference can be based on simulations, many types of freedom from likelihood, and many ways to approximate Bayesian computation. This page is about the dominant use of that term, which is the use of Simulation-based inference to do Bayes updates where the likelihood is not available but where we can simulate from the generative model.

Obviously, there are other ways you can approximate Bayesian computation — see e.g. variational Bayes.

TBD: relationship between this and simulation-based inference in a frequentist setting, often called indirect inference. They look similar but tend not to cite each other. Is this a technical or sociological hurdle?

Miles Cranmer’s Introduction to Simulation-based inference.

Maybe check the reading list for ABC toolkit ELFI.

1 SMC for ABC

One can solve for ABC using Sequential Monte Carlo. TBD.

2 Bayesian Synthetic Likelihood

Figure 2

TBD. Something about assuming the summary statistic is close to jointly Gaussian.

3 Neural methods

See neural likelihood-free methods.

4 SBC

Simulation-based Bayes calibration. Is this the same thing?

Martin Modrák, SBC Tutorial

5 Generalized Bayesian computation

A KL-free Bayesian computation extension. See generalized Bayes.

6 References

Alquier. 2020. “Approximate Bayesian Inference.” Entropy.
Baydin, Shao, Bhimji, et al. 2019. “Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale.” In arXiv:1907.03382 [Cs, Stat].
Beaumont, Zhang, and Balding. 2002. “Approximate Bayesian Computation in Population Genetics.” Genetics.
Blum, and François. 2010. “Non-Linear Regression Models for Approximate Bayesian Computation.” Statistics and Computing.
Corenflos, Thornton, Deligiannidis, et al. 2021. “Differentiable Particle Filtering via Entropy-Regularized Optimal Transport.” arXiv:2102.07850 [Cs, Stat].
Cranmer, Brehmer, and Louppe. 2020. “The Frontier of Simulation-Based Inference.” Proceedings of the National Academy of Sciences.
Cranmer, Pavez, and Louppe. 2015. “Approximating Likelihood Ratios with Calibrated Discriminative Classifiers.”
Dax, Wildberger, Buchholz, et al. 2023. “Flow Matching for Scalable Simulation-Based Inference.”
Diggle, and Gratton. 1984. “Monte Carlo Methods of Inference for Implicit Statistical Models.” Journal of the Royal Statistical Society: Series B (Methodological).
Drovandi, Christopher, and Frazier. 2021. “A Comparison of Likelihood-Free Methods With and Without Summary Statistics.” arXiv:2103.02407 [Stat].
Drovandi, Christopher C., Grazian, Mengersen, et al. 2018. “Approximating the Likelihood in Approximate Bayesian Computation.” arXiv:1803.06645 [Stat].
Durkan, Papamakarios, and Murray. 2018. “Sequential Neural Methods for Likelihood-Free Inference.”
Fan, Nott, and Sisson. 2013. “Approximate Bayesian Computation via Regression Density Estimation.” Stat.
Forneron, and Ng. 2015. “The ABC of Simulation Estimation with Auxiliary Statistics.” arXiv:1501.01265 [Stat].
Frazier, and Drovandi. 2021. “Robust Approximate Bayesian Inference With Synthetic Likelihood.” Journal of Computational and Graphical Statistics.
Frazier, Martin, and Robert. 2015. “On Consistency of Approximate Bayesian Computation.”
Frazier, Nott, Drovandi, et al. 2021. “Bayesian Inference Using Synthetic Likelihood: Asymptotics and Adjustments.” arXiv:1902.04827 [Stat].
Gelman, Vehtari, Simpson, et al. 2020. “Bayesian Workflow.” arXiv:2011.01808 [Stat].
Gourieroux, and Monfort. 1993. “Simulation-Based Inference: A Survey with Special Reference to Panel Data Models.” Journal of Econometrics.
Hermans, Delaunoy, Rozet, et al. 2023. “A Crisis In Simulation-Based Inference? Beware, Your Posterior Approximations Can Be Unfaithful.” Transactions on Machine Learning Research.
Izbicki, Lee, and Pospisil. 2019. “ABC–CDE: Toward Approximate Bayesian Computation With Complex High-Dimensional Data and Limited Simulations.” Journal of Computational and Graphical Statistics.
Le, Baydin, and Wood. 2017. “Inference Compilation and Universal Probabilistic Programming.” In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS). Proceedings of Machine Learning Research.
Lei, and Bickel. 2009. “Ensemble Filtering for High Dimensional Nonlinear State Space Models.” University of California, Berkeley, Rep.
Lueckmann, Bassetto, Karaletsos, et al. 2019. “Likelihood-Free Inference with Emulator Networks.” In Symposium on Advances in Approximate Bayesian Inference.
Meeds, and Welling. 2014. “GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation.” In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence. UAI’14.
Mohamed, and Lakshminarayanan. 2016. “Learning in Implicit Generative Models.”
Neal. 2008. “Computing Likelihood Functions for High-Energy Physics Experiments When Distributions Are Defined by Simulators with Nuisance Parameters.”
Nott, Drovandi, and Frazier. 2023. “Bayesian Inference for Misspecified Generative Models.”
Nott, Marshall, and Ngoc. 2012. “The Ensemble Kalman Filter Is an ABC Algorithm.” Statistics and Computing.
Ong, Victor M.-H., Nott, Tran, et al. 2018a. “Variational Bayes with Synthetic Likelihood.” Statistics and Computing.
Ong, Victor M. -H., Nott, Tran, et al. 2018b. “Likelihood-Free Inference in High Dimensions with Synthetic Likelihood.” Computational Statistics & Data Analysis.
Papamakarios, and Murray. 2016. “Fast ε-Free Inference of Simulation Models with Bayesian Conditional Density Estimation.” In Advances in Neural Information Processing Systems 29.
Park, Jitkrittum, and Sejdinovic. 2015. “K2-ABC: Approximate Bayesian Computation with Kernel Embeddings.”
Rubin. 1984. “Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician.” Annals of Statistics.
Säilynoja, Bürkner, and Vehtari. 2021. “Graphical Test for Discrete Uniformity and Its Applications in Goodness of Fit Evaluation and Multiple Sample Comparison.” arXiv:2103.10522 [Stat].
Schad, Betancourt, and Vasishth. 2021. “Toward a Principled Bayesian Workflow in Cognitive Science.” Psychological Methods.
Schmon, Cannon, and Knoblauch. 2021. “Generalized Posteriors in Approximate Bayesian Computation.” arXiv:2011.08644 [Stat].
Sisson, Scott A., Fan, and Beaumont. 2018. Handbook of Approximate Bayesian Computation.
Sisson, S. A., Fan, and Tanaka. 2007. “Sequential Monte Carlo Without Likelihoods.” Proceedings of the National Academy of Sciences.
Stoye, Brehmer, Louppe, et al. 2018. “Likelihood-Free Inference with an Improved Cross-Entropy Estimator.” arXiv:1808.00973 [Hep-Ph, Physics:physics, Stat].
Talts, Betancourt, Simpson, et al. 2020. “Validating Bayesian Inference Algorithms with Simulation-Based Calibration.”
Tran, Minh-Ngoc, Nott, and Kohn. 2017. “Variational Bayes With Intractable Likelihood.” Journal of Computational and Graphical Statistics.
Tran, Dustin, Ranganath, and Blei. 2017. “Hierarchical Implicit Models and Likelihood-Free Variational Inference.” In Advances in Neural Information Processing Systems 30.
Warne, Prescott, Baker, et al. 2021. “Multifidelity Multilevel Monte Carlo to Accelerate Approximate Bayesian Parameter Inference for Partially Observed Stochastic Processes.” arXiv:2110.14082 [q-Bio, Stat].