Inverse problems
March 30, 2016 — June 30, 2022
Robert Ackroyd introduces some handy phrasing for the connections between statistical estimation theory and inverse problem solving.
Photogrammetry, MRIs, SLAM, volumetric reconstruction, and X-ray crystallography are all examples of inverse problems. Any of these can be constructed as classical belief propagation, especially Gaussian BP, or in a basic case least squares.
I think this is a case where it’s much easier to explain in terms of Bayesian inference, so my attempt at an actual explanation is under Bayesian inverse problems.
We can do it in terms of frequentist methods, but it doesn’t add much in the way of explanatory value; we end up considering regularizers instead of priors, but the working in between is pretty much the same. (IMO) However, in doing so we focus on point estimates rather than entire densities, which encourages us to solve the problem by optimisation rather than integration, which is a useful insight, for example, when we consider Laplace approximations.
1 Domain-specific model inversion
PEST, PEST++, and pyemu are some integrated systems for uncertainty quantification that use some weird terminology, such as FOSM (First-order-second-moment) models. They use various linear-algebra tricks to find plausible subspaces and samples.
This is a Python package to invert or solve many classical problems in atmospheric sciences and physical oceanography. These geophysical fluid dynamics (GFD) problems are formulated as second-order partial differential equations (PDEs), and can be inverted using successive-over-relaxation (SOR) iteration with proper boundary conditions. This project is published on GitHub and can be cited using its Zenodo DOI.
2 Interesting specific techniques
Learning to reconstruct introduces a partly-learned, partly designed reconstruction operator trick. 🏗️
3 Radiance fields
A fun way of reconstructing objects from photos; differentiable photogrammetry.