Foundation models for spatiotemporal systems

November 14, 2024 — November 14, 2024

dynamical systems
machine learning
neural nets
sciml
SDEs
signal processing
spatial
stochastic processes
time series
Figure 1

Foundation models for the whole planet, as opposed to classical NNs, as opposed to classical geospatial techniques.

1 Via modelling the governing equations

See foundation models for PDEs.

2 Incoming

3 References

Bodnar, Bruinsma, Lucic, et al. 2024. “Aurora: A Foundation Model of the Atmosphere.”
Duraisamy, Iaccarino, and Xiao. 2019. “Turbulence Modeling in the Age of Data.” Annual Review of Fluid Mechanics.
Grohs, and Herrmann. 2022. “Deep Neural Network Approximation for High-Dimensional Elliptic PDEs with Boundary Conditions.” IMA Journal of Numerical Analysis.
Guibas, Mardani, Li, et al. 2021. “Adaptive Fourier Neural Operators: Efficient Token Mixers for Transformers.”
Hoffimann, Zortea, de Carvalho, et al. 2021. “Geostatistical Learning: Challenges and Opportunities.” Frontiers in Applied Mathematics and Statistics.
Lam, Sanchez-Gonzalez, Willson, et al. 2023. “GraphCast: Learning Skillful Medium-Range Global Weather Forecasting.”
Mahesh, Collins, Bonev, et al. 2024. “Huge Ensembles Part I: Design of Ensemble Weather Forecasts Using Spherical Fourier Neural Operators.”
Pathak, Subramanian, Harrington, et al. 2022. “Fourcastnet: A Global Data-Driven High-Resolution Weather Model Using Adaptive Fourier Neural Operators.”
Schmude, Roy, Trojak, et al. 2024. “Prithvi WxC: Foundation Model for Weather and Climate.”
Willig, Zečević, Dhami, et al. 2022. “Can Foundation Models Talk Causality?”