Symbolic system identification

March 14, 2023 — September 22, 2024

compsci
dynamical systems
machine learning
neural nets
optimization
physics
probabilistic algorithms
sciml
statistics
statmech
stochastic processes
stringology

Symbolic regression + system identification = symbolic system identification.

There is some nifty work in learning approximations to physics, like the SINDy method. The trick seems to be sparse regression plus interpretable features; so we might recover the equations of motion for a system from data, discover the laws of gravity from watching apples fall, that kind of thing. Brunton, Proctor, and Kutz (2016) is the canonical modern reference for this, although see Schmidt and Lipson (2009) for a primordial reference before autodifferentiation was a mainstream. It’s hard to imagine scaling this up to big things like large image sensor arrays and other such weakly structured input. I could be wrong.

1 Bayesian

Figure 1

Tricky! it seems to be that the sparse Bayes model selection problem is no easier when we mix symbolic regression in (Champneys and Rogers 2024; Fung, Fasel, and Juniper 2024; Hirsh, Barajas-Solano, and Kutz 2022; More et al. 2023).

2 Weak form in

Messenger et al. (2024):

The weak form is a ubiquitous, well-studied, and widely-utilized mathematical tool in modern computational and applied mathematics. In this work we provide a survey of both the history and recent developments for several fields in which the weak form can play a critical role. In particular, we highlight several recent advances in weak form versions of equation learning, parameter estimation, and coarse graining, which offer surprising noise robustness, accuracy, and computational efficiency.

Code: MathBioCU/WENDy: WENDy (Weak-form Estimation of Nonlinear Dynamics) is a forward solver-free algorithm for estimating parameters in differential equations.

3 Tools

4 References

A.Messenger, Dall’Anese, and Bortz. 2022. “Online Weak-Form Sparse Identification of Partial Differential Equations.” In Proceedings of Mathematical and Scientific Machine Learning.
Atkinson, Subber, and Wang. 2019. “Data-Driven Discovery of Free-Form Governing Differential Equations.” In.
Brunton, and Kutz. 2019. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control.
Brunton, Proctor, and Kutz. 2016. “Discovering Governing Equations from Data by Sparse Identification of Nonlinear Dynamical Systems.” Proceedings of the National Academy of Sciences.
Champneys, and Rogers. 2024. “BINDy – Bayesian Identification of Nonlinear Dynamics with Reversible-Jump Markov-Chain Monte-Carlo.”
Fung, Fasel, and Juniper. 2024. “Rapid Bayesian Identification of Sparse Nonlinear Dynamics from Scarce and Noisy Data.”
Hirsh, Barajas-Solano, and Kutz. 2022. “Sparsifying Priors for Bayesian Uncertainty Quantification in Model Discovery.” Royal Society Open Science.
Messenger, Tran, Dukic, et al. 2024. “The Weak Form Is Stronger Than You Think.”
More, Tripura, Nayek, et al. 2023. “A Bayesian Framework for Learning Governing Partial Differential Equation from Data.” Physica D: Nonlinear Phenomena.
Russo, Laiu, and Archibald. 2023. “Streaming Compression of Scientific Data via Weak-SINDy.”
Schmidt, and Lipson. 2009. “Distilling Free-Form Natural Laws from Experimental Data.” Science.
“Weak-Form Sparse Identification of Differential Equations from Noisy Measurements - ProQuest.” n.d.
Xu, Srivastava, Gutfreund, et al. 2021. “A Bayesian-Symbolic Approach to Reasoning and Learning in Intuitive Physics.” In Advances in Neural Information Processing Systems.