Elliptical distributions

Generalising multivariate Gaussians to anything that has a density function of the form \[ f(x)\propto g((x-\mu )'\Sigma ^{-1}(x-\mu ))\] where \(\mu\) is the mean vector, \(\Sigma\) is a positive definite matrix, and \(g:\mathbb{R}^+\to\mathbb{R}^+\). In fact, we do not need the density function to exist; it’s ok if \(\Sigma\) is positive semi-definite or to allow \(g\) to be a generalised function.

Baby steps, though; let us have densities for now. If the mean of such an \(X\sim f\) RV exists, it is \(\mu\), and \(\Sigma\) is proportional to the covariance matrix of \(X\), if such a covariance matrix exists.

I assume they did not invent this idea, but Davison and Ortiz (2019) points out that if you have a least-squares-compatible model, usually it can generalize to any elliptical density, which includes many M-estimator-style robust losses.