3 ms·
I find it helpful to view least as fitting the noise to a Gaussian distribution.
by ryang2718 9mo ago
I find it helpful to view least as fitting the noise to a Gaussian distribution.
- LudwigNagasena 9mo agoOLS estimator is the minimum-variance linear unbiased estimator even without the assumption of Gaussian distribution.
- rjdj377dhabsn 9mo agoYes, and if I remember correctly, you get the Gaussian because it's the minimum entropy (least additional assumptions about the shape) continuous distribution given a certain variance.
- porridgeraisin 9mo agoAnd given a mean.
- contravariant 9mo agoBoth of these do, in a way. They just differ in which gaussian distribution they're fitting to. And how I suppose. PCA is effectively moment matching, least squares is max likelihood. These correspond to the two ways of minimizing the Kullback Leibler divergence to or from a gaussian distribution.
- MontyCarloHall 9mo agoThey both fit Gaussians, just different ones! OLS fits a 1D Gaussian to the set of errors in the y coordinates only, whereas TLS (PCA) fits a 2D Gaussian to the set of all (x,y) pairs.
- ryang2718 9mo agoWell, that was a knowledge gap, thank you! I certainly need to review PCA but python makes it a bit too easy.