5 ms·
IMHO, a very confusing thing about Linear Regression is that there's widespread disagreement about whether the data are supposed to be Normally Distributed or n
by hackaflocka 11y ago
IMHO, a very confusing thing about Linear Regression is that there's widespread disagreement about whether the data are supposed to be Normally Distributed or not, and on how to measure said Normality.
- fats_tromino 11y agoLook at normal qq-plot to measure normality
- fats_tromino 11y agoAlso some features of regression and hypothesis testing are actually fairly robust to non-normality, particularly with large sample sizes.
- stdbrouw 11y agoYou're right, there is lots of confusion in the topic and not all texts on regression get it right. Theoretically, linear regression depends only on a normally distributed outcome variable, the predictors don't have to be normally distributed. Practically speaking even that requirement can mostly be ignored: OLS is fairly insensitive to non-normal outcomes. Violations of the assumptions of regression usually don't affect the estimates much, they mainly affect the uncertainty around those parameter estimates -- the standard errors. This too is easily solved: use bootstrapping to calculate the standard errors. In short: you can't just throw linear regression at anything and expect to get reasonable results, but it's pretty damn close.
- hooloovoo_zoo 11y agoEven that normality is not required; see, for instance, https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem.
- stdbrouw 11y agoGood catch. Outcome (or error, which boils down to the same thing) can't just be anything either, though, e.g. a common pattern is that the variance in an outcome increases when the predictor increases (e.g. a machine behaves more erratically when it spins faster) and that can mess with estimates.