3 ms·
> Let's say your data really does fit a straight line, but the later data points you get are really high variance. Intuitively, including the later data points
by em500 7y ago
> Let's say your data really does fit a straight line, but the later data points you get are really high variance. Intuitively, including the later data points will make your estimated parameters more noisy than if we had simply thrown them out.
> So in this sense, 'more data is worse', because your model assumes all the data points are the same, but the later points you collected are actually worse (noisier).
The first example in the paper makes it less-than-obvious by writing in formula form that the error noise variance increases with time, but what's described is essentially this picture: https://en.wikipedia.org/wiki/Heteroscedasticity#/media/File:Hsked_residual_compare.svg https://en.wikipedia.org/wiki/Heteroscedasticity#/media/File...
So it's intuitively easy to grok that if the more recent variances are bad enough it's probably better not to use them at all (or to properly downweight them like with MLE or GLS).
The described result is a bit more subtle though. The classical results are that with heteroskedasticity, OLS is not an efficient estimator [1] (you could do better with for example GLS/WLS), but it's still unbiased [2] (the estimated beta's are centered around the true beta's) and consistent [3] (the probability limit of the OLS beta is the true beta by the Law of Large Numbers). These are what the paper calls "robustness" of OLS in the first example. The consistency of OLS even with heteroskedastic errors is what makes it surprising that adding more (true!) observations can make your estimates worse. Intuitively most people expect consistent estimators to get better with more observations.
[1] https://en.wikipedia.org/wiki/Efficient_estimator https://en.wikipedia.org/wiki/Efficient_estimator
[2] https://en.wikipedia.org/wiki/Bias_of_an_estimator https://en.wikipedia.org/wiki/Bias_of_an_estimator
[3] https://en.wikipedia.org/wiki/Consistent_estimator https://en.wikipedia.org/wiki/Consistent_estimator