4 ms·
One of the issues with problems of reproducibility in the social sciences/psychology is that early studies usually choose WEIRD (white/educated from a rich/indu
by a_bonobo 8y ago
One of the issues with problems of reproducibility in the social sciences/psychology is that early studies usually choose WEIRD (white/educated from a rich/industrialised/democractic country) subjects, which are often uni students who are very different from the rest of the world.
One article: https://slate.com/technology/2013/05/weird-psychology-social-science-researchers-rely-too-much-on-western-college-students.html https://slate.com/technology/2013/05/weird-psychology-social...
Could you not interpret this, in the widest sense, as overfitting interpretations to a 'weird' dataset, with the results that do not generalize, even though the stats (in this case, likely t-test) say everything is fine? In which case overfitting isn't a ML-only problem?
- rocqua 8y agoOverfitting is about reading too much in a properly sampled data set from any distribution. It comes about when your model has so many parameters it can be made to fit anything. Regularization keeps it in check, but it remains an issue. The example you cite has an issue earlier in the chain. Here, we are dealing with a biased sample from some distrubution. The results from that won't generalize to the unbiased distrubution, no matter how great your model is.