3 ms·
This doesn't seem like quite the same issue as the reproducibility problems in psychology, for instance. Researchers were presented with some data, and they ca
by mcphage 3y ago
This doesn't seem like quite the same issue as the reproducibility problems in psychology, for instance. Researchers were presented with some data, and they came up with different analyses for it. That doesn't seem especially surprising—for a complex phenomenon, where you don't have a ton of data, and maybe the thing you're studying isn't very strong—yeah, you're going to get different results.
It's like people arguing over which sports team is better—the facts of the game are the facts of the game, but there's plenty of room for disagreement over what stats are most important, who has an easier season—and even if the two teams were to play each other directly, it's not always that the "better" team wins.
But the reproducibility crisis in social sciences seemed much deeper—one team would say that some effect exists, and other teams would try to reproduce the results and come up with nothing.
- tgv 3y agoThere is a relation between them, but not a strong one, I agree. The more conclusions you can draw from a data set, the more will be wrong. A possible reason, hinted at in the article, is underestimating the noise. It's one of the things that went bad in psych studies, but also in genetics (in a lecture some 10 years ago, a statistician explained how results in genetics would be reproducible only at 5 sigma or higher).