4 ms·
Sure, single high-quality studies can be better than a large number of low-quality studies. But a single study ultimately is just that, regardless of how well-d
by derbOac 3y ago
Sure, single high-quality studies can be better than a large number of low-quality studies. But a single study ultimately is just that, regardless of how well-done.
There's too much that can be hidden in a single study for me to trust them completely. Even when well intended, there's all sorts of unintentional biases or design problems that lead to potential issues.
I've seen too many "well-designed" studies fail on replication for me to trust them too much. For me the gold standard will always be adversarial replication.
Even small studies in aggregate can tell us something about what moves effects around.
Appeals to single gold standard studies tend to become appeals to or assertions of authority, which is a poor way to do rigorous science.
- HPsquared 3y agoIt all comes back to "you need to actually read the study".
- scythe 3y agoIt's nice to have a meta-analysis. I would usually prefer the meta-analysis for an effect size estimate. But I also generally live by the saying "you never see a clear picture of Bigfoot". If a phenomenon of serious interest is to be considered proven, there should be at least one conclusive high-quality study alongside any meta-analyses. As with many supposed dilemmas, it's nice to have both.
- anonymouskimmer 3y agoReplication is always very important. But for me the "gold standard study" is mechanistical, not statistical. It explains how a process putatively works, not that an effect was seen.