3 ms·
This debate isn't that simple. The other side of this isn't that everything magically just gets more accurate. The cost will be withholding true discoveries fro
by feral 7y ago
This debate isn't that simple. The other side of this isn't that everything magically just gets more accurate. The cost will be withholding true discoveries from the medical community for longer.
Imagine you've done a study of something else and you accidentally discover a (properly FDR controlled) correlation between some drug and heart attacks, significant at 0.03 and with no big contrary prior.
Should you publish or wait 5 years for a follow-up study to complete?
What if three different groups stumble across this same finding? Should they all publish and maybe someone will realize 'shit this drug is killing people' or should they all wait to hit some other higher standard of significance?
The point is theres always a balance between specificity and sensitivity, a tradeoff in terms of costs.
I'd personally be happy with keeping 0.05 as the threshold for 'probably something here'. The real issue are about publication bias, incentives, naive interpretation of published work 'heres one study in Nature so it must be true'. I don't see any other purely statistical change, Bayesian (essentially has all the same problems, aside from taking into account priors) or whatever, that will solve these in any way that won't come without unacceptable sensitivity cost.