4 ms·
Particle physicists generate data by the petabyte. If they used 3 sigma confidence they'd confidently declare all sorts of false things. We know this, because i
by jerf 15y ago
Particle physicists generate data by the petabyte. If they used 3 sigma confidence they'd confidently declare all sorts of false things. We know this, because if you pay attention to particle physics you hear about all kinds of 3 sigmas that turn out to be spurious. (About every six months to a year or so, I'd say.) 5 is just where they start to feel comfortable, they'll take more if they can get it.
http://www.nature.com/news/2011/110119/full/469282a.html http://www.nature.com/news/2011/110119/full/469282a.html
- carbocation 15y agoYour answer is good and sufficient. For people who aren't yet getting an intuitive feel: if 99% confidence means you're right 99% of the time (it doesn't quite mean that but lets go with it), then you'll still be wrong 1% of the time. If 3000 studies are published each year with that cutoff for significance, you should expect about 30 spurious results accepted as true each year. Ouch.
- Estragon 15y agoThat reasoning neglects the bias towards papers with interesting results, which leads to a much higher rate of false reports in hot journals.
- carbocation 15y agoIn human genetics, studies will meta-analyze each of the ~million SNPs tested across each study, rather than just picking the "candidate" SNPs that came to attention because of their association signal in any particular study. So the file drawer problem is still relevant, but the risk is greatly mitigated.