4 ms·
Right, after the bump reaches 3x sigma they have some confidence it deserves their own attention, and at 4x they are sure, but the rule is you don't publish unt
by ncmncm 4y ago
Right, after the bump reaches 3x sigma they have some confidence it deserves their own attention, and at 4x they are sure, but the rule is you don't publish until you have enough data for a 5x sigma result, which just takes a lot more data and a lot longer.
"Sigma" here refers to standard deviations off the Gaussian normal mean. Zero means completely random. In psychology they publish at 2x sigma, 95%, which means 20:1 odds against a spurious result, and they publish a lot of spurious results because you can generate an unlimited number of hypotheses. In physics, things are considered more deterministic, and an experiment doesn't need to recruit undergrads to be data points, so you run your LHC for a few more months and avoid wasting people's attention.
- huijzer 4y agoThe chance of falsely rejecting the null hypothesis increases as you gather more data. Put more simply, finding something that differs "significantly" from some distribution becomes easier as you gather more data. Imagine having only 3 psychology student in a study, the required effect size has to be huge for the test to say that it is significantly different. However, the approach taken by CERN is of course right. They find a result at a certain significance level and then collect more data to verify the result. As long as there aren’t thousands of simultaneous verifications running, this approach is sound. Obviously yes, physicist’s know what they’re doing. Having said that, please don’t read this comment as me approving of frequentists statistics. Bayesian or cross-validations are way easier to interpret where possible.
- ncmncm 4y agoI would never accuse you of frequentism. But if you and a frequentist ever get different answers, one of you just made a booboo.