3 ms·

That's insanely accurate. For comparison most discoveries are usually only considered proven when it's 5 sigma. For example they waited years to announce the H

by obedm 5y ago

That's insanely accurate. For comparison most discoveries are usually only considered proven when it's 5 sigma. For example they waited years to announce the Higgs bosson because they wanted to reach 5 sigma.

guerrilla 5y ago
Thanks, that's what I thought. I was doubting it was even the same metric :)
gbrown 5y ago
> That's insanely accurate. This is misleading, and it's a misconception arising from the way that even physicists sometimes misspeak when talking about statistics. The "sigma" is a reference to a standard deviation, and we use it to measure evidence against a null hypothesis (which is generally defined relative to a model of some type). An 11 sigma result means that there's strong evidence against (hypothesis + model), not anything in absolute terms about "accuracy". The meaning of such a result depends on the model being used; sometimes it's as simple and easy to defend as there being some sort of distribution with a mean which exists, and other times there's a complicated model which may itself be incorrect (producing significant results).
jjoonathan 5y ago
You can trivially get to 11 sigma by making your null hypothesis sufficiently garbage. That's called "p-hacking" and is unfortunately very common. It is encouraged whenever someone innocently reads meaning into a sigma value without evaluating whether or not the null hypothesis is any good. If I saw someone going around saying "The sky is reddish green, 11 sigma result!" my first suspicion would not be that their numbers were incorrect but rather that their null hypothesis was something silly like "the sky is entirely the solid sRGB color #0000FF." Obviously this garbage null hypothesis can be rejected with enormous confidence, that's where the 11 sigma came from, but the confident rejection of a garbage null hypothesis should not be taken to support their proposed alternative theory unless you have reason to believe the null hypothesis was well modeled, either because you checked it or because someone else did. It all comes back to trust, in other words.

by obedm 5y ago

guerrilla 5y ago

Thanks, that's what I thought. I was doubting it was even the same metric :)

gbrown 5y ago

> That's insanely accurate. This is misleading, and it's a misconception arising from the way that even physicists sometimes misspeak when talking about statistics. The "sigma" is a reference to a standard deviation, and we use it to measure evidence against a null hypothesis (which is generally defined relative to a model of some type). An 11 sigma result means that there's strong evidence against (hypothesis + model), not anything in absolute terms about "accuracy". The meaning of such a result depends on the model being used; sometimes it's as simple and easy to defend as there being some sort of distribution with a mean which exists, and other times there's a complicated model which may itself be incorrect (producing significant results).

jjoonathan 5y ago

You can trivially get to 11 sigma by making your null hypothesis sufficiently garbage. That's called "p-hacking" and is unfortunately very common. It is encouraged whenever someone innocently reads meaning into a sigma value without evaluating whether or not the null hypothesis is any good. If I saw someone going around saying "The sky is reddish green, 11 sigma result!" my first suspicion would not be that their numbers were incorrect but rather that their null hypothesis was something silly like "the sky is entirely the solid sRGB color #0000FF." Obviously this garbage null hypothesis can be rejected with enormous confidence, that's where the 11 sigma came from, but the confident rejection of a garbage null hypothesis should not be taken to support their proposed alternative theory unless you have reason to believe the null hypothesis was well modeled, either because you checked it or because someone else did. It all comes back to trust, in other words.