7 ms·

7 sigma is actually less than one thinks because these distributions are not normal distributions.

sdfgdf 5y ago
bawolff 5y ago
Why aren't they normal? I know very little about this topic, but i would generally assume that measuring most natural phenomenon would be normal.
- rich_sasha 5y ago
  Especially since, isn't this an average / error of a mean estimate? So even if individual observations are non-normal, this would be a perfect place for Central Limit Theorem. I know nothing about Quantum though, only maths.
  - freemint 5y ago
    Because it's a non-linear world? And the graphs seem very obviously skewed? And it's kurtosis also seems to differ from three?
    - rich_sasha 5y ago
      None of those things matter to the central limit theorem. If I have IID observations with finite 2nd moment (variance), then their average will pretty quickly converge to a Gaussian distribution. And I can relax a lot of this and still recover a variant of CLT. Of course maybe the calculation is different, eg it’s not like there are N independent observations, but rather some other complex condition solved for the mean estimate.
- krona 5y ago
  Also not knowing anything about this topic, I'd assume it wasn't normal because we're talking about mass close to zero, and mass must be greater than zero.
  - gus_massa 5y ago
    The mass in their result is 80433.5±9.4 MeV/c^2. The result of the experiment is a Gaussian like distribution. If you consider a Gaussian distribution with μ=80433.5 and σ=9.4, the probability to get a result that is less than 0 is 4E-15899105. I filled this widget https://www.wolframalpha.com/widgets/view.jsp?id=53fa34c5c669159296075c3c4ef3dc20 https://www.wolframalpha.com/widgets/view.jsp?id=53fa34c5c66... And got this result https://www.wolframalpha.com/input?i=mean%3D%5B%2F%2Fnumber%3A80433.5%2F%2F%5D%2C+standarddeviation%3D%5B%2F%2Fnumber%3A9.4%2F%2F%5D%2C+left+endpoint%3D%5B%2F%2Fnumber%3A0%2F%2F%5D%2C+right+endpoint%3D%5B%2F%2Fnumber%3A%2F%2F%5D https://www.wolframalpha.com/input?i=mean%3D%5B%2F%2Fnumber%... Note in the graphic that σ is 10000 smaller than μ so the probability to get a negative result is almost zero and you can just ignore it.
- cshimmin 5y ago
  Basically, the result of an experiment has to be boiled down to a single numerical value, called the test-statistic. Typically the test-statistic is a (log) likelihood ratio. It is the distribution of the t.s. that must be considered when determining the significance of a measurement. Obviously the measurement itself only gives you a single value of the t.s., so you need to know the distribution to ask "does this result seem significant?". This is done by considering all the factors of random variation (statistical and systematic) that could have an effect on the t.s. Often, the distributions of these individual random factors are assumed to be Normal, but the resulting distribution considering all of their conspiring effects is very seldom normal distribution. Even in the central limit theorem, I think the distribution of the LLR ends up being something like a noncentral chi^2 distribution.
ivad 5y ago
A measurement being 7 sigma out would still be Chebyshev bounded by 1/7^2 ≈ 0.02 I.e. the probability of it being ≥7 sigma out is interestingly at most 0.02.
- freemint 5y ago
  Neat i didn't think about that. But that is less improbable then 1 in 12450197393 which is what you might get with normal distribution.
  - lupire 5y ago
    That's just because Chevyshev bounds is a very weak general statement about all distributions. High Energy Physics sigma is calibrated to match normal distribution quantiles.
- lupire 5y ago
  That is not so interesting because it could be far less.
deleted 5y ago
[deleted]
cshimmin 5y ago
I work in this field (different experiment); that's not really true. In particle physics, sigma denotes "significance", not standard deviation. Technically what we're quoting as "sigmas" are "z-values", where z=Phi^{-1}(1 - p), where Phi^{-1} is the inverse CDF of the Normal distribution and p is the p-value of the experimental result. So, 7 sigma is defined to be the level of significance (for an arbitrary distribution) corresponding to the same quantile as 7 standard deviations out in a Normal distribution.
- Aachen 5y ago
  I'm one of those dumb people that didn't have much math or greek in school, so this weird-looking o in the title was quite literally Chinese to me. Now it turns out that people in the know also misunderstood its intended meaning because it's in a different field. For years I've argued foreign symbols and single-letter variable names mainly seem to serve to keep a walled garden around the sciences, and this was cemented when I eventually went for a master's degree and I was expected to do this as well in compsci to get a better grade even if there is no advantage. If we could just write what we mean, I suspect people would find that more useful even if it makes it look less cultivated and more mainstream. (To be clear, this is not criticism on the person I'm replying to, but split between the author of this specific title and most of the sciences as a whole because it's a universally supported barrier (if only ever implicitly), aside from a few science communicators.) Edit: scrolled further in the thread. Looks like I'm not the only one, though this person at least knew to name the sigma: https://news.ycombinator.com/item?id=30955621 https://news.ycombinator.com/item?id=30955621
  - LudwigNagasena 5y ago
    Notation is the easiest (and a very helpful) part of physics, statistics and probably all other scientific areas. That just sounds like an excuse.
    - Aachen 5y ago
      I've heard my complaint be called an excuse before, but consider that it's also the first barrier. Not a big one when you already decided to take a course, when you're seriously interested, when you just look them up and soon enough you know the conventions in the field. That's different from casual reading on HN, though. And tell me, are scientists not a class that is looked up to where you live? The common Joe might not explicitly say so, but if someone is a "scientist" then you don't expect them to be stupid or muck out stalls; they do have some real status. This doesn't come from doing things that seem like any common Joe can do it, yet a lot of the work is just that. Once you start paying attention to how often "new research" in the news amounted to a big survey and very basic statistics, or playing around with a Kinect in a train station to learn about walking patterns, it doesn't seem so different from what regular HN readers do for a living. If you take the paper behind such a survey, it'll turn out to be full of complexity that is a lot harder to get through than necessary. It could be a lot more accessible, but then they would lose status. It seems to me that brevity is the real excuse here. Moreover, if it were just about symbols but papers were otherwise accessibly written (to reasonable extents, obviously), that would be different still. This is not the case. Appearances are probably also important for funding. I'd bet that if you submitted same proposal twice, once phrased in a convoluted way and once phrased in a "we're gonna blow up some material multiple times and see how far the shards fly" style, a number of times to independent funding committees, there would be a statistically significant correlation with which proposal gets funded.

Why aren't they normal? I know very little about this topic, but i would generally assume that measuring most natural phenomenon would be normal.

rich_sasha 5y ago
Especially since, isn't this an average / error of a mean estimate? So even if individual observations are non-normal, this would be a perfect place for Central Limit Theorem. I know nothing about Quantum though, only maths.
- freemint 5y ago
  Because it's a non-linear world? And the graphs seem very obviously skewed? And it's kurtosis also seems to differ from three?
  - rich_sasha 5y ago
    None of those things matter to the central limit theorem. If I have IID observations with finite 2nd moment (variance), then their average will pretty quickly converge to a Gaussian distribution. And I can relax a lot of this and still recover a variant of CLT. Of course maybe the calculation is different, eg it’s not like there are N independent observations, but rather some other complex condition solved for the mean estimate.
krona 5y ago
Also not knowing anything about this topic, I'd assume it wasn't normal because we're talking about mass close to zero, and mass must be greater than zero.
- gus_massa 5y ago
  The mass in their result is 80433.5±9.4 MeV/c^2. The result of the experiment is a Gaussian like distribution. If you consider a Gaussian distribution with μ=80433.5 and σ=9.4, the probability to get a result that is less than 0 is 4E-15899105. I filled this widget https://www.wolframalpha.com/widgets/view.jsp?id=53fa34c5c669159296075c3c4ef3dc20 https://www.wolframalpha.com/widgets/view.jsp?id=53fa34c5c66... And got this result https://www.wolframalpha.com/input?i=mean%3D%5B%2F%2Fnumber%3A80433.5%2F%2F%5D%2C+standarddeviation%3D%5B%2F%2Fnumber%3A9.4%2F%2F%5D%2C+left+endpoint%3D%5B%2F%2Fnumber%3A0%2F%2F%5D%2C+right+endpoint%3D%5B%2F%2Fnumber%3A%2F%2F%5D https://www.wolframalpha.com/input?i=mean%3D%5B%2F%2Fnumber%... Note in the graphic that σ is 10000 smaller than μ so the probability to get a negative result is almost zero and you can just ignore it.
cshimmin 5y ago
Basically, the result of an experiment has to be boiled down to a single numerical value, called the test-statistic. Typically the test-statistic is a (log) likelihood ratio. It is the distribution of the t.s. that must be considered when determining the significance of a measurement. Obviously the measurement itself only gives you a single value of the t.s., so you need to know the distribution to ask "does this result seem significant?". This is done by considering all the factors of random variation (statistical and systematic) that could have an effect on the t.s. Often, the distributions of these individual random factors are assumed to be Normal, but the resulting distribution considering all of their conspiring effects is very seldom normal distribution. Even in the central limit theorem, I think the distribution of the LLR ends up being something like a noncentral chi^2 distribution.

ivad 5y ago

A measurement being 7 sigma out would still be Chebyshev bounded by 1/7^2 ≈ 0.02 I.e. the probability of it being ≥7 sigma out is interestingly at most 0.02.

freemint 5y ago
Neat i didn't think about that. But that is less improbable then 1 in 12450197393 which is what you might get with normal distribution.
- lupire 5y ago
  That's just because Chevyshev bounds is a very weak general statement about all distributions. High Energy Physics sigma is calibrated to match normal distribution quantiles.
lupire 5y ago
That is not so interesting because it could be far less.

deleted 5y ago

[deleted]

cshimmin 5y ago

I work in this field (different experiment); that's not really true. In particle physics, sigma denotes "significance", not standard deviation. Technically what we're quoting as "sigmas" are "z-values", where z=Phi^{-1}(1 - p), where Phi^{-1} is the inverse CDF of the Normal distribution and p is the p-value of the experimental result. So, 7 sigma is defined to be the level of significance (for an arbitrary distribution) corresponding to the same quantile as 7 standard deviations out in a Normal distribution.

Aachen 5y ago
I'm one of those dumb people that didn't have much math or greek in school, so this weird-looking o in the title was quite literally Chinese to me. Now it turns out that people in the know also misunderstood its intended meaning because it's in a different field. For years I've argued foreign symbols and single-letter variable names mainly seem to serve to keep a walled garden around the sciences, and this was cemented when I eventually went for a master's degree and I was expected to do this as well in compsci to get a better grade even if there is no advantage. If we could just write what we mean, I suspect people would find that more useful even if it makes it look less cultivated and more mainstream. (To be clear, this is not criticism on the person I'm replying to, but split between the author of this specific title and most of the sciences as a whole because it's a universally supported barrier (if only ever implicitly), aside from a few science communicators.) Edit: scrolled further in the thread. Looks like I'm not the only one, though this person at least knew to name the sigma: https://news.ycombinator.com/item?id=30955621 https://news.ycombinator.com/item?id=30955621
- LudwigNagasena 5y ago
  Notation is the easiest (and a very helpful) part of physics, statistics and probably all other scientific areas. That just sounds like an excuse.
  - Aachen 5y ago
    I've heard my complaint be called an excuse before, but consider that it's also the first barrier. Not a big one when you already decided to take a course, when you're seriously interested, when you just look them up and soon enough you know the conventions in the field. That's different from casual reading on HN, though. And tell me, are scientists not a class that is looked up to where you live? The common Joe might not explicitly say so, but if someone is a "scientist" then you don't expect them to be stupid or muck out stalls; they do have some real status. This doesn't come from doing things that seem like any common Joe can do it, yet a lot of the work is just that. Once you start paying attention to how often "new research" in the news amounted to a big survey and very basic statistics, or playing around with a Kinect in a train station to learn about walking patterns, it doesn't seem so different from what regular HN readers do for a living. If you take the paper behind such a survey, it'll turn out to be full of complexity that is a lot harder to get through than necessary. It could be a lot more accessible, but then they would lose status. It seems to me that brevity is the real excuse here. Moreover, if it were just about symbols but papers were otherwise accessibly written (to reasonable extents, obviously), that would be different still. This is not the case. Appearances are probably also important for funding. I'd bet that if you submitted same proposal twice, once phrased in a convoluted way and once phrased in a "we're gonna blow up some material multiple times and see how far the shards fly" style, a number of times to independent funding committees, there would be a statistically significant correlation with which proposal gets funded.