6 ms·

Are you sure? I'm pretty confident in this case the assumption is that the distribution is normal. With no knowledge of the shape of distribution, the 5 sigma b

by sigterm 13y ago

Are you sure? I'm pretty confident in this case the assumption is that the distribution is normal. With no knowledge of the shape of distribution, the 5 sigma bound is extremely weak (one given by Chebyshev's inequality [0], in this case, ~4%). Prove me wrong though, as I'm not a physicist or mathematician. [0] http://en.wikipedia.org/wiki/Chebyshev's_inequality#Sharpness_of_bounds http://en.wikipedia.org/wiki/Chebyshev's_inequality#Sharpnes...

lutusp 13y ago
You're correct. Here's a short article that makes the point that what's being discussed are areas of the normal distribution: http://blogs.scientificamerican.com/observations/2012/07/17/five-sigmawhats-that/ http://blogs.scientificamerican.com/observations/2012/07/17/...
- Myrmornis 13y ago
  No, I think you're wrong. See my comment above. Note that nowhere in the article you link does it say there's an assumption of normality. Doubtless the journalist or whoever chose the picture of th normal distribution was a bit confused also. Edit: oh, no, actually I think you're right :). I thought it was referring to the location of the test statistic in its null distribution. But it seems it's a scale for measuring p-values. This explains it clearly: What does "five sigma" mean? It means that the results would occur by chance alone as rarely as a value sampled from a Gaussian distribution would be five standard deviations from the mean. http://www.graphpad.com/www/data-analysis-resource-center/blog/statistical-significance-defined-using-the-five-sigma-standard/ http://www.graphpad.com/www/data-analysis-resource-center/bl...
  - lutusp 13y ago
    All that aside, remember that the context is a nontechnical person asking what 5σ means in statistical analysis in physics. That greatly narrows the possible interpretations. By the way, this same conversation took place after the LHC Higgs anouncement -- the same five-sigma standard for discovery, and the same detailed discussion of what that means. > But it seems it's a scale for measuring p-values. Yes, and that was a point I made in my reply to the OP -- that the context assumed an association with p-values, which in turn assume a normal distribution.
    - Myrmornis 13y ago
      Sorry, I'm either not understanding or I still am not convinced you are correct. p-values don't assume a normal distribution. A p-value is the probability of generating an equal or more extreme value of your test statistic under the null hypothesis. Normal distributions are nowhere in sight. The only way that the Normal distribution comes into play is that physicists are measuring the smallness of their p-values by stating a number of standard deviations from the Normal mean that would have the same p-value. I'm finding this discussion helpful by the way. I have worked in applied statistics but not in any fields which use this "sigma" scale or would talk about "sigma values".
      - lutusp 13y ago
        > p-values don't assume a normal distribution. No, not unless a p-value is expressed in terms of sigma as in this case and similar ones. In this case, and commonly in experimental physics, there's a relationship between n-sigma (usually 3σ or 5σ in different circumstances) and how a p-value is acquired from a sigma expression. The p-value is acquired from a sigma value like this: http://i.imgur.com/pcjr6cN.gif http://i.imgur.com/pcjr6cN.gif My point? In experimental physics there's a connection between (a) an expression including an integer and "sigma", (b) a resulting, widely quoted numerical value, and (c) the method for converting one to the other, using a Gaussian distribution as shown. http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-discovery.html http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-d... Quote: "But what does a 5-sigma result mean, and why do particle physicists use this as a benchmark for discoveries? To answer these questions, we'll have to look at one of the statistician's oldest friends and C-student's worst enemies: the normal distribution or bell curve." Couldn't have said it better myself. > The only way that the Normal distribution comes into play is that physicists are measuring the smallness of their p-values by stating a number of standard deviations from the Normal mean that would have the same p-value. Hmm. Yes, that's right. That's why I replied as I did in my original post.
- mendicantB 13y ago
  It's correct that a normal distribution is assumed in physics, but you are not correct that a standard deviation is defined in terms of a normal distribution.
  - Myrmornis 13y ago
    Nit: I don't think the physicists are assuming a Gaussian distribution. See my reply in this thread https://news.ycombinator.com/item?id=7422688 https://news.ycombinator.com/item?id=7422688
  - lutusp 13y ago
    The present conversation got started by someone non-technical asking what 5 sigma meant in a physics context. Within that context, and with the intent to avoid exotic digressions, my reply is correct. In physics, a sigma value maps to a p-value, and that relationship is most often defined with respect to a normal distribution. Therefore, in most cases, to go from a sigma value to a p-value, one performs this integral: http://i.imgur.com/YhC302b.gif http://i.imgur.com/YhC302b.gif Specifically, the above definite integral, when performed with arguments of 5 and +oo, yields the often-quoted one-tailed p-value for "5 sigma", which we can get here as well: https://www.wolframalpha.com/input/?i=5+sigma https://www.wolframalpha.com/input/?i=5+sigma It seems Wolfram Alpha makes the same default assumption I do: a normal distribution. If I weren't answering an inquiry from someone who wanted the clearest possible answer, I might have replied differently.
graycat 13y ago
> Are you sure? I'm absolutely correct in what I said. But it appears that you are correct that that part of physics makes some Gaussian assumptions and, then, has some conventions based on those assumptions. In this case, apparently physics is not making its mathematical assumptions clear and explicit and is doing sloppy writing. For more detail, see my longer explanation in this thread https://news.ycombinator.com/item?id=7421505 https://news.ycombinator.com/item?id=7421505 Maybe the situation is a little like getting from a little French restaurant the recipe for French salad dressing, sauce vinaigrette, making it at home, and concluding it tasted better in the little French restaurant. Hmm .... But the French restaurant did something not in the recipe -- took a large clove of garlic, peeled it, cut it in half, and wiped the salad bowl with the cut surface of the garlic! The recipe didn't mention that!
- Myrmornis 13y ago
  I don't think those physicists are making any Gaussian assumptions, right? They're computing a p-value as normal (how often would my test statistic generated under the null be more extreme than observed), and then using the Gaussian distribution as a scale to communicate how small their p-value is.
- gammarator 13y ago
  They are definitely making a Gaussian assumption, and wouldn't feel the need to clarify that as it's a widely accepted convention in the field. In your restaurant analogy, it's like you overheard two chefs talking.
mendicantB 13y ago
He's correct. The poster he was responding to made an idiotic statement: "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)." He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one.
- lutusp 13y ago
  > The poster he was responding to made an idiotic statement: > "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)." Not only is that not idiotic, that's the default definition in a statistical context (see below). Anyone can argue that σ is just another Greek letter with no special significance, but that require one to ignore the context in which the term is used. Link: http://en.wikipedia.org/wiki/Standard_deviation http://en.wikipedia.org/wiki/Standard_deviation Quote: "In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists." > He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one. Again, this disregards context. When nontechnical people ask what the significance of 5σ is to scientific statistical analysis in physics (which is how this thread got started), there is precisely one answer.
  - mendicantB 13y ago
    You disproved your own point. In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation). In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists. I think you're missing the difference.
    - lutusp 13y ago
      If the unit normal distribution is the distribution under discussion, there's no difference. In that case they become the same. Here's how a physicist computes a p-value given a sigma value: http://i.imgur.com/pcjr6cN.gif http://i.imgur.com/pcjr6cN.gif Note that the unit normal distribution is the underlying context.

by sigterm 13y ago

lutusp 13y ago

You're correct. Here's a short article that makes the point that what's being discussed are areas of the normal distribution: http://blogs.scientificamerican.com/observations/2012/07/17/five-sigmawhats-that/ http://blogs.scientificamerican.com/observations/2012/07/17/...

Myrmornis 13y ago
No, I think you're wrong. See my comment above. Note that nowhere in the article you link does it say there's an assumption of normality. Doubtless the journalist or whoever chose the picture of th normal distribution was a bit confused also. Edit: oh, no, actually I think you're right :). I thought it was referring to the location of the test statistic in its null distribution. But it seems it's a scale for measuring p-values. This explains it clearly: What does "five sigma" mean? It means that the results would occur by chance alone as rarely as a value sampled from a Gaussian distribution would be five standard deviations from the mean. http://www.graphpad.com/www/data-analysis-resource-center/blog/statistical-significance-defined-using-the-five-sigma-standard/ http://www.graphpad.com/www/data-analysis-resource-center/bl...
- lutusp 13y ago
  All that aside, remember that the context is a nontechnical person asking what 5σ means in statistical analysis in physics. That greatly narrows the possible interpretations. By the way, this same conversation took place after the LHC Higgs anouncement -- the same five-sigma standard for discovery, and the same detailed discussion of what that means. > But it seems it's a scale for measuring p-values. Yes, and that was a point I made in my reply to the OP -- that the context assumed an association with p-values, which in turn assume a normal distribution.
  - Myrmornis 13y ago
    Sorry, I'm either not understanding or I still am not convinced you are correct. p-values don't assume a normal distribution. A p-value is the probability of generating an equal or more extreme value of your test statistic under the null hypothesis. Normal distributions are nowhere in sight. The only way that the Normal distribution comes into play is that physicists are measuring the smallness of their p-values by stating a number of standard deviations from the Normal mean that would have the same p-value. I'm finding this discussion helpful by the way. I have worked in applied statistics but not in any fields which use this "sigma" scale or would talk about "sigma values".
    - lutusp 13y ago
      > p-values don't assume a normal distribution. No, not unless a p-value is expressed in terms of sigma as in this case and similar ones. In this case, and commonly in experimental physics, there's a relationship between n-sigma (usually 3σ or 5σ in different circumstances) and how a p-value is acquired from a sigma expression. The p-value is acquired from a sigma value like this: http://i.imgur.com/pcjr6cN.gif http://i.imgur.com/pcjr6cN.gif My point? In experimental physics there's a connection between (a) an expression including an integer and "sigma", (b) a resulting, widely quoted numerical value, and (c) the method for converting one to the other, using a Gaussian distribution as shown. http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-discovery.html http://physicsbuzz.physicscentral.com/2012/07/does-5-sigma-d... Quote: "But what does a 5-sigma result mean, and why do particle physicists use this as a benchmark for discoveries? To answer these questions, we'll have to look at one of the statistician's oldest friends and C-student's worst enemies: the normal distribution or bell curve." Couldn't have said it better myself. > The only way that the Normal distribution comes into play is that physicists are measuring the smallness of their p-values by stating a number of standard deviations from the Normal mean that would have the same p-value. Hmm. Yes, that's right. That's why I replied as I did in my original post.
mendicantB 13y ago
It's correct that a normal distribution is assumed in physics, but you are not correct that a standard deviation is defined in terms of a normal distribution.
- Myrmornis 13y ago
  Nit: I don't think the physicists are assuming a Gaussian distribution. See my reply in this thread https://news.ycombinator.com/item?id=7422688 https://news.ycombinator.com/item?id=7422688
- lutusp 13y ago
  The present conversation got started by someone non-technical asking what 5 sigma meant in a physics context. Within that context, and with the intent to avoid exotic digressions, my reply is correct. In physics, a sigma value maps to a p-value, and that relationship is most often defined with respect to a normal distribution. Therefore, in most cases, to go from a sigma value to a p-value, one performs this integral: http://i.imgur.com/YhC302b.gif http://i.imgur.com/YhC302b.gif Specifically, the above definite integral, when performed with arguments of 5 and +oo, yields the often-quoted one-tailed p-value for "5 sigma", which we can get here as well: https://www.wolframalpha.com/input/?i=5+sigma https://www.wolframalpha.com/input/?i=5+sigma It seems Wolfram Alpha makes the same default assumption I do: a normal distribution. If I weren't answering an inquiry from someone who wanted the clearest possible answer, I might have replied differently.

graycat 13y ago

> Are you sure? I'm absolutely correct in what I said. But it appears that you are correct that that part of physics makes some Gaussian assumptions and, then, has some conventions based on those assumptions. In this case, apparently physics is not making its mathematical assumptions clear and explicit and is doing sloppy writing. For more detail, see my longer explanation in this thread https://news.ycombinator.com/item?id=7421505 https://news.ycombinator.com/item?id=7421505 Maybe the situation is a little like getting from a little French restaurant the recipe for French salad dressing, sauce vinaigrette, making it at home, and concluding it tasted better in the little French restaurant. Hmm .... But the French restaurant did something not in the recipe -- took a large clove of garlic, peeled it, cut it in half, and wiped the salad bowl with the cut surface of the garlic! The recipe didn't mention that!

Myrmornis 13y ago
I don't think those physicists are making any Gaussian assumptions, right? They're computing a p-value as normal (how often would my test statistic generated under the null be more extreme than observed), and then using the Gaussian distribution as a scale to communicate how small their p-value is.
gammarator 13y ago
They are definitely making a Gaussian assumption, and wouldn't feel the need to clarify that as it's a widely accepted convention in the field. In your restaurant analogy, it's like you overheard two chefs talking.

mendicantB 13y ago

He's correct. The poster he was responding to made an idiotic statement: "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)." He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one.

lutusp 13y ago
> The poster he was responding to made an idiotic statement: > "In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation)." Not only is that not idiotic, that's the default definition in a statistical context (see below). Anyone can argue that σ is just another Greek letter with no special significance, but that require one to ignore the context in which the term is used. Link: http://en.wikipedia.org/wiki/Standard_deviation http://en.wikipedia.org/wiki/Standard_deviation Quote: "In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists." > He was just stating that the standard deviation is a general concept of spread, that any distribution has, and not just the normal one. Again, this disregards context. When nontechnical people ask what the significance of 5σ is to scientific statistical analysis in physics (which is how this thread got started), there is precisely one answer.
- mendicantB 13y ago
  You disproved your own point. In statistics, σ refers to an area under the normal distribution defined in terms of standard deviations (1σ = 1 standard deviation). In statistics and probability theory, the standard deviation (SD) (represented by the Greek letter sigma, σ) shows how much variation or dispersion from the average exists. I think you're missing the difference.
  - lutusp 13y ago
    If the unit normal distribution is the distribution under discussion, there's no difference. In that case they become the same. Here's how a physicist computes a p-value given a sigma value: http://i.imgur.com/pcjr6cN.gif http://i.imgur.com/pcjr6cN.gif Note that the unit normal distribution is the underlying context.