3 ms·

CMS observes the peak with a global significance of 1.2σ http://cds.cern.ch/record/2114808 http://cds.cern.ch/record/2114808 ATLAS observes the peak with a glo

by dukwon 11y ago

CMS observes the peak with a global significance of 1.2σ http://cds.cern.ch/record/2114808 http://cds.cern.ch/record/2114808 ATLAS observes the peak with a global significance of 2σ http://cds.cern.ch/record/2114853 http://cds.cern.ch/record/2114853 5σ is the gold standard for a discovery

pc86 11y ago
Is it true that 2σ holds some significance? I imagine if it was common to get erroneous 2σ or 3σ readings this wouldn't have made the news in the way that it did. Just trying to determine if this is closer to "this is probably something but we will need to confirm" or "something weird happened but it's probably nothing"
- dukwon 11y ago
  Let's put it this way: the LHC has on the order 2000 results so far, assuming the null hypothesis, you expect about 100 >2σ deviations and about 5 that are >3σ. The combined significance from ATLAS and CMS will be higher, but it's quite an involved process to get all the systematics right and get approval from both collaborations.
madaxe_again 11y ago
Although in "real" terms 2σ is still pretty significant - but not firm enough to go "yes, absolutely" - but if they've got 2σ it's unlikely to be just noise, albeit still perfectly possible. IIRC 2σ is about a 1/20 chance of being erroneous. 1σ is almost meaningless - 2/5 chance of being erroneous.
- jessaustin 11y ago
  1/20? So if 20 tests were performed at the facility, we'd expect at least one of these to appear spuriously? Presumably there have been 20 tests performed there?
  - saulrh 11y ago
    This failure mode is correct and very real. It's actually one of the reasons why we're starting to see significant pushback against p-values. https://xkcd.com/882/ https://xkcd.com/882/
  - deleted 11y ago
    [deleted]
- daveguy 11y ago
  This is a helpful table for converting sigma levels to percents and fractions (although those you listed are pretty good rules of thumb): https://en.m.wikipedia.org/wiki/68–95–99.7_rule#Table_of_numerical_values https://en.m.wikipedia.org/wiki/68–95–99.7_rule#Table_of_num...
- goodcanadian 11y ago
  The significance of 2σ really depends on the context. If we are looking for something at a particular place in the energy spectrum and we see a 2σ detection in the right place . . . well, that is tantalizing but not convincing. However, in this case, we are looking for unexpected signals across a whole range of energies. By sheer random chance, there will be a number of 2σ peaks that are meaningless noise. It gets a little more interesting if we see weak detections in two different experiments because the combined likelihood of two experiments having random peaks at the same place is considerably smaller. However, 1.2σ is nothing but noise, so I would say nothing to see here . . . at least, not yet.
  - dukwon 11y ago
    The look-elsewhere effect is already taken into account. This is what is meant by "global".
    - goodcanadian 11y ago
      Hmm . . . you did write that; I missed that. That brings the result up to tantalizing, but not convincing.

by dukwon 11y ago

pc86 11y ago

Is it true that 2σ holds some significance? I imagine if it was common to get erroneous 2σ or 3σ readings this wouldn't have made the news in the way that it did. Just trying to determine if this is closer to "this is probably something but we will need to confirm" or "something weird happened but it's probably nothing"

dukwon 11y ago
Let's put it this way: the LHC has on the order 2000 results so far, assuming the null hypothesis, you expect about 100 >2σ deviations and about 5 that are >3σ. The combined significance from ATLAS and CMS will be higher, but it's quite an involved process to get all the systematics right and get approval from both collaborations.

madaxe_again 11y ago

Although in "real" terms 2σ is still pretty significant - but not firm enough to go "yes, absolutely" - but if they've got 2σ it's unlikely to be just noise, albeit still perfectly possible. IIRC 2σ is about a 1/20 chance of being erroneous. 1σ is almost meaningless - 2/5 chance of being erroneous.

jessaustin 11y ago
1/20? So if 20 tests were performed at the facility, we'd expect at least one of these to appear spuriously? Presumably there have been 20 tests performed there?
- saulrh 11y ago
  This failure mode is correct and very real. It's actually one of the reasons why we're starting to see significant pushback against p-values. https://xkcd.com/882/ https://xkcd.com/882/
- deleted 11y ago
  [deleted]
daveguy 11y ago
This is a helpful table for converting sigma levels to percents and fractions (although those you listed are pretty good rules of thumb): https://en.m.wikipedia.org/wiki/68–95–99.7_rule#Table_of_numerical_values https://en.m.wikipedia.org/wiki/68–95–99.7_rule#Table_of_num...
goodcanadian 11y ago
The significance of 2σ really depends on the context. If we are looking for something at a particular place in the energy spectrum and we see a 2σ detection in the right place . . . well, that is tantalizing but not convincing. However, in this case, we are looking for unexpected signals across a whole range of energies. By sheer random chance, there will be a number of 2σ peaks that are meaningless noise. It gets a little more interesting if we see weak detections in two different experiments because the combined likelihood of two experiments having random peaks at the same place is considerably smaller. However, 1.2σ is nothing but noise, so I would say nothing to see here . . . at least, not yet.
- dukwon 11y ago
  The look-elsewhere effect is already taken into account. This is what is meant by "global".
  - goodcanadian 11y ago
    Hmm . . . you did write that; I missed that. That brings the result up to tantalizing, but not convincing.