4 ms·

Yet another example of the confusion in popular science reporting between the probability of A given B with the probability of B given A. How is this acceptabl

by DavidSJ 15y ago

Yet another example of the confusion in popular science reporting between the probability of A given B with the probability of B given A. How is this acceptable? the Tevatron would be able to identify the Higgs with 'three-sigma' certainty. This is a statistical term that indicates the finding only has a tenth of a percent chance of being due to a random statistical fluctuation. No no no! It actually means that random statistical fluctuation has a tenth of a percent chance of producing the finding!

extension 15y ago
What's the difference?
- DavidSJ 15y ago
  It's the same as the difference between "0.1% of non-spam messages are marked as spam" (false positive rate) vs. "0.1% of messages marked as spam are not spam" (false negative rate).
  - prodigal_erik 15y ago
    Those seem like the same false positive case to me. For a false negative, did you mean "0.1% of spam messages are not marked as spam"?
    - DavidSJ 15y ago
      Whoops, you're right, they're both false positives, but one is number of false positives as a percentage of messages marked as spam, and the other is number of false positives as a percentage of messages that aren't spam. The ratios would be completely different.
      - prodigal_erik 15y ago
        Oh yeah, there's only one absolute number of false positives, but when you turn it into a rate you have to choose either "of all positive results, what portion were false" or "of all test results, what portion were positive and false", which are far apart for tests that give many negative results.
espinchi 15y ago
Hmm are not those two sentences equivalent? That is, P(finding | random fluctuation) = 0.10
- disgruntledphd2 15y ago
  Nope, statistical significance is based on two things. 1) Assume the model is true. 2) Conditional on the model, what is the probability that the observed differences are due to chance? The p value is then the probability that a test statistic at least this extreme could have been produced by chance. Its actually p=0.001, not p=.10 also.

by DavidSJ 15y ago

extension 15y ago

What's the difference?

DavidSJ 15y ago
It's the same as the difference between "0.1% of non-spam messages are marked as spam" (false positive rate) vs. "0.1% of messages marked as spam are not spam" (false negative rate).
- prodigal_erik 15y ago
  Those seem like the same false positive case to me. For a false negative, did you mean "0.1% of spam messages are not marked as spam"?
  - DavidSJ 15y ago
    Whoops, you're right, they're both false positives, but one is number of false positives as a percentage of messages marked as spam, and the other is number of false positives as a percentage of messages that aren't spam. The ratios would be completely different.
    - prodigal_erik 15y ago
      Oh yeah, there's only one absolute number of false positives, but when you turn it into a rate you have to choose either "of all positive results, what portion were false" or "of all test results, what portion were positive and false", which are far apart for tests that give many negative results.

espinchi 15y ago

Hmm are not those two sentences equivalent? That is, P(finding | random fluctuation) = 0.10

disgruntledphd2 15y ago
Nope, statistical significance is based on two things. 1) Assume the model is true. 2) Conditional on the model, what is the probability that the observed differences are due to chance? The p value is then the probability that a test statistic at least this extreme could have been produced by chance. Its actually p=0.001, not p=.10 also.