5 ms·

I don't see how the BBC's statement can translate into P(hypothesis|data). To me it's P(data|H0). Consider this simpler example: "Scientists always assumed the

I don't see how the BBC's statement can translate into P(hypothesis|data). To me it's P(data|H0). Consider this simpler example: "Scientists always assumed their coin was fair. Well, they flipped it 10 times, and it came up heads every time. This is a 3-sigmal result. There's a 1 in 1024 chance of this measurement being a statistical coincidence." The reader would take this "1 in 1024 chance of coincidence" to mean "if the coin was fair, they'd have a 1 in 1024 chance of seeing this event occur". That's P(data|fair) = P(10 heads|fair) = 1/1024. I don't see how you read that and translate it into P(fair|10 heads).

DavidSJ 6y ago
If I say: "there's a 25% chance that Pat is from North Dakota", I'm taking as a given that there is a person named Pat, and I'm talking about the chance that Pat, whom we know to exist, is from North Dakota. In this case, grammatically, they're talking about the chance that the measurement (data) is a statistical coincidence, as opposed to not being a statistical coincidence. They're not talking about the chance that the data exists, as opposed to not existing.
- dataflow 6y ago
  > If I say: "there's a 25% chance that Pat is from North Dakota", I'm taking as a given that there is a person named Pat, and I'm talking about the chance that Pat, whom we know to exist, is from North Dakota. Yes, and if I say "there's a 0.001% chance that these coin flip observations were coincidental", I'm taking it as a given that there were coin flips observed (indeed, there were!) I'm talking about the chance that the observations, which we (indeed) know to exist, were coincidental. That is... literally P(observations | null hypothesis) = P(10 heads | fair). It's clearly not P(null hypothesis | observations) = P(fair | 10 heads)... > In this case, grammatically, they're talking about the chance that the measurement (data) is a statistical coincidence, as opposed to not being a statistical coincidence. They're not talking about the chance that the data exists, as opposed to not existing. Which is... perfectly fine. The data clearly does exist (as would the coin flips in my example), so the chance of the data existing is 100%... there's nothing interesting to talk about there. But the chance of it having been coincidental is 0.001%. This all seems correct to me...
  - DavidSJ 6y ago
    There's been a misunderstanding between us. To clarify, I'm not saying we know some data exists (that's of course true, since an experiment was done). I'm saying we know this data exists. Grammatically, they've worded it so that the question is whether this particular data that we have seen (as opposed to other data we could have seen) was in fact produced by chance or not. The difference between the data being a statistical coincidence and the data not being a statistical coincidence is precisely the difference between the null hypothesis being true and the null hypothesis not being true. So they're making a claim about the probability of a hypothesis.
    - dataflow 6y ago
      > There's been a misunderstanding between us. To clarify, I'm not saying we know some data exists (that's of course true, since an experiment was done). I'm saying we know this data exists. But that's exactly what I'm saying too? When I flip 10 coins and see 10 heads, that is the data, and it very much exists. Nobody is talking about any data except that one. Not me, not you, not BBC. > Grammatically, they've worded it so that the question is whether this particular data that we have seen (as opposed to other data we could have seen) was in fact produced by chance or not. Yes, and as I see it this is 100% correct. Like in my example where I saw 10 heads (that is the one and only dataset we know exists), and I'm wondering if they were produced by chance or not. That is precisely P(10 heads | fair coin) = P(observations | H0). > The difference between the data being a statistical coincidence and the data not being a statistical coincidence is precisely the difference between the null hypothesis being true and the null hypothesis not being true. Maybe your English analogy is the source of the confusion here? I don't know what rigorous definition you might have for "the difference" here(?), but whatever it is, "10 heads have a 0.001% chance with a fair coin" doesn't imply "10 heads have a 99.999% chance under an unfair coin"... right? Even though the former assumes H0 and the latter assumes !H0. Or to put it another way, just because H0 is true in one statement and false in another, that doesn't mean we're talking about the probability of H0 being true in one statement and the probability of H0 being false in another.
      - DavidSJ 6y ago
lovemenot 6y ago
Since neither heads nor tails bias had been hypothesised, 1/512 rather than 1/1024.
davrosthedalek 6y ago
It's like this: The chance to get 10 heads with a fair coin is <1/1000. I.e., if you use the same coin many-many times, <1/1000 of these 10-throw runs will have all heads, on average. This does not mean that, if you get 10 heads, it's a fair coin with 1/1000 probability. In the strongest sense, without further information, one can not make any statement about that probability.
- dataflow 6y ago
  I don't think either of us disagrees with this. We're not debating what is mathematically true. We're debating what the BBC wrote.
  - davrosthedalek 6y ago
    Yeah. My point is this "There's a 1 in 1024 chance of this measurement being a statistical coincidence." I understand this as "There is a 1:1024 chance that a statistical coincidence is the cause of this result". But that's wrong, the statement must be: If it is a statistical coincidence, it had a probability of 1:1024 of occurring. But then the probability that it was cause by a statistical fluctuation is 100% -- It's assumed to be true.

DavidSJ 6y ago

If I say: "there's a 25% chance that Pat is from North Dakota", I'm taking as a given that there is a person named Pat, and I'm talking about the chance that Pat, whom we know to exist, is from North Dakota. In this case, grammatically, they're talking about the chance that the measurement (data) is a statistical coincidence, as opposed to not being a statistical coincidence. They're not talking about the chance that the data exists, as opposed to not existing.

dataflow 6y ago
> If I say: "there's a 25% chance that Pat is from North Dakota", I'm taking as a given that there is a person named Pat, and I'm talking about the chance that Pat, whom we know to exist, is from North Dakota. Yes, and if I say "there's a 0.001% chance that these coin flip observations were coincidental", I'm taking it as a given that there were coin flips observed (indeed, there were!) I'm talking about the chance that the observations, which we (indeed) know to exist, were coincidental. That is... literally P(observations | null hypothesis) = P(10 heads | fair). It's clearly not P(null hypothesis | observations) = P(fair | 10 heads)... > In this case, grammatically, they're talking about the chance that the measurement (data) is a statistical coincidence, as opposed to not being a statistical coincidence. They're not talking about the chance that the data exists, as opposed to not existing. Which is... perfectly fine. The data clearly does exist (as would the coin flips in my example), so the chance of the data existing is 100%... there's nothing interesting to talk about there. But the chance of it having been coincidental is 0.001%. This all seems correct to me...
- DavidSJ 6y ago
  There's been a misunderstanding between us. To clarify, I'm not saying we know some data exists (that's of course true, since an experiment was done). I'm saying we know this data exists. Grammatically, they've worded it so that the question is whether this particular data that we have seen (as opposed to other data we could have seen) was in fact produced by chance or not. The difference between the data being a statistical coincidence and the data not being a statistical coincidence is precisely the difference between the null hypothesis being true and the null hypothesis not being true. So they're making a claim about the probability of a hypothesis.
  - dataflow 6y ago
    > There's been a misunderstanding between us. To clarify, I'm not saying we know some data exists (that's of course true, since an experiment was done). I'm saying we know this data exists. But that's exactly what I'm saying too? When I flip 10 coins and see 10 heads, that is the data, and it very much exists. Nobody is talking about any data except that one. Not me, not you, not BBC. > Grammatically, they've worded it so that the question is whether this particular data that we have seen (as opposed to other data we could have seen) was in fact produced by chance or not. Yes, and as I see it this is 100% correct. Like in my example where I saw 10 heads (that is the one and only dataset we know exists), and I'm wondering if they were produced by chance or not. That is precisely P(10 heads | fair coin) = P(observations | H0). > The difference between the data being a statistical coincidence and the data not being a statistical coincidence is precisely the difference between the null hypothesis being true and the null hypothesis not being true. Maybe your English analogy is the source of the confusion here? I don't know what rigorous definition you might have for "the difference" here(?), but whatever it is, "10 heads have a 0.001% chance with a fair coin" doesn't imply "10 heads have a 99.999% chance under an unfair coin"... right? Even though the former assumes H0 and the latter assumes !H0. Or to put it another way, just because H0 is true in one statement and false in another, that doesn't mean we're talking about the probability of H0 being true in one statement and the probability of H0 being false in another.
    - DavidSJ 6y ago

lovemenot 6y ago

Since neither heads nor tails bias had been hypothesised, 1/512 rather than 1/1024.

davrosthedalek 6y ago

It's like this: The chance to get 10 heads with a fair coin is <1/1000. I.e., if you use the same coin many-many times, <1/1000 of these 10-throw runs will have all heads, on average. This does not mean that, if you get 10 heads, it's a fair coin with 1/1000 probability. In the strongest sense, without further information, one can not make any statement about that probability.

dataflow 6y ago
I don't think either of us disagrees with this. We're not debating what is mathematically true. We're debating what the BBC wrote.
- davrosthedalek 6y ago
  Yeah. My point is this "There's a 1 in 1024 chance of this measurement being a statistical coincidence." I understand this as "There is a 1:1024 chance that a statistical coincidence is the cause of this result". But that's wrong, the statement must be: If it is a statistical coincidence, it had a probability of 1:1024 of occurring. But then the probability that it was cause by a statistical fluctuation is 100% -- It's assumed to be true.