3 ms·

> Do 20 experiments with a p<5% criterion, and it’s likely that one will be a false positive. That would be true if p were the probability of null hypothesis b

> Do 20 experiments with a p<5% criterion, and it’s likely that one will be a false positive. That would be true if p were the probability of null hypothesis being true given the data observed, but that’s not what it is.

lucianbr 2y ago
I really have no clue what p is, but I also really believe Randall Munroe does. Not that he's above making mistakes, but come on.
- mtts 2y ago
  p is the chance that you would have gotten this result if the null hypothesis were true.
stonemetal12 2y ago
>For example, if 1,000,000 tests are carried out, then 5% of them (that is, 50,000 tests) are expected to lead to p < 0.05 by chance when the null hypothesis is actually true for all these tests. https://eurradiolexp.springeropen.com/articles/10.1186/s41747-020-0145-y#:~:text=Keeping%20the%20significance%20threshold%20at,true%20for%20all%20these%20tests https://eurradiolexp.springeropen.com/articles/10.1186/s4174....
kerkeslager 2y ago
Even if you are right, how would this response be helpful? You're not giving the right answer, you're just saying the answer given is wrong. Nobody is coming out of this interaction with corrected knowledge in their head. This is the sort of thing I used to do when I was younger, and looking back, the reason I did it was because I was basing my sense of self-worth in being smarter than other people. Ironically, this made me dumber, because I was less open to the possibility of being wrong and therefore was slower to learn. And, I found doing this made people dislike me.
- deleted 2y ago
  [deleted]
Dylan16807 2y ago
> That would be true if p were the probability of null hypothesis being true given the data observed, but that’s not what it is. I think you're interpreting that sentence as talking about one false positive out of 20 positives? Which would be a very incorrect statement. But it says one false positive out of 20 experiments. That's a valid statement. (Though it does depend on the proportion of negatives you're getting.)

lucianbr 2y ago

I really have no clue what p is, but I also really believe Randall Munroe does. Not that he's above making mistakes, but come on.

mtts 2y ago
p is the chance that you would have gotten this result if the null hypothesis were true.

stonemetal12 2y ago

>For example, if 1,000,000 tests are carried out, then 5% of them (that is, 50,000 tests) are expected to lead to p < 0.05 by chance when the null hypothesis is actually true for all these tests. https://eurradiolexp.springeropen.com/articles/10.1186/s41747-020-0145-y#:~:text=Keeping%20the%20significance%20threshold%20at,true%20for%20all%20these%20tests https://eurradiolexp.springeropen.com/articles/10.1186/s4174....

kerkeslager 2y ago

Even if you are right, how would this response be helpful? You're not giving the right answer, you're just saying the answer given is wrong. Nobody is coming out of this interaction with corrected knowledge in their head. This is the sort of thing I used to do when I was younger, and looking back, the reason I did it was because I was basing my sense of self-worth in being smarter than other people. Ironically, this made me dumber, because I was less open to the possibility of being wrong and therefore was slower to learn. And, I found doing this made people dislike me.

deleted 2y ago
[deleted]

Dylan16807 2y ago

> That would be true if p were the probability of null hypothesis being true given the data observed, but that’s not what it is. I think you're interpreting that sentence as talking about one false positive out of 20 positives? Which would be a very incorrect statement. But it says one false positive out of 20 experiments. That's a valid statement. (Though it does depend on the proportion of negatives you're getting.)