5 ms·

He is coming at that conclusion from a Bayesian point of view to statistics. He is seeing the p-value as a random variable that can take values from 0 to 1 and

He is coming at that conclusion from a Bayesian point of view to statistics. He is seeing the p-value as a random variable that can take values from 0 to 1 and follows some distribution. Under these hypotheses, observing a p-value of 0.20 and 0.005 is completely reasonable even if unlikely. Those are just two draws from a random variable. Edit. Under Bayesian statistics testing the null hypothesis is a moot point as it becomes possible to directly model the distribution of the possible effects. Thinking of it as being able to look at a picture of something (the p-value) vs looking at a movie of it (the distribution of the effects).

pfortuny 7y ago
What he says is (I gather) worse: those events are only separated be 1.1std deviations, which is little.
- mehrdadn 7y ago
  I think that's what he's saying too, but what is that supposed to show? Is he arguing against some claim that every interval of 1 standard deviation is equally significant? Did anybody make this claim? So far as I know, nobody considers (say) a 6-sigma effect to be 6 times stronger than a 1-sigma effect...
  - kgwgk 7y ago
    I don’t get it either. It’s a trivial consequence of having a threshold: if we say two cities are “far” when they are at least 1000 miles away then Washington D.C. is not far from Jacksonville while Boston is far from Jacksonville, even though Boston is not far from Washington.
    - nerdponx 7y ago
      That's not quite the problem. Let's assume you've already decided in advance what "far" means. Without moving either city from its current location, the same experiment can give you "very far" and "very close" in identical replications.
      - kgwgk 7y ago
        I’m commenting on pfortuny’s (correct) interpretation “those events are only separated be 1.1std deviations, which is little” of the original post by Gelman (the difference between a significant result and a non-significant result may not be significant = a city which is far from Jacksonville and a city which is not far from Jacksonville may not be far from each other).
mehrdadn 7y ago
I still don't get it. What p-value observations would not be "reasonable" here? It seems to me he's saying that anything between 0 and 1 is completely reasonable, which is a completely pointless statement as I see it.
- lonelappde 7y ago
  His point is that the whole point of the field of statistics is that cherry picking results or getting lucky once is not proof of anything, unlike nonstatistical mathematics where one example is enough to justify a claim.
JustFinishedBSG 7y ago
That's not even a bayesian point of view, even in a frequentist setting the p-value is a conditional probability, conditioned on the dataset.
kgwgk 7y ago
> He is seeing the p-value as a random variable that can take values from 0 to 1 and follows some distribution. That is the frequentist point of view!

pfortuny 7y ago

What he says is (I gather) worse: those events are only separated be 1.1std deviations, which is little.

mehrdadn 7y ago
I think that's what he's saying too, but what is that supposed to show? Is he arguing against some claim that every interval of 1 standard deviation is equally significant? Did anybody make this claim? So far as I know, nobody considers (say) a 6-sigma effect to be 6 times stronger than a 1-sigma effect...
- kgwgk 7y ago
  I don’t get it either. It’s a trivial consequence of having a threshold: if we say two cities are “far” when they are at least 1000 miles away then Washington D.C. is not far from Jacksonville while Boston is far from Jacksonville, even though Boston is not far from Washington.
  - nerdponx 7y ago
    That's not quite the problem. Let's assume you've already decided in advance what "far" means. Without moving either city from its current location, the same experiment can give you "very far" and "very close" in identical replications.
    - kgwgk 7y ago
      I’m commenting on pfortuny’s (correct) interpretation “those events are only separated be 1.1std deviations, which is little” of the original post by Gelman (the difference between a significant result and a non-significant result may not be significant = a city which is far from Jacksonville and a city which is not far from Jacksonville may not be far from each other).

mehrdadn 7y ago

I still don't get it. What p-value observations would not be "reasonable" here? It seems to me he's saying that anything between 0 and 1 is completely reasonable, which is a completely pointless statement as I see it.

lonelappde 7y ago
His point is that the whole point of the field of statistics is that cherry picking results or getting lucky once is not proof of anything, unlike nonstatistical mathematics where one example is enough to justify a claim.

JustFinishedBSG 7y ago

That's not even a bayesian point of view, even in a frequentist setting the p-value is a conditional probability, conditioned on the dataset.

kgwgk 7y ago

> He is seeing the p-value as a random variable that can take values from 0 to 1 and follows some distribution. That is the frequentist point of view!