3 ms·

4.) You mean that papers are published with P<0.05? When did this become a "standard"? Most papers I have seen in respectable journals and other sources would

4.) You mean that papers are published with P<0.05? When did this become a "standard"? Most papers I have seen in respectable journals and other sources would not strive for such a high Pvalue, it would be much lower like 0.001.

lisper 9y ago
> You mean that papers are published with P<0.05? I'm guessing you meant to say that no papers are published with P<0.05. And of course they are, but that's irrelevant. What matters it that many journals, especially in "softer" sciences (i.e. not physics) use p<0.05 as their threshold, which guarantees that at least one paper in 20 will be "wrong". Then add self-suppression of negative results and the actual percentage of "wrong" papers becomes higher, potentially much higher if real positive results are rare. > When did this become a "standard"? I have no idea. You'll have a ask a historian.
- hpcjoe 9y ago
  Part of the mistake here is in interpreting the p value as being a measure of anything really meaningful. P<0.05 doesn't mean 1 in 20 are wrong. It does mean that in the narrow confines of the definition of the statistical model used, it is possible that 1 in 20 would be explainable by the null hypothesis. It is not, in any way a measure of reality. It is a guess at how accurately you've modeled the system, how accurate your priors are, and so forth.
  - lisper 9y ago
    > P<0.05 doesn't mean 1 in 20 are wrong > It does mean that ... 1 in 20 would be explainable by the null hypothesis. That seems like splitting a pretty fine hair to me, particularly since I haven't actually committed to any of the many possible interpretations of the word "wrong".
haZard_OS 9y ago
p>0.05 has been a mainstay in entire fields of study for quite some time. Most notably, the psychological and biological science journals have unofficially endorsed 0.05 as a cutoff for "significant" results. While not everyone in a particular field has been a party to this myopic view of statistical significance, enough have been that the American Statistical Association released a statement on the topic not long ago. That may not seem like a big deal but, given the organization's historical reticence to officially weigh in on such issues, it really is. ASA statement on p-Values: https://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf https://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf
Tenoke 9y ago
>When did this become a "standard"? Most people would say in the 20s and point at Fischer's publications, though you can have arguments for later and earlier. Some of that is mentioned in the Wiki page, and you can see 0.05 all over the article [0], and the other related wiki pages[1]. You can find some key quotes there: "The significance level for a study is chosen before data collection, and typically set to 5% or much lower, depending on the field of study" "In 1925, Ronald Fisher advanced the idea of statistical hypothesis testing, which he called "tests of significance", in his publication Statistical Methods for Research Workers. Fisher suggested a probability of one in twenty (0.05) as a convenient cutoff level to reject the null hypothesis.In a 1933 paper, Jerzy Neyman and Egon Pearson called this cutoff the significance level, which they named α." If you mostly see lower p-values, that most likely means that you usually read non-social science papers. I guarantee that most social science (and related) fields use 0.05 regardless of the respectability of the journal. As an example of fields with lower p-values - 0.001 is common in medicine, and physics commonly has even lower values. 0. https://en.wikipedia.org/wiki/P-value#History https://en.wikipedia.org/wiki/P-value#History 1. https://en.wikipedia.org/wiki/Statistical_significance#History https://en.wikipedia.org/wiki/Statistical_significance#Histo...
chrisfosterelli 9y ago
> When did this become a "standard"? I think this depends on the field. In some areas of neuroscience for example, brain data can have extremely high variability and is very expensive to collect a lot of, so p<0.05 is not uncommon.

lisper 9y ago

> You mean that papers are published with P<0.05? I'm guessing you meant to say that no papers are published with P<0.05. And of course they are, but that's irrelevant. What matters it that many journals, especially in "softer" sciences (i.e. not physics) use p<0.05 as their threshold, which guarantees that at least one paper in 20 will be "wrong". Then add self-suppression of negative results and the actual percentage of "wrong" papers becomes higher, potentially much higher if real positive results are rare. > When did this become a "standard"? I have no idea. You'll have a ask a historian.

hpcjoe 9y ago
Part of the mistake here is in interpreting the p value as being a measure of anything really meaningful. P<0.05 doesn't mean 1 in 20 are wrong. It does mean that in the narrow confines of the definition of the statistical model used, it is possible that 1 in 20 would be explainable by the null hypothesis. It is not, in any way a measure of reality. It is a guess at how accurately you've modeled the system, how accurate your priors are, and so forth.
- lisper 9y ago
  > P<0.05 doesn't mean 1 in 20 are wrong > It does mean that ... 1 in 20 would be explainable by the null hypothesis. That seems like splitting a pretty fine hair to me, particularly since I haven't actually committed to any of the many possible interpretations of the word "wrong".

haZard_OS 9y ago

p>0.05 has been a mainstay in entire fields of study for quite some time. Most notably, the psychological and biological science journals have unofficially endorsed 0.05 as a cutoff for "significant" results. While not everyone in a particular field has been a party to this myopic view of statistical significance, enough have been that the American Statistical Association released a statement on the topic not long ago. That may not seem like a big deal but, given the organization's historical reticence to officially weigh in on such issues, it really is. ASA statement on p-Values: https://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf https://www.amstat.org/asa/files/pdfs/P-ValueStatement.pdf

Tenoke 9y ago

>When did this become a "standard"? Most people would say in the 20s and point at Fischer's publications, though you can have arguments for later and earlier. Some of that is mentioned in the Wiki page, and you can see 0.05 all over the article [0], and the other related wiki pages[1]. You can find some key quotes there: "The significance level for a study is chosen before data collection, and typically set to 5% or much lower, depending on the field of study" "In 1925, Ronald Fisher advanced the idea of statistical hypothesis testing, which he called "tests of significance", in his publication Statistical Methods for Research Workers. Fisher suggested a probability of one in twenty (0.05) as a convenient cutoff level to reject the null hypothesis.In a 1933 paper, Jerzy Neyman and Egon Pearson called this cutoff the significance level, which they named α." If you mostly see lower p-values, that most likely means that you usually read non-social science papers. I guarantee that most social science (and related) fields use 0.05 regardless of the respectability of the journal. As an example of fields with lower p-values - 0.001 is common in medicine, and physics commonly has even lower values. 0. https://en.wikipedia.org/wiki/P-value#History https://en.wikipedia.org/wiki/P-value#History 1. https://en.wikipedia.org/wiki/Statistical_significance#History https://en.wikipedia.org/wiki/Statistical_significance#Histo...

chrisfosterelli 9y ago

> When did this become a "standard"? I think this depends on the field. In some areas of neuroscience for example, brain data can have extremely high variability and is very expensive to collect a lot of, so p<0.05 is not uncommon.