5 ms·
> The statistical tests currently in widespread use as a criterion for publication in peer-reviewed journals guarantee that at least one result in 20 will be d
by yellowstuff 8y ago
> The statistical tests currently in widespread use as a criterion for publication in peer-reviewed journals guarantee that at least one result in 20 will be due to chance and not because the hypothesis being tested is actually true.
I have a pedantic correction, but a relevant one. If a journal makes a rule that results must have p < .05, and then scientists go off and do a bunch of well-managed science they will get results with a range of p-values. .05 will be a ceiling, not a floor, on the possible p-values. More papers have a p-values right around .05 than they would if results were being produced fairly, so that's actually evidence of bad practices.
- sfink 8y agoIs it? You're capped at 0.05. You can't print "obvious" results. ("With a confidence of p=0.00002173, we find that people are more likely to be angry in the slapped-in-the-face-by-the-researcher group than the control group...") So there's a natural floor determined by what is worth doing research on. Now, it turns out that the floor is far lower than researchers believe and the field would be better off by slower incremental progress built atop "obvious" results, but still, I assert that publishability is an adequate explanation for high p values and obscures the evidence for bad practices. Not that I have any doubt that bad practices abound; I have an undergraduate degree in psychology and am well aware of how things are done. It's not even malicious; people don't understand statistics, don't understand significance, don't understand models, and are mostly just doing whatever they can to make themselves feel good about their own intuitions and biases.