5 ms·

I applaud for the P < 0.01. There are too many non reproducible results with real life harm: http://infoproc.blogspot.com/2017/02/perverse-incentives-and-repli

• feral 10y ago
  Just to note, there's a tradeoff here - not publishing work until you are massively certain of it would also cause real life harm. Reducing the p value doesn't automatically reduce harm. Physicists require extremely low values before confirming a discovery has been made, but that's different from requiring it before publishing. The problem is with people interpreting published work as if once its published, its completely certain. Maybe each publication should come with a headline 'confidence' stat beside the title. I guess this is a step in that direction.
• mattkrause 10y ago
  You really think so? I'd argue the fetishization of a specific p-value threshold, be it p<0.05, p<0.01, or even p<0.0001, is a much bigger problem. There is an excellent quote from Rosnow and Rosenthal: [D]ichotomous significance testing has no ontological basis. That is, we want to underscore that, surely, God loves the .06 nearly as much as the .05. Can there be any doubt that God views the strength of evidence for or against the null as a fairly continuous function of the magnitude of p?” Wouldn't you prefer a few correct but tenative studies that hint at an effect while paving the way for larger, more expensive replications to a scenario where the data is sliced, diced, and tortured to hit some arbitrary p-value threshold?