This is a common misunderstanding of p-values. There is a 5% chance of observing the effect, if the effect is not there. I think the difference is best illust

This is a common misunderstanding of p-values. There is a 5% chance of observing the effect, if the effect is not there. I think the difference is best illustrated by an example: If you have 20 researchers investigating a hypothesis that turns out to be false 1 of them will report evidence for a false hypothesis, which will likely not replicate. Thus, if for every true hypotheses investigated, 20 false hypotheses are investigated, and the 19 researchers with false hypotheses do not report their result, that means that 1/2 of the reported positive results will not replicate.