The point I think you missed is that people only publish results when they succeed. So, if you look at the entire scientific landscape, let's pretend there is

The point I think you missed is that people only publish results when they succeed. So, if you look at the entire scientific landscape, let's pretend there is an experiment done where p=.05. The reality is that there is no effect, but in fact 20 of these experiments are done around the global at roughly the same time. At p=.05, we would expect 1 of those 20 experiments to support the null hypothesis /by random chance/. So the author here says that the problem is that all 20 of those experiments don't get published, but instead only the 1 that happened to show an effect. That's misleading because it makes the result seem significant, when in fact it's just an artifact. So then you say, but wait! It's okay, because now we can try to replicate those results so the bogus result from before will be found out, and we can all rest easy knowing that the brave men and women of statistics are vigilant. But here's the problem: this one reported result leads to a wave of experiments trying to replicate the result, and according to the author there's a strong bias against publishing a paper that doesn't support the result. So let's say the finding was SUPER EXCITING (OMG), and 100 teams went out to replicate it. 5 of them should support the original result just by coincidence, even though they are all wrong. Who publishes? Those 5! And maybe a couple others. So now, even though the work has been done (but not published) that pretty clearly demonstrates that there is no effect, we have in public view: 1 Study that shows an effect to p=.05 5 studies that support the findings of the first also to p=.05 2 studies that reject the original null hypothesis 6 studies supporting. 2 against. Ah, interesting. It's a small controversy, but "clearly" there's something here worth studying. Right? Not so much. What about the other 112 studies?