> 50% of scientists not being able to reproduce their own paper In the case of the psychology replication crisis (note that I'm NOT talking about climate scien

> 50% of scientists not being able to reproduce their own paper In the case of the psychology replication crisis (note that I'm NOT talking about climate science here), the problem was the result of a few factors: 1. the lack of incentives to replicate others' work 2. the use of p-values to determine what gets published 3. the low probability of a true positive These factors are not part of the critique process of peer review. That is, reviewers are looking at whether the methodology is valid. "Did the scientists follow proper procedures? Do the results match up with what they claim?" While these questions can identify methodological errors, they cannot identify when an experiment is a statistical fluke. Suppose the true positive rate is 5% and the false positive rate is 5%. That means that only 10% of experiments will give results that suggest publishing. But if you look at published papers only, the probability that the paper is false is not 5%, it's 50%! The solution to this problem is not to throw out peer review, it's to fix the three issues above. We must create a system that incentivizes replication. Perhaps 1st-year grad students will be required to attempt to replicate existing work. (This would be more interesting for the grad student than taking an additional class!) Perhaps we should move away from p-values (but what do we replace that with?) However, the replication crisis is not really the problem of bad science or corruption -- it's mostly the result of bad incentives and false positives.