High profile journals like Nature already require code and have relatively detailed guidelines for submission [1]. They also have detailed guidelines for statis

High profile journals like Nature already require code and have relatively detailed guidelines for submission [1]. They also have detailed guidelines for statistics, e.g. you must report control parameters when using MCMC. My experience in Nature, Science, and Cell, both as a submitter and as a reviewer, is that problems tend to be in the way inference is done. I don't think people lie with their code. They lie with the way they do inference, but inference tends to be faithfully implemented as code. For example, the null hypothesis tends to be false almost all the time. So, it is a matter of accumulating a sufficient amount of data to get a small enough p-value to call the result "significant". Solution, encourage designs with a continuous explanatory variable, emphasize effect sizes, use partial pooling / shrinking, etc. And perhaps hire technical editors that get the right to veto flawed methods. [1] https://www.nature.com/documents/nr-software-policy.pdf https://www.nature.com/documents/nr-software-policy.pdf