There's another issue at play here that's tightly connected to this one. Underpowered studies. Here's the process. I look elsewhere (e.g. similar studies) to ma

There's another issue at play here that's tightly connected to this one. Underpowered studies. Here's the process. I look elsewhere (e.g. similar studies) to make an educated guess about the size of effect I'm going to look for. Let's say I land on 2 units. Then I ask how many samples I need to detect a 2 unit effect with some low false positive procedure. I want to detect my 2 unit effect with high probability if it's there. So you end up with three numbers. The probability you find something given that there's nothing, often 5%. The probability you find nothing given that there's something (something being a 2 unit effect), often 20%. The size of effect you're using in these calculations (2 units). The smaller effect size you're looking for, the more data you need. The lower error rates you're looking to get, the more data you need. So you make sacrifices and accept higher error rates. You make optimistic effect size estimates (making your true error rates even higher). Using p < 5% as a positive result publishing threshold is bad enough. Using power > 80% as a negative result publishing threshold is even crazier. That's a 20% error rate for publishing negatives. Higher if you used optimistic effect estimates. What this all means is yes we should publish negatives, but a negative worth listening to is going to be expensive. In a bayesian sense, it's the difference between me spending little money to learn that the effect of coffee on mood is probably between -1 and 1 (a bad negative result) vs spending a lot of money to learn that it's probably between -0.001 and 0.001 (a more useful negative result).