2 ms·
Publication bias is a substantially more descriptive and accurate phrase. Even when you do submit studies with negative results for publication, many journals s
by apathy 10y ago
Publication bias is a substantially more descriptive and accurate phrase. Even when you do submit studies with negative results for publication, many journals simply will not accept them. Little to do with file drawers and much to do with editors.
That said, power and experimental design -- can we confidently reject the hypothesis that an effect size of at least X is present between groups A and B (or models A and B) at some specified error bound? -- cannot be ignored. Bayesian or frequentist, at some point you have to conclude the experiment or write an interim report, and this involves making a decision in the face of uncertainty.
Being a statistician means never having to say you're certain. It also means never being able to do so (unless you're an irresponsible asshole). You can declare a decision boundary and, based on that, evaluate the evidence. In light of this, it is important that negative studies not have inferior designs or sample sizes to the positive studies, and vice versa.
In humans, it is exceedingly difficult to control for all of the factors you might want to. Nevertheless, the tendency of journals and PR outlets to favor sexy outliers over the weight of evidence is a huge source of this bias, which cannot be solely ascribed to unwillingness of researchers to submit the works for publication. (It is also why preprints are so valuable and their uptake so important.)