3 ms·
By replication, let's refer to refutations. I'd unfortunately agree that positive replications would provide relatively little value. But replications that seem
by rjf72 7y ago
By replication, let's refer to refutations. I'd unfortunately agree that positive replications would provide relatively little value. But replications that seem to refute previously published studies are critical for at least two reasons I immediately see, and undoubtedly many more:
1) Replication efforts need to be strongly incentivized. Science in modern times has been taking repeated hammer blows to its credibility. The systems we have in place seem to result in more and more simply 'fake' science being published, including in the top journals. This is not an easy problem to solve since replication efforts can be very costly and things like pre-publishing still ends up relying on good behavior, which is something we can no longer necessarily take for granted. The one small tool we do have against bad science is replication efforts. Every effort should be made to incentivize these. The willingness to publish significant refutations to influential studies is well below the bare minimum we should expect from journals on this front.
2) What is the purpose of a journal, outside of making money? It's to inform people by providing a filtered repository of the information the reaches a high level on the scale between reliability and relevance. When a journal publishes a study that may have been false, they end up doing the exact opposite of this - they are misinforming society. And this is especially true for the study which seems to have been refuted. It was not a footnote - it has been widely referenced and had a significant impact on public discourse. This all leads to a strong obligation to publish.
3) (another point derived from #1 while writing) - Publication of refutations aligns journalistic interests with public interests. In our current system journals are primarily motivated to publish the most 'meaningful' results which is ultimately a euphemism for the most shocking results. So they have an incentive to want to give these studies the benefit of the doubt. Yet these are the very studies which should be held to the highest degree of scrutiny and criticism. A journal that is willing to publish refutations of the studies it publishes returns this alignment since if Science found itself full of little more than refutation of 'meaningful' studies it published in the past, it would quickly lose its prestige. Yet at the same time if it was willing to publish refutations, yet they were few and far between, one could hold what was published within the journal to a much higher standard of reliability - which would be good for both the journal and the authors who were able to meet what would undoubtedly be their now increased standards.