3 ms·
While I agree with the general sentiment of the paper and creating incentives for more replication is definitely a good idea, I do think the approach is flawed
by cycomanic 3y ago
While I agree with the general sentiment of the paper and creating incentives for more replication is definitely a good idea, I do think the approach is flawed in several ways.
The main point is that the paper seriously underestimates the difficulty and time it requires to replicate experiments in many experimental fields. Who will decide which work needs to be replicated? Should capable labs somehow become bogged down with just doing replication work? Even if they don't find the results not interesting?
In reality if labs find results interesting enough to replicate they will try to do so. The current LK-99 hurrah is a perfect example of that, but it happens on a much smaller scale all the time. Researchers do replicate and build on other work all the time, they just use that replication to create new results (and acknowledge the previous work) instead of publishing a "we replicated paper".
Where things usually fail is in publication of "failed replication" studies, and those are tricky. It is not always clear if the original research was flawed or the people trying to reproduce made an error (again just have a look at what's happening with LK-99 at the moment). Moreover, it can be politically difficult to try to publish a "fail to reproduce" result if you are small unknown lab, if the original result came from a big known group. Most people will believe that you are the one who made the error (and unfortunately big egos might get in the way, and the small lab will have a hard time).
More generally, in my opinion the lack of replication of results is just one symptom of a bigger problem in science today. We (as in society) have essentially turned the scientific environment increasingly competitive, under the guise of "value for tax payer money". Academic scientists now have to constantly compete for grant funding, publish to keep the funding going. It's incredibly competitive to even get in ... At the same time they are supposed to constantly provide big headlines for university press releases, communicate their results to the general public and investigate (and patent) the potential for commercial exploitation. No wonder we see less cooperation.