3 ms·
I tend to find some of the headlines around the 'reproducibility crisis' quite overblown, but ultimately I agree with Arnold's interpretation of the situation.
by Obi_Juan_Kenobi 10y ago
I tend to find some of the headlines around the 'reproducibility crisis' quite overblown, but ultimately I agree with Arnold's interpretation of the situation. Some fields have strong foundations, often based in some experimental methods that are good at providing robust results. Once set, the mores of a field are mostly unchanging, for good or ill. This is a very important, perhaps essential, part of the way that science operates in the real world.
In some cases this is very good. Part of my difficulty in understanding this issue is that my background is in molecular biology. Moreover, I studied models that had very little to do with cancer funding, and had a strong focus on 'classic' methods (protein and RNA blots and 'hardcore' genetics). From my/our view, even the new genomic methods (which are usually very good, overall) are viewed with very deep skepticism. Your quality as an investigator, in large part, depends on how skeptical of your own work you are. A good presentation often leaves you feeling beat over the head with consistent data from all manner of different methods. As a new investigator, it really feels excessive, but it's the deep suspicion that drives the field. It has real impact on who gets funded and what gets published.
Ben Goldacre's work was what first made me realize how different this was to some fields. Not just the results that he uncovered, but the response he received for very thorough and fair critique.
Reproducibility will always be difficult; even for 'true' results, you wouldn't expect all reproduction attempts to work. You need people with a lot wet-lab experience and suitable troubleshooting skill. Even the best, most robust methods will present some difficulty, but that alone does not constitute a failure to replicate. But Goldacre's epidemiological approach avoids those sorts of pitfalls.
I suppose it is somewhat troubling that a single person/couple can have so much influence over science, but I find their approach and results genuine. They are funding, as best as I can tell, very deserving investigators.