3 ms·

http://blogs.nature.com/news/2011/09/reliability_of_new_drug_target.html http://blogs.nature.com/news/2011/09/reliability_of_new_drug... The problem is that th

http://blogs.nature.com/news/2011/09/reliability_of_new_drug_target.html http://blogs.nature.com/news/2011/09/reliability_of_new_drug... The problem is that there is a lot of incentive to publish peer reviewed articles and very little incentive to publish articles confirming another paper. In my experience, the reproducibility of papers is tested in the worst possible way: during the process of more novel research. If we are to continue this exponential growth in technological and scientific progress we need to set up a far more efficient way of reproducing the papers, independently of ongoing novel research. This seems like a great way, especially considering the benefit it will give to startups and other companies that can allow new R&D to rely on papers that have been tested outside of the peer review/slow meta-review process in science.

timr 14y ago
It depends on how you look it it: is it "more efficient" to spend the time and energy independently reproducing every paper that comes down the pipe, or to wait for researchers verify the work that actually matters? I think this kind of thing is probably a well-intentioned pipe dream. You can't predict which work is going to be important, so you can't predict, a priori, which work to verify. So you either have to verify all of it, or you have to guess, or...you can just let people do what they do right now: verify the work they care about.
- akiselev 14y ago
  Well for one they don't HAVE to reproduce every paper. They only have to reproduce protocols proposed for these papers, and mostly you can stick to the ones with obviously wide implications. If, for example, there was one group who tested all of the methods for manufacturing graphene for efficiency and repeatability, that would make work much easier for dozens if not hundreds of labs around the world. Not to mention it would open up the way for startups, who can rely on tested protocols, to begin industrial scale work with graphene. You can't predict which work would be important in general, yes, but when it comes to identifying important protocols in organic chemistry and biology, there is a lot more information on what is important and what is not. Most of the time, in o. chem, protocols are incremental improvements over previous protocols and it's very easy to figure out which of them are most often used.

timr 14y ago

It depends on how you look it it: is it "more efficient" to spend the time and energy independently reproducing every paper that comes down the pipe, or to wait for researchers verify the work that actually matters? I think this kind of thing is probably a well-intentioned pipe dream. You can't predict which work is going to be important, so you can't predict, a priori, which work to verify. So you either have to verify all of it, or you have to guess, or...you can just let people do what they do right now: verify the work they care about.

akiselev 14y ago
Well for one they don't HAVE to reproduce every paper. They only have to reproduce protocols proposed for these papers, and mostly you can stick to the ones with obviously wide implications. If, for example, there was one group who tested all of the methods for manufacturing graphene for efficiency and repeatability, that would make work much easier for dozens if not hundreds of labs around the world. Not to mention it would open up the way for startups, who can rely on tested protocols, to begin industrial scale work with graphene. You can't predict which work would be important in general, yes, but when it comes to identifying important protocols in organic chemistry and biology, there is a lot more information on what is important and what is not. Most of the time, in o. chem, protocols are incremental improvements over previous protocols and it's very easy to figure out which of them are most often used.