4 ms·

We can quibble over the number, maybe it is low as 10%. The cost to reproduce a study will be significantly higher than to produce it in the first place, due to

We can quibble over the number, maybe it is low as 10%. The cost to reproduce a study will be significantly higher than to produce it in the first place, due to different expertises, equipment and so forth. I estimate at least a 10X factor. And I am intimately familiar with what researchers “get away with’ while ‘working at home’. As a researcher who tried to reproduce several research papers only to discover the original scientists were wildly exaggerating their claims or cleverly disguising fundamental shortcomings, I can assure the cost is quite high to the scientific community, well in excess of 25% of the annual $48B NIH budget. I hold a healthy disdain for my fellow scientists. The only way to get them to play by the rules in my view is to have a threat of a research audit hanging over them.

nextos 2y ago
This is the key part, even legitimate findings tend to be exaggerated. A great step forward would be to distribute grants to reproduce new key findings. For example, the NIH could identify the top findings from 2024 that need to be reproduced, and seek expressions of interest to reproduce these and/or other important findings identified by applicants. Perhaps, also reach an agreement with top journals to publish replications as a new article type, and link them to the original one, just like they do with comments/news & views. It would instantly make those publishing super edgy findings much more careful, just in case, and things would become more efficient.
- BenFranklin100 2y ago
  I love the idea of putting each year’s top papers under the microscope. Present day hucksters publishing in Nature will think thrice about making unsupported claims. Further, the automatic reproduction of the research will serve as cement in the fundamental building block nature of science.
  - nextos 2y ago
    Yes, exactly. It'd act as deterrent for anyone considering to exaggerate or misrepresent findings. Also, it'd lead to a quicker dissemination of true findings. Currently, academic publications are in a bit of market for lemons situation [1], where the seller (authors) have much more information than the buyers (readers, funders). Time to change that. [1] https://en.wikipedia.org/wiki/The_Market_for_Lemons https://en.wikipedia.org/wiki/The_Market_for_Lemons
hoseja 2y ago
>The cost to reproduce a study will be significantly higher than to produce it in the first place wdym? You're on the happy path when reproducing, the cost of the original study includes all the failed attempts.

nextos 2y ago

This is the key part, even legitimate findings tend to be exaggerated. A great step forward would be to distribute grants to reproduce new key findings. For example, the NIH could identify the top findings from 2024 that need to be reproduced, and seek expressions of interest to reproduce these and/or other important findings identified by applicants. Perhaps, also reach an agreement with top journals to publish replications as a new article type, and link them to the original one, just like they do with comments/news & views. It would instantly make those publishing super edgy findings much more careful, just in case, and things would become more efficient.

BenFranklin100 2y ago
I love the idea of putting each year’s top papers under the microscope. Present day hucksters publishing in Nature will think thrice about making unsupported claims. Further, the automatic reproduction of the research will serve as cement in the fundamental building block nature of science.
- nextos 2y ago
  Yes, exactly. It'd act as deterrent for anyone considering to exaggerate or misrepresent findings. Also, it'd lead to a quicker dissemination of true findings. Currently, academic publications are in a bit of market for lemons situation [1], where the seller (authors) have much more information than the buyers (readers, funders). Time to change that. [1] https://en.wikipedia.org/wiki/The_Market_for_Lemons https://en.wikipedia.org/wiki/The_Market_for_Lemons

hoseja 2y ago

>The cost to reproduce a study will be significantly higher than to produce it in the first place wdym? You're on the happy path when reproducing, the cost of the original study includes all the failed attempts.