3 ms·

Indeed. At the present time many or most scientific studies are not reproducible.

epistasis 9y ago
I object tremendously to that characterization, because "reproducible" is a fuzzy word that never gets defined properly. These results are almost certainly "reproducible" in that you could take the same people, regenerate all the measurements again, and get the same results by running the methods described in the paper. These results may not be "reproducible" in the sense that when they do this with a set of 2000 new individuals, they will probably not find the same SNPs at the same significance. Public discussion that conflates these two only serves to discredit science, without improving it or its process. Worse, saying that only only the second sense of "reproducible" is worthy of "science", would severely impede the process of discovery. As long as "reproducible" in the first sense holds, scientists can learn from each other. Scientists also need to be able to communicate with each other before announcing to the world "we've solved it all!" I find that most people (and I'm not including you with this statement, I don't know what you think!) who are bemoaning the lack of "reproducibility" are only interested in that second sense of reproducibility, because they want to open a journal to a random page make broad statements that are widely applicable. Scientists need to be able to talk about things in narrow terms, and specifics, or they will never be able to get to the broad rules that are generally useful.
- tremendulo 9y ago
  >they will never be able to get to the broad rules that are generally useful. Well at least we agree that discovering universal laws is desirable. Because we want to explain the world, right? Some more info about reproducibility: http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970?WT.mc_id http://www.nature.com/news/1-500-scientists-lift-the-lid-on-...

I object tremendously to that characterization, because "reproducible" is a fuzzy word that never gets defined properly. These results are almost certainly "reproducible" in that you could take the same people, regenerate all the measurements again, and get the same results by running the methods described in the paper. These results may not be "reproducible" in the sense that when they do this with a set of 2000 new individuals, they will probably not find the same SNPs at the same significance. Public discussion that conflates these two only serves to discredit science, without improving it or its process. Worse, saying that only only the second sense of "reproducible" is worthy of "science", would severely impede the process of discovery. As long as "reproducible" in the first sense holds, scientists can learn from each other. Scientists also need to be able to communicate with each other before announcing to the world "we've solved it all!" I find that most people (and I'm not including you with this statement, I don't know what you think!) who are bemoaning the lack of "reproducibility" are only interested in that second sense of reproducibility, because they want to open a journal to a random page make broad statements that are widely applicable. Scientists need to be able to talk about things in narrow terms, and specifics, or they will never be able to get to the broad rules that are generally useful.

tremendulo 9y ago
>they will never be able to get to the broad rules that are generally useful. Well at least we agree that discovering universal laws is desirable. Because we want to explain the world, right? Some more info about reproducibility: http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970?WT.mc_id http://www.nature.com/news/1-500-scientists-lift-the-lid-on-...