3 ms·

yes, probably false. This is not news, e.g. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ came out i

yes, probably false. This is not news, e.g. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1182327/ came out in 2005. of course, research "proving" that research tends to be false can get us into a little circular game...

a_bonobo 13y ago
There's a follow-up paper to the one you linked that claims that much less published research findings are false: >We estimate that the overall rate of false discoveries among reported results is 14% (s.d. 1%), contrary to previous claims. We also found that there is no a significant increase in the estimated rate of reported false discovery results over time (0.5% more false positives (FP) per year, P=0.18) or with respect to journal submissions (0.5% more FP per 100 submissions, P=0.12). [1] http://biostatistics.oxfordjournals.org/content/early/2013/09/24/biostatistics.kxt007 http://biostatistics.oxfordjournals.org/content/early/2013/0... The circle has begun :)
- capnrefsmmat 13y ago
  You should also read the responses to that article, such as Ioannidis's, which is scathing. They're all open-access, thankfully, and you can find them here: http://simplystatistics.org/2013/09/25/is-most-science-false-the-titans-weigh-in/ http://simplystatistics.org/2013/09/25/is-most-science-false... Jager and Leek put their code on GitHub, so the commenters were able to review their code and tinker with it to see what would happen.
  - a_bonobo 13y ago
    Thanks for all these links! I skimmed them, and it's fun to see that two of them rip into Ionnadis' work, while Ionnadis rips into the Jager/Leek work... It's also a cool exercise in reproducible research and open peer review, both of which are far from common. In my opinion, the entire exercise of data-mining the published literature is pretty much futile. We already know there are problems in the published literature and that scientists are pretty mediocre at statistics. Pin-pointing the exact value of how mediocre only leads to, as your link shows, a mountain of published works, hurt feelings (on Ionnadis side I guess) and doesn't solve anything.

a_bonobo 13y ago

There's a follow-up paper to the one you linked that claims that much less published research findings are false: >We estimate that the overall rate of false discoveries among reported results is 14% (s.d. 1%), contrary to previous claims. We also found that there is no a significant increase in the estimated rate of reported false discovery results over time (0.5% more false positives (FP) per year, P=0.18) or with respect to journal submissions (0.5% more FP per 100 submissions, P=0.12). [1] http://biostatistics.oxfordjournals.org/content/early/2013/09/24/biostatistics.kxt007 http://biostatistics.oxfordjournals.org/content/early/2013/0... The circle has begun :)

capnrefsmmat 13y ago
You should also read the responses to that article, such as Ioannidis's, which is scathing. They're all open-access, thankfully, and you can find them here: http://simplystatistics.org/2013/09/25/is-most-science-false-the-titans-weigh-in/ http://simplystatistics.org/2013/09/25/is-most-science-false... Jager and Leek put their code on GitHub, so the commenters were able to review their code and tinker with it to see what would happen.
- a_bonobo 13y ago
  Thanks for all these links! I skimmed them, and it's fun to see that two of them rip into Ionnadis' work, while Ionnadis rips into the Jager/Leek work... It's also a cool exercise in reproducible research and open peer review, both of which are far from common. In my opinion, the entire exercise of data-mining the published literature is pretty much futile. We already know there are problems in the published literature and that scientists are pretty mediocre at statistics. Pin-pointing the exact value of how mediocre only leads to, as your link shows, a mountain of published works, hurt feelings (on Ionnadis side I guess) and doesn't solve anything.