4 ms·

Most research spending is wasted on bad studies

9 points by dankohn1 11y ago

adrianN 11y ago
I once heard a great talk by Eytan Domany, a physicist who moved to biology. He had a look at Genome Wide Association studies, where you sequence a number of people with an illness and without the illness, and try to find genes that correlate with having the illness. The problem with these studies, he told us, was that quite often the number of genes they find correlating with having the illness is about the same order of magnitude as the number of participants in the study. But because many biologists are not so proficient at statistics, they don't see that this smells of overfitting their model to the data. He took a particularly popular such study and applied a standard technique to detect overfitting, cross validation, and indeed found that the correlation they reported was spurious. If I remember correctly, even after he destroyed the study, people kept citing it...
ommunist 11y ago
To be honest, try to compare overall expenses with outcome from good studies, over relatively long period. You need ecosystem of bad studies, because this is the agar from which bacteria of good studies can propagate.
ap22213 11y ago
Anyone in academic research able to give a summary of how grants are decided? I'm under the impression that most science research is guided by military, corporate, and imperial needs. And, far less pure science gets funded just to empirically understand the universe. It's a horrible shame in my opinion. Can't crowd funding come into play somehow? I'd love to donate money to scientific research. But, how is it possible for a layman to validate the 'real science' from the quakery?
ap22213 11y ago
Anyone in academic research able to give a summary of how grants are decided? I'm under the impression that most science research is guided by military, corporate, and imperial needs. And, far less pure science gets funded to empirically understand the universe. It's a horrible shame in my opinion. Can't crowd funding come into play somehow? I'd love to donate money to scientific research.
kevin_morrill 11y ago
There is a great EconTalk episode on poor incentives in research: http://www.econtalk.org/archives/2012/09/nosek_on_truth.html http://www.econtalk.org/archives/2012/09/nosek_on_truth.html Much of this research is funding through the NIH, which has a $30B budget. I have to believe there are tremendous politics in that.

9 points by dankohn1 11y ago

I once heard a great talk by Eytan Domany, a physicist who moved to biology. He had a look at Genome Wide Association studies, where you sequence a number of people with an illness and without the illness, and try to find genes that correlate with having the illness. The problem with these studies, he told us, was that quite often the number of genes they find correlating with having the illness is about the same order of magnitude as the number of participants in the study. But because many biologists are not so proficient at statistics, they don't see that this smells of overfitting their model to the data. He took a particularly popular such study and applied a standard technique to detect overfitting, cross validation, and indeed found that the correlation they reported was spurious. If I remember correctly, even after he destroyed the study, people kept citing it...

ommunist 11y ago

To be honest, try to compare overall expenses with outcome from good studies, over relatively long period. You need ecosystem of bad studies, because this is the agar from which bacteria of good studies can propagate.

ap22213 11y ago

Anyone in academic research able to give a summary of how grants are decided? I'm under the impression that most science research is guided by military, corporate, and imperial needs. And, far less pure science gets funded just to empirically understand the universe. It's a horrible shame in my opinion. Can't crowd funding come into play somehow? I'd love to donate money to scientific research. But, how is it possible for a layman to validate the 'real science' from the quakery?

ap22213 11y ago

Anyone in academic research able to give a summary of how grants are decided? I'm under the impression that most science research is guided by military, corporate, and imperial needs. And, far less pure science gets funded to empirically understand the universe. It's a horrible shame in my opinion. Can't crowd funding come into play somehow? I'd love to donate money to scientific research.

kevin_morrill 11y ago

There is a great EconTalk episode on poor incentives in research: http://www.econtalk.org/archives/2012/09/nosek_on_truth.html http://www.econtalk.org/archives/2012/09/nosek_on_truth.html Much of this research is funding through the NIH, which has a $30B budget. I have to believe there are tremendous politics in that.