4 ms·

This misses the point that grant agencies are rubbish at funding great ideas. And also that some scientists shamelessly oversell their ideas and even falsify da

by Gatsky 9y ago

This misses the point that grant agencies are rubbish at funding great ideas. And also that some scientists shamelessly oversell their ideas and even falsify data - behaviour which is of course encouraged by the system of perverse incentives you advocate.

hyperbovine 9y ago
This reminds of that Churchill quote about democracy: "the worst form of government, except for all those other forms that have been tried." Besides competitive grant awards, do you have a better mechanism for reliably funding brilliant ideas? I doubt it. I agree the system could be improved in certain ways, like tilting the funding stream less heavily towards older PIs, and funding a few moonshots, both problems the NIH at least claims to be addressing now. Also, some scientists do overhype their ideas, and a very few might make up data, but the majority do neither.
- Gatsky 9y ago
  The problem is of course your argument justifies any incumbent system no matter how heinous. I don't think the analogy holds up. And I don't think I am required to present a plan to revolutionise research funding in a comment on HN.
  - tertius 9y ago
    I don't think you're required to no, but you can at least attempt an answer or cite someone else that has put thought into an alternative. And just because you fail at your attempt does not make your initial point void.
  - hyperbovine 9y ago
    If the current system is so "heinous" then seemingly that should not be very difficult.
tertius 9y ago
Yes, but they're at least adequate at funding good science. That is all that they are attempting. I did mention that there are people with salesmanship and that's an advantage that you really can't control for. Well except through scrutiny of only the science, which seems to be fairly well done through a group of scientists who have to approve proposals. "Oversell ideas and falsify data": Of course a competitive market will encourage these behaviors. But just because there are unethical people in science doesn't mean that we should do away with the system. I would add to "falsify data", misinterpret and misunderstand data (and specifically the statistics involved). Again, something you can control for to some degree with scrutiny of the applications by other good scientists.
- Gatsky 9y ago
  I don't know about adequate - there are lamentations about science funding from young and old researchers alike, all the time. Many breakthrough areas struggled along for years with little funding, while a ton of money went into dead ends. Eg immunotherapy in cancer, H. pylori as the major cause of gastric ulcers... Any system you implement creates incentives, these need to be properly aligned to ensure an effective and humane system. If your system encourages or rewards unethical behaviour that is a big problem. Having applied for and reviewed grants, the idea of 'good scientists' scrutinising a proposal carefully and controlling for all these factors is a bit fantastical. Maybe in your area it's different.
  - tertius 9y ago
    > Many breakthrough areas struggled along for years with little funding, while a ton of money went into dead ends. Eg immunotherapy in cancer, H. pylori as the major cause of gastric ulcers... This looks an awful lot like survivor bias. Tons of money was also put into well established research to move it along. And ideas that were stupid also didn't get any funding and struggled. > If your system encourages or rewards unethical behaviour that is a big problem. That's markets and that's why we have a justice system. A reason why the NIH now requires grant applications to be of publishable quality to help diminish completely turnaround in experimental design after grant have been issued. A part of the evolution that I speak of. > the idea of 'good scientists' scrutinising a proposal carefully and controlling for all these factors is a bit fantastical. I doubt you want to do away with peer review. I'll be the first to say that It's not perfect, but it's gotten us pretty far. "Good scientists" have their orthodoxy and "know" certain things to be to true and won't (most of the time) fund anything that goes against that orthodoxy. I.e. one funeral at a time. Well funded PIs have the ability to fund this type of science and many do. I think we're pretty much in agreement though. The system is slow, not great, but like another commenter suggested, the best we can do.

by Gatsky 9y ago

hyperbovine 9y ago

This reminds of that Churchill quote about democracy: "the worst form of government, except for all those other forms that have been tried." Besides competitive grant awards, do you have a better mechanism for reliably funding brilliant ideas? I doubt it. I agree the system could be improved in certain ways, like tilting the funding stream less heavily towards older PIs, and funding a few moonshots, both problems the NIH at least claims to be addressing now. Also, some scientists do overhype their ideas, and a very few might make up data, but the majority do neither.

Gatsky 9y ago
The problem is of course your argument justifies any incumbent system no matter how heinous. I don't think the analogy holds up. And I don't think I am required to present a plan to revolutionise research funding in a comment on HN.
- tertius 9y ago
  I don't think you're required to no, but you can at least attempt an answer or cite someone else that has put thought into an alternative. And just because you fail at your attempt does not make your initial point void.
- hyperbovine 9y ago
  If the current system is so "heinous" then seemingly that should not be very difficult.

tertius 9y ago

Yes, but they're at least adequate at funding good science. That is all that they are attempting. I did mention that there are people with salesmanship and that's an advantage that you really can't control for. Well except through scrutiny of only the science, which seems to be fairly well done through a group of scientists who have to approve proposals. "Oversell ideas and falsify data": Of course a competitive market will encourage these behaviors. But just because there are unethical people in science doesn't mean that we should do away with the system. I would add to "falsify data", misinterpret and misunderstand data (and specifically the statistics involved). Again, something you can control for to some degree with scrutiny of the applications by other good scientists.

Gatsky 9y ago
I don't know about adequate - there are lamentations about science funding from young and old researchers alike, all the time. Many breakthrough areas struggled along for years with little funding, while a ton of money went into dead ends. Eg immunotherapy in cancer, H. pylori as the major cause of gastric ulcers... Any system you implement creates incentives, these need to be properly aligned to ensure an effective and humane system. If your system encourages or rewards unethical behaviour that is a big problem. Having applied for and reviewed grants, the idea of 'good scientists' scrutinising a proposal carefully and controlling for all these factors is a bit fantastical. Maybe in your area it's different.
- tertius 9y ago
  > Many breakthrough areas struggled along for years with little funding, while a ton of money went into dead ends. Eg immunotherapy in cancer, H. pylori as the major cause of gastric ulcers... This looks an awful lot like survivor bias. Tons of money was also put into well established research to move it along. And ideas that were stupid also didn't get any funding and struggled. > If your system encourages or rewards unethical behaviour that is a big problem. That's markets and that's why we have a justice system. A reason why the NIH now requires grant applications to be of publishable quality to help diminish completely turnaround in experimental design after grant have been issued. A part of the evolution that I speak of. > the idea of 'good scientists' scrutinising a proposal carefully and controlling for all these factors is a bit fantastical. I doubt you want to do away with peer review. I'll be the first to say that It's not perfect, but it's gotten us pretty far. "Good scientists" have their orthodoxy and "know" certain things to be to true and won't (most of the time) fund anything that goes against that orthodoxy. I.e. one funeral at a time. Well funded PIs have the ability to fund this type of science and many do. I think we're pretty much in agreement though. The system is slow, not great, but like another commenter suggested, the best we can do.