3 ms·
If you could systematically predict Karikó's, they would not be Karikó's. For all we know, any attempt to "solve" this problem might just be a fool's errand.
by zozbot234 5y ago
If you could systematically predict Karikó's, they would not be Karikó's. For all we know, any attempt to "solve" this problem might just be a fool's errand. At least in the absence of a meaningful argument to the contrary.
> After enough filtering, all grants are essentially indistinguishable in quality.
I don't believe this. You can always soften incentives (and thereby prevent wasteful gaming) by adding random noise/dithering to your scoring function, but a threshold-only approach seems quite blunt to me.
- derbOac 5y agoActually, this model has been suggested by former heads of NIH and NSF, who have publicly expressed concern about the current funding paradigm. The problem is that impact by reasonable metrics is correlated about 0.30 with grant score, so it's difficult to predict what will be successful. (Note that impact itself is controversial, given self-fulfilling fad dynamics in research -- FOMO popularity spikes and so forth). One of the biggest predictors of grant success is having previously published with people on the review committees. The idea is that you randomly fund research because it's difficult to know what will be important, but try to balance that against grants that might be poorly conceived to begin with.
- cycomanic 5y ago> > After enough filtering, all grants are essentially indistinguishable in quality. > I don't believe this. You can always soften incentives (and thereby prevent wasteful gaming) by adding random noise/dithering to your scoring function, but a threshold-only approach seems quite blunt to me. These assessments come from statistically investigating grant scoring and success. What you find that 90%of grants are essentially scored the same, within the noise. In these situations what typically happens is that previous grant success will determine current success, i.e. the person previously got a grant so what they are doing must be good (the "safe" bet). The problem that creates that you create empires of people who get a lot of grants based essentially on one luck of the draw. The biggest issue is though not even that you miss some good people with good ideas, but the incentive structure you create. Because there is an expiry time on the benefit from previous success and it is research area specific, what happens is academics have to write grants constantly and you can't really change fields. Academics spend 30% of their time applying for and reviewing grants. None (hardly any) of the people who are very successful with getting grants are doing research themselves any more. I suspect this is also the reason why most Nobel winning research is done in people's 30s.more senior academics don't have time to do the research (and can't take the necessary risks).