3 ms·

Over the past decade I've sat on 10+ NIH study sections, and chaired a few. This commentary is spot on. Consider the response to a flaw in a grant. For a newer

Over the past decade I've sat on 10+ NIH study sections, and chaired a few. This commentary is spot on. Consider the response to a flaw in a grant. For a newer investigator or one no one has heard of, there will be much hemming and hawing about concern about ability to execute the project. This is code for, "who is this person in our sandbox anyway?" For one of us older guys (yes, unfortunately the pronoun is mostly right), the response is, "but I'm sure he can get this done, he must have thought about this." Even a modest scoring 'bonus' for new investigators doesn't correct this bias. On the novelty front, reviewers are trained to focus on finding flaws or absence of detail or preliminary data. The presence of any of these requires that we score down - so it's just about impossible for something truly new to get funded. Until we do the project, we don't necessarily know what problems we'll encounter or what the data will look like. The small number of NIH grants aimed at funding high-risk work by earlier-career folks (eg, the DP2 mechanism) are a great development but a tiny fraction of the portfolio. The key thing to recognize is that any move to distribute funding towards younger folks or higher-risk work causes the old guard to scream bloody murder - "what do you mean I can't have 5 R01's at a time?". For study sections, I've observed first hand that old habits favoring established investigators and low-risk, incremental work are hard to break.

sharemywin 9y ago
could you explain 5 R01's?
- cassowary37 9y ago
  sure - they're the most common NIH grant mechanism - generally 4 or 5 years @ up to $500k/year (though they can be smaller, which NIH program officers always appreciate). If you plot the number of NIH grants per investigator, there are a small but significant number of investigators out in the right tail with 4+ R01's. In general grants beget grants, such that bigger and better-funded labs are better positioned to get the next grant. Details re R01 mechanism are here [1]. A recent proposal that would cap the number of grants an investigator could lead at one time was shot down by the old guard [2]. NIH continues to struggle for alternatives, because unless the overall budget grows, which it will not in the foreseeable future, any strategy will involve taking from the rich(er) and giving to the poor(er). [1] https://grants.nih.gov/grants/funding/r01.htm https://grants.nih.gov/grants/funding/r01.htm [2] http://www.sciencemag.org/news/2017/06/updated-nih-abandons-controversial-plan-cap-grants-big-labs-creates-new-fund-younger http://www.sciencemag.org/news/2017/06/updated-nih-abandons-...
  - neuromantik8086 9y ago
    Also for those who are curious, past and present NIH grants (e.g., grant names, numbers, summaries, award amounts, principal investigators) can be queried here: https://projectreporter.nih.gov/reporter.cfm https://projectreporter.nih.gov/reporter.cfm
dekhn 9y ago
By the way, I'm one of those people whose grants you turned down (well, I don't know if it was your study section). After several attempts at getting R01s with smart new ideas and receiving no feedback and no scores, and seeing the same proposals being funded a few years later for more experienced applicants, I reasoned that I could get more done in industry. I left for Google, and got more done in 20% time (research-wise) than I ever did using 100% time in academia. When I did sit on study sections, I worked hard to help NIH understsand why the older groups asking for closet clusters on a cloud grant weren't helping. But NIH is very slow to move- it took at least 7 years to get to the point where people could even apply for cloud credits instead of closet clusters.
- cli 9y ago
  Is your current research done during that 20% time theoretical work (in biology)?
  - dekhn 9y ago
    Mostly theoretical, see https://research.google.com/pubs/pub41893.html https://research.google.com/pubs/pub41893.html although I have built a high throughput microscope for computer vision experiments.
ssivark 9y ago
You make a very good point. As PG says explains in [1], in a regime dominated by tail events, if most attempts are a moderate success (and no black swan successes), then the portfolio was probably too conservative. On the other hand, due to limited resources, I get the feeling that most academics are operating in the risk-averse mode. See, for eg [2] -- it seems to be solid advice for succeeding in academia (I'd like to hear counterpoints to that from people who've had an extended/successful academic career), but it seems wholly focused on small predictable improvements. I wonder whether this problem is a consequence of enforcing "accountability" in a regime dominated by fat tails. Maybe we've picked up all the low-hanging fruit in research and we can only hope for slow incremental progress (which is what the management structure is set up to optimize), and we need to come to terms with that and stop wistful discussions about not having black swan successes. [1]: http://paulgraham.com/swan.html http://paulgraham.com/swan.html [2]: http://www.stochasticlifestyle.com/tips-google-summer-code-application/ http://www.stochasticlifestyle.com/tips-google-summer-code-a...
rhombocombus 9y ago
That was my experience, and in retrospect it took my graduate advisor a decade to land an R01 based on the research I did while I was in school. She is a mid career tenured faculty at a decent school. Her husband is an endowed professor at a research one school and they practically call him on the phone to give him money (That is total hyperbole, but he lands funding at a much higher rate in spite of the fact that both have successful research programs). It seems really difficult to break into the sphere of 'successful PI' when so mnany other successful PIs win the majority of the funding. It was one of the reasons I left academia. Not the only reason, but it figured into it.

sharemywin 9y ago

could you explain 5 R01's?

cassowary37 9y ago
sure - they're the most common NIH grant mechanism - generally 4 or 5 years @ up to $500k/year (though they can be smaller, which NIH program officers always appreciate). If you plot the number of NIH grants per investigator, there are a small but significant number of investigators out in the right tail with 4+ R01's. In general grants beget grants, such that bigger and better-funded labs are better positioned to get the next grant. Details re R01 mechanism are here [1]. A recent proposal that would cap the number of grants an investigator could lead at one time was shot down by the old guard [2]. NIH continues to struggle for alternatives, because unless the overall budget grows, which it will not in the foreseeable future, any strategy will involve taking from the rich(er) and giving to the poor(er). [1] https://grants.nih.gov/grants/funding/r01.htm https://grants.nih.gov/grants/funding/r01.htm [2] http://www.sciencemag.org/news/2017/06/updated-nih-abandons-controversial-plan-cap-grants-big-labs-creates-new-fund-younger http://www.sciencemag.org/news/2017/06/updated-nih-abandons-...
- neuromantik8086 9y ago
  Also for those who are curious, past and present NIH grants (e.g., grant names, numbers, summaries, award amounts, principal investigators) can be queried here: https://projectreporter.nih.gov/reporter.cfm https://projectreporter.nih.gov/reporter.cfm

dekhn 9y ago

By the way, I'm one of those people whose grants you turned down (well, I don't know if it was your study section). After several attempts at getting R01s with smart new ideas and receiving no feedback and no scores, and seeing the same proposals being funded a few years later for more experienced applicants, I reasoned that I could get more done in industry. I left for Google, and got more done in 20% time (research-wise) than I ever did using 100% time in academia. When I did sit on study sections, I worked hard to help NIH understsand why the older groups asking for closet clusters on a cloud grant weren't helping. But NIH is very slow to move- it took at least 7 years to get to the point where people could even apply for cloud credits instead of closet clusters.

cli 9y ago
Is your current research done during that 20% time theoretical work (in biology)?
- dekhn 9y ago
  Mostly theoretical, see https://research.google.com/pubs/pub41893.html https://research.google.com/pubs/pub41893.html although I have built a high throughput microscope for computer vision experiments.

ssivark 9y ago

You make a very good point. As PG says explains in [1], in a regime dominated by tail events, if most attempts are a moderate success (and no black swan successes), then the portfolio was probably too conservative. On the other hand, due to limited resources, I get the feeling that most academics are operating in the risk-averse mode. See, for eg [2] -- it seems to be solid advice for succeeding in academia (I'd like to hear counterpoints to that from people who've had an extended/successful academic career), but it seems wholly focused on small predictable improvements. I wonder whether this problem is a consequence of enforcing "accountability" in a regime dominated by fat tails. Maybe we've picked up all the low-hanging fruit in research and we can only hope for slow incremental progress (which is what the management structure is set up to optimize), and we need to come to terms with that and stop wistful discussions about not having black swan successes. [1]: http://paulgraham.com/swan.html http://paulgraham.com/swan.html [2]: http://www.stochasticlifestyle.com/tips-google-summer-code-application/ http://www.stochasticlifestyle.com/tips-google-summer-code-a...

rhombocombus 9y ago

That was my experience, and in retrospect it took my graduate advisor a decade to land an R01 based on the research I did while I was in school. She is a mid career tenured faculty at a decent school. Her husband is an endowed professor at a research one school and they practically call him on the phone to give him money (That is total hyperbole, but he lands funding at a much higher rate in spite of the fact that both have successful research programs). It seems really difficult to break into the sphere of 'successful PI' when so mnany other successful PIs win the majority of the funding. It was one of the reasons I left academia. Not the only reason, but it figured into it.