3 ms·

Having personally read through a large number of grant applications I would say 25% is about accurate. It really doesn’t change anything if we were to raise the

Having personally read through a large number of grant applications I would say 25% is about accurate. It really doesn’t change anything if we were to raise the threshold to say the top 50%. The key to success is to remove the human element of trying to determine what is a top 5% grant from a top 10% grant. This has been proven to be impossible. You can stop the grant stuffing gaming by limiting the number of grants people can apply for every year.

evandijk70 6y ago
Don't forget that the people writing the grants are heavily influenced by what is required to obtain a grant. If the aim of writing a grant becomes to not be in the bottom 25%, than the average quality of grants will go down. I think it has not been proven to be impossible to determine the difference between a top 5% and a top 10% grant, it has just been proven that the current system does not do that. In trainings for grant applications it is often mentioned that members of grant committees spend extremely little time per grant proposal, which I imagine is just sufficient to weed out the bottom 25% proposals (those that contain obvious flaws)
- danieltillett 6y ago
  It doesn’t matter if the average quality of the grants goes down (it would actually be a good thing if it reflected less time being spent on grant writing), what matters if the average quality of the science goes up. Grants are a means to an end, not an end in themselves (administrators feel different about this I will admit). It has been proven that grant reviewers can’t agree amongst themselves what are the top 5% verses the top 10%. Having personally gone through the process as a reviewer I can identify the difference between a good and a bad grant, but I can’t with any reliability determine the difference between a top 5% and a top 10% grant. Once you get into the top 25% of grants the ranking gets very arbitrary. I might like one grant more than another purely because I like the area more, but my taste should not be the criteria on which grants are chosen. The crazy thing is all of this is known by everyone involved in the grant process yet everyone continues on as though the system is working.

evandijk70 6y ago

Don't forget that the people writing the grants are heavily influenced by what is required to obtain a grant. If the aim of writing a grant becomes to not be in the bottom 25%, than the average quality of grants will go down. I think it has not been proven to be impossible to determine the difference between a top 5% and a top 10% grant, it has just been proven that the current system does not do that. In trainings for grant applications it is often mentioned that members of grant committees spend extremely little time per grant proposal, which I imagine is just sufficient to weed out the bottom 25% proposals (those that contain obvious flaws)

danieltillett 6y ago
It doesn’t matter if the average quality of the grants goes down (it would actually be a good thing if it reflected less time being spent on grant writing), what matters if the average quality of the science goes up. Grants are a means to an end, not an end in themselves (administrators feel different about this I will admit). It has been proven that grant reviewers can’t agree amongst themselves what are the top 5% verses the top 10%. Having personally gone through the process as a reviewer I can identify the difference between a good and a bad grant, but I can’t with any reliability determine the difference between a top 5% and a top 10% grant. Once you get into the top 25% of grants the ranking gets very arbitrary. I might like one grant more than another purely because I like the area more, but my taste should not be the criteria on which grants are chosen. The crazy thing is all of this is known by everyone involved in the grant process yet everyone continues on as though the system is working.