3 ms·

Not just a lottery, but it feels like randomness should be part of the process. The mechanisms and people we have in place to make decisions are horribly imperf

by tikhonj 11y ago

Not just a lottery, but it feels like randomness should be part of the process. The mechanisms and people we have in place to make decisions are horribly imperfect, so there's a lot of noise in the system as is. Many perfectly good grants don't get funded, often for unclear reasons. Adding randomness past a certain minimum bar just makes that noise explicit and ensures that it's not biased or spuriously correlated. This would also deemphasize the importance of narrow skills specific to writing and getting grants. Right now, understanding the grant process and granting agency is probably just as important as, you know, coming up with good proposals. Not ideal! Among other things, this could also help academics to spread out a bit—right now, research agendas are extremely fashion-driven, in large part because of how grants are awarded. Think about it like a search algorithm which, with some probability, randomly jumps out of its current state as a way to avoid getting stuck in local maxima. Originally, I thought of this idea as applied to college admissions. Harvard likes to talk about how it could assemble a few classes of qualified students each year, but only has room for one. At that point, it's distinctly non-obvious that the process they use to choose between similarly qualified candidates is any good; some actual randomness could have benefits. But this idea feels even more applicable to research grants.

danieltillett 11y ago
Great post. One thing about making the process (more) random is it would encourage people to take risks. When I was an academic I had to spend most of my time working on relatively low risk projects that I knew would generate results (and get me publications) or else I knew I would not get anymore grants. By removing this need I could have afforded to take risks that might have resulted in real breakthroughs. We need scientists to take risks and make the great discoveries.
leoc 11y ago
> Originally, I thought of this idea as applied to college admissions. Harvard likes to talk about how it could assemble a few sets of qualified students each year, and has to choose one. At that point, it's distinctly not obvious that the process they use to choose between similarly qualified candidates is any good; some actual randomness could have some benefits. The only problem is that there's no sign that elite universities have any intention of choosing students on merit, or through any system which is blind to factors other than merit (plus pure chance). Harvard's selection system seems to be very good at its actual intended purpose: it's more like casting Glee than making a strict assessment of merit. But for the rare university which both has a large oversupply of very good applicants and actually wishes to operate admissions on merit then yes, randomness is the answer. Another place that it would seem to make sense to apply randomness is in performance-related pay.

by tikhonj 11y ago

danieltillett 11y ago

Great post. One thing about making the process (more) random is it would encourage people to take risks. When I was an academic I had to spend most of my time working on relatively low risk projects that I knew would generate results (and get me publications) or else I knew I would not get anymore grants. By removing this need I could have afforded to take risks that might have resulted in real breakthroughs. We need scientists to take risks and make the great discoveries.

leoc 11y ago

> Originally, I thought of this idea as applied to college admissions. Harvard likes to talk about how it could assemble a few sets of qualified students each year, and has to choose one. At that point, it's distinctly not obvious that the process they use to choose between similarly qualified candidates is any good; some actual randomness could have some benefits. The only problem is that there's no sign that elite universities have any intention of choosing students on merit, or through any system which is blind to factors other than merit (plus pure chance). Harvard's selection system seems to be very good at its actual intended purpose: it's more like casting Glee than making a strict assessment of merit. But for the rare university which both has a large oversupply of very good applicants and actually wishes to operate admissions on merit then yes, randomness is the answer. Another place that it would seem to make sense to apply randomness is in performance-related pay.