4 ms·

I don't find the dismissal of the Matthew Effect ('the rich get richer') even vaguely convincing here. The Bol (2018) paper shows that scientists that are just

I don't find the dismissal of the Matthew Effect ('the rich get richer') even vaguely convincing here. The Bol (2018) paper shows that scientists that are just above the cut-off to receive an early career grant tend to do much better in their subsequent career than those just below it. Both the paper and blogpost explain this away as a "participation effect": the winners end up applying for many more grants, while the non-winners "cease" doing so. However, it seems disingenuous to frame this as an issue of motivation or encouragement without thinking about why else this could happen. In many cases, a scientist needs to bring in grant money to keep their job. There's a very clear causal chain between <no grant> -> <no money> -> <no job>. Less severely, it also takes (grant) money to make (grant) money. Most funding agencies want to see preliminary data, which takes money (and staff) to collect. They also want to see evidence of productivity and 'excellence', which is again dependent on money; in some cases, bringing in money is even taken as prima face evidence of those qualities! In the previous section, the author also discusses how the "top scientists train future top scientists." Some of this may be due to their ability to select the most talented future researchers from a pool of 22 year olds, but it beggars belief that there aren't network effects here too on top of any improvement in training. For example, big well-funded labs have access to better equipment and technical support, which allows their members to fully express their scientific abilities. There are more opportunities for synergy and collaboration than in a small group. Since top scientists are frequently invited to give talks, their work is better advertised, with knock-on effects on citations. Recommendations and letters of support are vital for hinting, grants, and tenure too.

derbOac 6y ago
> in some cases, bringing in money is even taken as prima face evidence of those qualities! I can't tell how playful you're being, but in case it's not clear, in my experience at most R01 institutions, this is the norm, not the exception. I have so many problems with this piece and its line of thinking I don't even know where to start. The current academic system is characterized by huge gap between reality and the benchmarks that most attempts to try to evaluate theories about academic success and support fail from the get-go because they're so oversimplified. The piece basically adopts this standard paradigm where you have researchers just doing their thing, publishing, putting out into the world, and all you have to do is "read out" their output. The framing of it, in terms of "the distribution of talent", while meaningful at some level, completely ignores the context any given researcher finds themselves in, in terms of institutional culture, support, corruption, everything. There are metrics, and they are gamed in ways that are hard to even start to ennumerate. To take the Li and Agha paper discussed later in the piece: the author notes that "the paper tries to control for prestige by using dummy variables for having a PhD, an MD, or both, and how many years a PI has been in academia" before admitting "these are highly imperfect as proxies of prestige." This is an absurd understatement. In my experience, "prestige" is an extremely diffuse and elusive thing that has more to do with social networking than institutional prestige per se. Previous papers have shown, for example, that one of the biggest predictors of grant receipt is having previously published with people on the review panels. This alone should be striking to anyone, but it points to how this works: it's not superficial markers of prestige necessarily like US News rankings, but subtle things like social associations etc. In recent years there's been a certain amount of attention to sexism in academics of various forms. Putting sexism per se aside, it's nevertheless an example of a social process that has nothing to do with academic talent per se that neverthless can have huge impacts on academic careers. For every social-environmental variable like sexism, there's some other variable operating on some other person characteristics, like age, race, other social background etc. I could go on and on. I think it's telling that the vast majority of the work cited in the piece is pre-2005 (all but one paper?). I would say the academic environment before, say, 2000 is almost inapplicable to today in many respects, and even the period 2000-2005 is very different.
- cycomanic 6y ago
  Extremely insightful comment. I agree with you I had very similar feelings as you when reading the article. Essentially I feel the article is so off in its assumptions that it is hard to criticise except for saying "it's rubbish" and I thank you for pointing to some of the specifics that are wrong with it.
- e-_pusher 6y ago
  What do you think are the main differences between the pre-2000 academia and the academia now?
  - mattkrause 6y ago
    The NIH budget had been relatively flat until 1998-2003, when it literally doubled. This caused a sea-change in how biomedical research was organized. One might naively expect that a massive injection of money would only make research easier. However, universities reacted by expanding dramatically. Many of those new hires, especially at med schools, are on "soft money" contracts, where they're responsible for bringing in enough external grant money to cover their own salary AND research expenses. As a result, it has never been harder to get a research grant: success rates that were >30% have slipped into the low teens at some institutes. At the same time, the size of the standard "modular" grant at the NIH hasn't changed, so people are competing harder to get less inflation-adjusted money. On top of that, science is getting harder as we discover the easy stuff and expectations are rising for what constitutes "a story." In short, post-doubling science isn't a professor pottering around between classes; it's become a big, competitive business.
thewarrior 6y ago
At the risk of being controversial if you accept the premises of this article you have to concede that most scientific progress comes from rich white men and that there is no point in funding the rest. The way things are isn’t a good source of information.
- mattkrause 6y ago
  But I don't accept the premise of the article, and that's why. Heck, you could have said the same thing about baseball or basketball pre-1950. Is it more likely that.... A) Rich, white men have some innate and preternatural ability to form and test hypotheses, or... B) Like literally every other human endeavor, apparent scientific ability also heavily depends on the means and opportunity to express it, public awareness and acceptance of the results, and a dash of luck?
  - thewarrior 6y ago
    I don’t accept it either. It’s concerning to see such thoughts in the upper echelons of power.
ampdepolymerase 6y ago
One way to fix this is to completely break the apprenticeship model of higher academia and professional careers. For Law, Medicine, and any other field requiring significant time in the classroom, ensure that accreditation is state administered (or outsourced test like the SAT) with no requirement of time spend in the classroom. This breaks and limits the prestige factor conferred by institution and reduce their overall influence. The way software engineers are hired, while flawed, remains one of the most meritocratic ways of hiring and distributing funding.

derbOac 6y ago

> in some cases, bringing in money is even taken as prima face evidence of those qualities! I can't tell how playful you're being, but in case it's not clear, in my experience at most R01 institutions, this is the norm, not the exception. I have so many problems with this piece and its line of thinking I don't even know where to start. The current academic system is characterized by huge gap between reality and the benchmarks that most attempts to try to evaluate theories about academic success and support fail from the get-go because they're so oversimplified. The piece basically adopts this standard paradigm where you have researchers just doing their thing, publishing, putting out into the world, and all you have to do is "read out" their output. The framing of it, in terms of "the distribution of talent", while meaningful at some level, completely ignores the context any given researcher finds themselves in, in terms of institutional culture, support, corruption, everything. There are metrics, and they are gamed in ways that are hard to even start to ennumerate. To take the Li and Agha paper discussed later in the piece: the author notes that "the paper tries to control for prestige by using dummy variables for having a PhD, an MD, or both, and how many years a PI has been in academia" before admitting "these are highly imperfect as proxies of prestige." This is an absurd understatement. In my experience, "prestige" is an extremely diffuse and elusive thing that has more to do with social networking than institutional prestige per se. Previous papers have shown, for example, that one of the biggest predictors of grant receipt is having previously published with people on the review panels. This alone should be striking to anyone, but it points to how this works: it's not superficial markers of prestige necessarily like US News rankings, but subtle things like social associations etc. In recent years there's been a certain amount of attention to sexism in academics of various forms. Putting sexism per se aside, it's nevertheless an example of a social process that has nothing to do with academic talent per se that neverthless can have huge impacts on academic careers. For every social-environmental variable like sexism, there's some other variable operating on some other person characteristics, like age, race, other social background etc. I could go on and on. I think it's telling that the vast majority of the work cited in the piece is pre-2005 (all but one paper?). I would say the academic environment before, say, 2000 is almost inapplicable to today in many respects, and even the period 2000-2005 is very different.

cycomanic 6y ago
Extremely insightful comment. I agree with you I had very similar feelings as you when reading the article. Essentially I feel the article is so off in its assumptions that it is hard to criticise except for saying "it's rubbish" and I thank you for pointing to some of the specifics that are wrong with it.
e-_pusher 6y ago
What do you think are the main differences between the pre-2000 academia and the academia now?
- mattkrause 6y ago
  The NIH budget had been relatively flat until 1998-2003, when it literally doubled. This caused a sea-change in how biomedical research was organized. One might naively expect that a massive injection of money would only make research easier. However, universities reacted by expanding dramatically. Many of those new hires, especially at med schools, are on "soft money" contracts, where they're responsible for bringing in enough external grant money to cover their own salary AND research expenses. As a result, it has never been harder to get a research grant: success rates that were >30% have slipped into the low teens at some institutes. At the same time, the size of the standard "modular" grant at the NIH hasn't changed, so people are competing harder to get less inflation-adjusted money. On top of that, science is getting harder as we discover the easy stuff and expectations are rising for what constitutes "a story." In short, post-doubling science isn't a professor pottering around between classes; it's become a big, competitive business.

thewarrior 6y ago

At the risk of being controversial if you accept the premises of this article you have to concede that most scientific progress comes from rich white men and that there is no point in funding the rest. The way things are isn’t a good source of information.

mattkrause 6y ago
But I don't accept the premise of the article, and that's why. Heck, you could have said the same thing about baseball or basketball pre-1950. Is it more likely that.... A) Rich, white men have some innate and preternatural ability to form and test hypotheses, or... B) Like literally every other human endeavor, apparent scientific ability also heavily depends on the means and opportunity to express it, public awareness and acceptance of the results, and a dash of luck?
- thewarrior 6y ago
  I don’t accept it either. It’s concerning to see such thoughts in the upper echelons of power.

ampdepolymerase 6y ago

One way to fix this is to completely break the apprenticeship model of higher academia and professional careers. For Law, Medicine, and any other field requiring significant time in the classroom, ensure that accreditation is state administered (or outsourced test like the SAT) with no requirement of time spend in the classroom. This breaks and limits the prestige factor conferred by institution and reduce their overall influence. The way software engineers are hired, while flawed, remains one of the most meritocratic ways of hiring and distributing funding.