23 ms·
Negative incentives in academic research
- mpweiher 4y agoIt goes beyond grants and extrinsic motivation...well maybe the latter category is pretty much all-encompassing. Turing Award winner Michael Stonebraker talked about this at length here: https://www.youtube.com/watch?v=DJFKl_5JTnA https://www.youtube.com/watch?v=DJFKl_5JTnA Some points: - Original (risky) research is detrimental and likely fatal to a career in science - Actually creating systems doubly so
- Tostino 4y agoWhat timestamp on that talk if you don't mind?
- psi75 4y ago[flagged]
- pfortuny 4y agoPlease: do not assign the cause of something toa group of anonymous and unverifiable people. This always end badly. Always. And there are more non-graduate boomers than graduate ones.
- psi75 4y ago[flagged]
- pfortuny 4y ago"Their generation's name has become..." is (and pardon my insistence) what racist people have said about blacks forever. "Black people's name is synonymous with theft", for instance. That is why those kind of expressions in an abstract and intellectual (hopefully, here in HN) setting do harm.
- gspr 4y ago> In 2022, evaluating original research is harder than ever, due to hyperspecialization. For a typical Ph.D. thesis, there are less than five people in the world qualified to evaluate whether it's worthy of a tenure-track position. That's simply not true for a typical PhD thesis at all. It certainly doesn't jive with my own experience, so I'd like to cough up a few examples please. (Do hyperspecialized PhD theses like the one you describe exist? Certainly! But I refute the idea that such these are typical.)
- quantum_mcts 4y agoI generally disagree with the premise that scientists are becoming less productive. (To me, the complaints that there are no more grand fundamental discoveries is alike to complaining that we are not discovering any new continents on Earth...) On the subject of incentives in academia - reflecting on my academia run, I've noticed that I was the most creatively productive when I had a longer planning horizon in front of me. It was either at the beginning of a several-years position. Or at the very end when I knew that I'm leaving and didn't care anymore. The least productive was the sequence of one-year postdocs - when I was constantly worrying about the next one.
- simonsarris 4y ago> I generally disagree with the premise that scientists are becoming less productive. Do you believe the average academic research paper or project written/done today is of the same quality or better as the average in 1980, or 1950?
- matthewdgreen 4y agoI can't speak to all science, but I am a CS (cryptography) researcher and routinely read papers from the 1980s. The level of rigor and quality of my field's papers has absolutely improved, by leaps and bounds. The formal definitions in those early papers are often non-existent (and sometimes wrong in retrospect) and the proposed constructions are often much simpler (and sometimes subtly wrong in retrospect.) And the number of papers has increased by at least an order of magnitude. On the flip side, those early papers contain the most fundamental discoveries in our field: you're only going to invent RSA or blind signatures or zero knowledge once. It's possible all those researchers were much smarter than we are now. (I grant this!) But there are a lot of absolutely brilliant people I know today. Alternatively, the lower-hanging fruit is all gone and the problems have become much harder.
- amelius 4y ago> Alternatively, the lower-hanging fruit is all gone and the problems have become much harder. Reminds me of a patent officer who said in 1899 that "everything that can be invented has been invented."
- crikeyjoe 4y ago
- psi75 4y agoMan, I wish I could use my real name for this reply, but for political reasons I can't (in fact, I change my username regularly) because I have a lot of insight into this problem, why it exists, and why it probably won't get better barring a complete overhaul of our socioeconomic system and the myriad corrupt institutions that support it. We live in an age of institutional decline and it is severe. You see this in (trade) publishing. Your publisher no longer builds your reputation; the publisher has pushed that responsibility unto the author. The ones who already have the personal resources necessary to market their books get further validation and credibility; the ones who don't will go unheard. Academia's the same way: universities no longer provide funding for people of excellence; rather, they have put the onus of funding on the professors themselves--you'll get more funding (and published in better places) on account of using their name, and for that they take a cut. The relationship has inverted; rather than nurturing emerging talent, these institutions are nurtured by emerging talent, and this vampirism is sustainable because those talented people have no other choice insofar as all the other institutions are failing at approximately the same right. Consequently, we have widespread duplicated effort, channel-flooding due to metrics-gaming (gotta get that h-index into the three digits before tenure time) and a corporatized, mediocre culture in which agreeability (negatively correlated with excellence and conscientiousness) matters far too much and salesmen run the day. Historically, there were nations of priests and nations of warriors and nations of farmers. We've become a nation of sellers; but we no longer have much to sell but our own talk. I don't know, for sure, how to solve this. Anyone who pays attention can see that capitalism (which invariably becomes corporate capitalism) is a dead end at a 21st-century technology level... but of course the eradication of capital is merely a necessary, not a sufficient, condition for scientific excellence. Going socialist is mandatory if we want to fix this, but alone does not guarantee much--there are a lot of cultural changes that probably need to happen before we can build healthy institutions again.
- fnovd 4y ago>Going socialist is mandatory if we want to fix this Of course, the lede is buried exactly where expected. How novel an idea it is that the the system is separable from its components and that utopia is indeed achievable if only the rest of the world would wake up and learn to serve your idea of a higher purpose. The inevitable counterpoint is that the system is responding predictably and efficaciously to a change to its environment: more undergraduate funding leads to more students pursuing graduate degrees and a greater deal of competition in the academic space. The "problem" is self-correcting. It turns out science doesn't need happy scientists. The reason why you don't attach your name to comments like this is because your possession of a name worth attaching would preclude this comment from existing.
- ISL 4y agoI agree with point #1, but disagree with point #2. Academic research is not a place to go for extrinsic motivation. If you're not primarily in it for love, you're going to have a bad time. An example: Who has heard of the Wolf Prize in physics? It is in the set of prizes known as "second prize to the Nobel Prize", yet essentially nobody outside of the field will react with, "Whoa, she won a Wolf Prize? She must be really good." Even in the well-resourced research groups, researchers will be find much better extrinsic reward outside of academia. Advancing the boundaries of our understanding is priceless intrinsic motivation for those who can find a way to stay, but doing so requires substantial sacrifice -- more than many of those in academia realize.
- BeetleB 4y agoExtrinsic here means "extrinsic to self", not "extrinsic to academia". There's plenty of "extrinsic" motivation within academia. When you go to top universities, you'll find plenty of academics who are extrinsically motivated: * Number of research papers * Cumulative value of grants * Awards * Titles. This was insane. Apparently being a Fellow of the IEEE wasn't enough, so they actively sought becoming Fellows in adjacent societies.
- zekrioca 4y ago> Extrinsic here means "extrinsic to self", not "extrinsic to academia". There's plenty of "extrinsic" motivation within academia. Yes, I've also understood it this way.
- jrumbut 4y agoI'm far from convinced that science is less productive as a whole than it was in 1900, though I'm sure the average scientist today isn't as productive as Max Planck (but of course he was above average in his own day as well). Something I think would help the incentive situation is providing more incentives for teaching and service. My grandfather was a professor emeritus and I can't find any record of him authoring a research paper (or a PhD thesis for that matter) but even 30+ years since he last taught I still run into people who rave about the formative experiences they had in his classes. He's a minor celebrity among local engineers of a certain age. Such a person could still provide a lot of value, but it's harder to measure.
- mmmmpancakes 4y agoThat's a nice story, but teaching =/= research and I fail to see how this is related to research productivity. If anything the relation is inverse because, as you said, he doesn't have a publication record, so from the perspective of research he was basically useless. If we are trying to improve research productivity, then removing or dramatically reducing teaching responsibilities for researchers sounds a lot more sensible. Besides, modern pedagogy is getting increasingly complicated and having teaching specialists can be much more beneficial to your average 1-2 year undergrad. Not to diminish his teaching career, it sounds very valuable, but not in a way that's directly relevant to the OP.
- jrumbut 4y agoIt's simple! Grant applications are so hard because they are so competitive. They are so competitive because a lot of people want to be professors. Currently, to succeed as a professor you need a research grant. If, instead, we made it easier to advance your career as exactly the kind of pedagogy specialist you described (as well as through leadership in professional organizations, work in the community, and all the other kinds of service), I believe you would get fewer applications to those research grants. It might result in better post-secondary education as well.
- bachmeier 4y ago> if grant applications are valued enough (e.g., needed to get promotion), scientists may be willing to spend even more time than is rational to do so This isn't worded properly. It's clearly rational for the scientists to spend time on the pursuit of grants. Otherwise they wouldn't be doing it. $100 in grant money is not the same as $100 in university income. In fact, the whole point of the grant is to cover the cost of doing research, and in principle should leave the university's net revenue unchanged. (In practice, overhead changes that, but that's a different topic.) The problem is that the university's objective is to spend as much on research as possible. Grant money is simply the fuel for that process. All else equal, more expensive research is very strongly preferred, because that increases research spending, and that improves the standing of the university. That is a seriously distorted set of incentives. All perfectly rational. Just insanely stupid.
- denton-scratch 4y ago> It's clearly rational for the scientists to spend time on the pursuit of grants. Otherwise they wouldn't be doing it. Is that worded properly? Are you really saying that nobody would ever do something irrational? If that's not what you mean, then we have to look at what "rational" means here. I think parent didn't mean "rational" in the sense of "maximising personal gain", but "maximising output of quality research".
- BrandoElFollito 4y ago> It's clearly rational for the scientists to spend time on the pursuit of grants. Otherwise they wouldn't be doing it. This is one of the reasons why I left academia. The insane run for grants was just starting and i said that I do not have any intent do do that. I was a scientist. Doing science. Not filling in papers and going to belly dancing shows. When I was told that this is compulsory and nobody else is going to do it, I sad good bye and left for the industry. I was earning 10x more, had no worries about funding or copier paper. Plenty of my friends did it and then academia was crying because of the brains drain and how unfair it was. The salme who said that grats are compulsory (they were sitting in the various jurys to accept them, instead of doing science)
- idiot900 4y agoI'm an assistant professor in a medical school in the US and agree in general with the article. It takes an absurd amount of time to write a grant application, time that could be spent doing science - but we faculty don't have any choice if we want to continue our careers.
- ray__ 4y agoI am the first grad student of an assistant professor (chemistry/biology). Seeing her writing workload has made me a lot less enthusiastic about being a professor myself. I like writing about science, but the constant treading water mentality makes it difficult to 1) find the mental space to be creative and 2) take the risks that are often necessary for innovation, especially since the careers of your trainees are on the line. The kicker is that most grant writing isn't even about science. Good luck with your lab!
- lurkervizzle 4y agoSame exact experience - I finished my CS PhD in 2005 and thought I would go into academia, but decided not to given how much of academia seemed to be just writing grant proposals vs. actual research. FWIW, also decided not to go into academia because of how much smarter I realized I needed to be to be a top tier academic!
- BurningFrog 4y agoThis part (at least) of science needs to be revolutionized. Somehow. https://fastgrants.org/ https://fastgrants.org/ did some good work for covid. Could maybe be a starting point for a new system.
- zekrioca 4y agoI agree, specially with the 2nd point. You can see this in virtually every academic institution where researchers will find all ways to simply "plot" good results, i.e., beat the data until you find what you want. The intrinsic value of the research is nearly 0, because all that matters is to get things published to attract citations that will help them in getting promotions and prizes.
- sycren 4y agoEarlier this year I was exploring a web3 industry 4.0 open science solution rethinking how research is performed, by who (or what), skills required, deliverables (no journals needed), a semantic knowledge base of results good & bad, validated through repetition, a framework for how experiments are managed. Here's my research on the 'Challenges facing Academic Research' - https://miro.com/app/board/uXjVOkNfljM=/?share_link_id=584278282036 https://miro.com/app/board/uXjVOkNfljM=/?share_link_id=58427... Quotes that I found important: "It discourages rigorous research as it is difficult to obtain enough results for a paper (and hence progress) in two to three years. The constant stress drives otherwise talented and intelligent people out of science also." - Anonymous "End the PhD or drastically change it. there is a high level of depression among phd students. long hours, limited career prospects, and low wages contribute to this emotion." - Don Gibson, Scientist at BioConsortia Funding "affects what we study, what we publish, the risks we (frequently don't) take, it nudges us to emphasise safe, predictable (read: fundable) science" - Gary Bennett - Neuroscientist at Duke University "We need to recognise academic journals for what they are: shop windows for incomplete descriptions of research, that make semi-arbitrary editorial [judgments] about what to publish and often have harmful policies that restrict access to important post-publication critical appraisal of published research." —Ben Goldacre, The Datalab, epidemiology researcher, physician, and author "An estimated $200 billion - or the equivalent of 85% of global spending on research - is routinely wasted on poorly designed and redundant studies." - https://www.thelancet.com/journals/lancet/article/PIIS0140-6736%2813%2962329-6/fulltext https://www.thelancet.com/journals/lancet/article/PIIS0140-6... "As much as 30% of the most influential original medical research papers turn out to be wrong or exaggerated." - https://jamanetwork.com/journals/jama/fullarticle/201218#COMMENT https://jamanetwork.com/journals/jama/fullarticle/201218#COM... "The BMJ found that one-third of university press releases contained either exaggerated claims of causation (when the study itself only suggested correlation), unwarranted implications about animal studies for people, or unfounded health advice." - https://www.vox.com/2014/12/10/7372921/health-journalism-science https://www.vox.com/2014/12/10/7372921/health-journalism-sci...
- jefftk 4y agoI agree that scientists are discovering less, but I don't think it's mostly about their incentives. Instead, I think it's that there is much less low-hanging fruit than there used to be, and a lot more people trying to pick what fruit there is. You can see the difference by looking at new fields that have recently stumbled upon something very different, like all the progress recently in deep learning. More in this general direction: https://astralcodexten.substack.com/p/the-low-hanging-fruit-argument-models https://astralcodexten.substack.com/p/the-low-hanging-fruit-...
- w10-1 4y agoCommenters and even Lemire are really missing the two points of major inefficiency. (1) Making 30 researchers spend time on grants when only 1 wins is a waste of researchers time, regardless of productivity. It's a system built for bureaucrats who don't know to evaluate work-product from those who do by comparison if not mutual critique. It's a wealth transfer from researchers to taxpayers. (2) Grants always are on topic, but the topic is mostly selected to avoid political embarrassment. In the 1990's, it was HIV. In 2010 it was Alzheimer's Amyloid Beta. Then computer security. Unlike the internet or jet engines, these have not panned out as foundational research (except perhaps for some of HIV). Again, topics are selected not to deliver value to citizens, but to politicians. Yes, this is inefficient. Yes, academic bureaucracy sucks and the publication game is easily corrupted and early advances are more relatable than later ones. But the real problem with science in the U.S. is that there are not enough jobs for scientists, because companies are happy to get research and development done more cheaply elsewhere. Companies have only disincentives to develop local labor pools. That's the negative incentive that dwarfs everything in academia and publicly-funded research.
- warkdarrior 4y ago> But the real problem with science in the U.S. is that there are not enough jobs for scientists Are there other places where the science-funding mechanisms work well? Otherwise I do not see why your argument about companies being cheap makes sense.
- jackpirate 4y agoThen computer security. Unlike the internet or jet engines, these have not panned out as foundational research (except perhaps for some of HIV) In what word is computer security not a foundational topic? There's lots of reasons to critique the way NSF/NIH/DOD/etc allocate funding, but this is definitely not one of them.
- deleted 4y ago[deleted]
- R0b0t1 4y ago> Yes, this is inefficient. Yes, academic bureaucracy sucks and the publication game is easily corrupted and early advances are more relatable than later ones. How would you promote teaching and knowledge transfer while not needing to do this? On the job learning?
- cshenton 4y agoAnother interesting factor. PhD stipends are getting gobbled up by inflation. I wonder how many people will forgo starting a research career because they want to be able to pay rent.
- gautamdivgi 4y ago>>> In the first half of the XXth century, there were relatively few scientists, and these scientists were generally not lavishly funded This statement may not be entirely accurate. Philosophers, scientists and mathematicians in the early 20th century and before that were independently wealthy or had some sort of “royal” patronage that allowed them to pursue their interests for “public good”. Things changed in the later part of the 20th century but a scientist was a rarified breed until the 1970s. I don’t think most people had the means to pursue a scientific career. Grants and funding exploded at the end of the 20th century and foreign students (especially from China and India) provided the “cheap labor” that’s needed for a lot of the STEM fields PhD/post-doc armies. This has essentially led to a proliferation of scientists in academia. Abundance of school loans means universities can expand teaching faculty and then add research responsibility. It’s a vicious cycle. At least this is my hypothesis.
- djenendik 4y agoLooking at this guy's bio, he would appear to enjoy some of that extrinsic motivation.
- ta988 4y agoTo comment on the 40 days to make a grant application, yes that's close to what I used to spend. But the worst part is that on most grants you are not alone. I'd estimate participation looks like a power law. PI spends most of the time, another or two helpers spent a good week or so. And then the long tail of others that contribute between a tiny paragraph to nothing. Plus you need to take into account the money and personal management, that takes a chunk of the researcher time. And usually you are also part of committees to review grants and this takes a lot of time too and make the system "work". Now science is not a single person thing on most grants, the PI isn't doing much or at all of the lab work and most of it is done by postdocs, PhDs and other non-tenured personnel. So you can't compare with the old days. Another factor is that in the old days, there was a lot of unnamed personnel that was doing the dirty work of setting up experiments, tabulating data, writing and typing the articles and so on. They also often contributed on the "scientific work" itself, but this often was not recognized either.
- Imnimo 4y ago>In the first half of the XXth century, there were relatively few scientists, and these scientists were generally not lavishly funded. But aren't there still some scientists who work outside of grant funding and prestige ratrace? People at industry research labs who don't write grants, and aren't trying to impress a tenure committee, for example. Shouldn't we be able to point to them and say, "look how much more productive they are"?
- robotresearcher 4y ago> for a single successful research grant, hundreds of days might have been spent, purely on the acquisition of funding. I'm a former professor who has written many proposals and won some of them. I've reviewed many more. I did a lot of thinking and research when writing proposals. In a proposal, I tried to clearly explain the state of the art and how it could be extended, what resources I would need to do that, and how that work could be structured to make success more likely. I might even do a pilot study to show that the key idea is promising. Note that 'how to extend the state of the art' can be a seriously creative inventive step, or it could be the obvious next thing. But creativity does happen at proposal stage, and it certainly factors in my reviews of other's proposals. My point is that writing proposals is not entirely a waste of time. It's a bit like writing a movie script. It's not a movie, many scripts never become movies, but for some movies some of the best ideas were in the script. Same for proposals.
- esyir 4y agoFrom what I remember, it's not uncommon for proposals and actual work to be disjoint. Back when I was in academia, you'd hear of PIs writing a proposal, getting a grant and then deploying that on a completely different project.
- robotresearcher 4y agoSure, if you have a better idea, redeploy the money. But if your best idea was in the proposal, you will work on it. That does not invalidate the proposal as a driver of clear thinking.
- fasteo 4y agoMostly accurate TL;DR; for tax payers Your money is not going to research. Instead, it is going to bureaucracy and "extrinsic motivations" (researchers status games)
- edumucelli 4y agoSidetracking a bit the conversation. What a coincidence that the author (Lemire) is also represented on Today's #1 "Ask HN: What are some cool but obscure data structures you know about?" as he is the main contributor of RoaringBitmap https://github.com/RoaringBitmap/RoaringBitmap https://github.com/RoaringBitmap/RoaringBitmap and one of the main authors of the data structure.
- arminsergiony 4y agoI'm not sure that science as a whole is less useful than it was in 1900, but I'm sure that the average scientist today isn't as useful as Max Planck (but of course he was above average in his own day as well).
- YeGoblynQueenne 4y ago>> You should not underestimate the effect that constant administrative and grant writing might have on a researcher: many graduate students will tell you of their disappointment when encountering high status scientists who cannot seem to do actual research anymore. While I don't have the data to support this my intuition is that there are now many, many more researchers than in earlier years- both senior researchers who already have a career and junior researchers that are trying to get theirs started. The amount of science funding probably has also increased, but there are just too many people exiting PhD programmers for all of us to find cozy research jobs where we are free to pursue our interllectural interests undisturbed while the public money slowly but steadily drips in. In other words, a moderate pot of grant money combined with a huge number of new PhDs makes for huge competition.