4 ms·

I think zooming further out, the incentive for academia is to churn out degrees + get grants. Getting a PhD is supposed to require doing something that nobody h

I think zooming further out, the incentive for academia is to churn out degrees + get grants. Getting a PhD is supposed to require doing something that nobody has done before. Most people getting PhDs/in academia are not actually good enough to do this (it’s really hard! There’s not a ton of low hanging fruit, and you’re “competing” with many others), which is why we end up with tons of garbage papers nobody will ever care about. Medical research may be performed by MDs but the incentives are still basically the same: papers are resume builders. It looks better when applying to a fellowship/job to have a nice publication history. Obviously the best case is to have worked on some really groundbreaking stuff - still really hard - but the next best case is to have a ton of meh publications, since that beats having a few publications, or no publications. In medical programs the grant thing is a lot bigger too because there is, rightfully, tons of money to throw to that area. You need grants as an academic to progress. You won’t keep getting grants if you take grants and then don’t publish anything, so even if you have nothing good come out of it, you need to publish something. That incentivizes fraud in the worst case and noise in the best. The better-best case would be if academia were more open to accepting null results

xpe 4y ago
I wish the extent to which a paper is well written (honest, clear, appropriate for the audience) was enough. Sure, I want scientist to "have a nose" for finding interesting results, but I don't like the framing that a solid piece of work is any less because it happened to not demonstrate a useful result. I'll add that I would like to see a LOT more work that synthesizes and analyzes other work; i.e. literature reviews and meta-analyses.
- opportune 4y ago
  That’s what most of those papers I am calling garbage are: they are not necessarily wrong but they are not interesting. A meta analysis is cheap and easy to do for even an undergrad, and doesn’t require any special insight or foresight. I think the bigger problem is we have too many people trying to chase after the highest tier of academic achievement relative to what that tier “should” be or was in the past. It’s benchmarked on novelty, but most people doing research are never going to produce any worthwhile novel results - in some cases it’s just bad luck but in most I think it is just lack of aptitude. Research is not supposed to just be a resume checkmark, and a PhD isn’t just supposed to be some structured degree program where you can get “on rails” and churn out papers to get a degree proving you have above-average intelligence. But that’s what it is, and it generates tons of noise, while cheapening the value of a PhD. If I were emperor of the world I’d split research into two tiers where one is more focused on basic science: investigative studies, verifying results, writing papers with solid structure, applying stats. This would be what most people get, and could soak up demand for people getting PhDs with no intention of becoming academics (ie to immigrate, to qualify for some kind of job). And then a second tier of research for the wickedly skilled researchers who are producing novel results and really moving the field forward. Right now that second group, in most science disciplines, is who progresses in academia anyway. But tons of people are “doing research” and getting advanced degrees based on research in a way that is more like a “basic scientist training program” rather than “moving forward a field of study.”
  - xpe 4y ago
    > A meta analysis is cheap and easy to do for even an undergrad, and doesn’t require any special insight or foresight. This is too harsh, too narrow, and too judgmental. This only mentions the skill required, but neglects to account for the benefits to the body of knowledge (literature). I reject the idea that scientific papers impact should be judged on a curve according to how difficult the work was: 1. This is premised on a false notion of scarcity. 2. Breakthroughs can seem obvious to a lucky person in the right place and time. 3. Others may ascribe "genius-level" status to such lucky people, but this elevation is often unwarranted and not useful. 4. The time delay before impact and benefit is uncertain. Often work takes time to be synthesized, appreciated, applied, and so on. 5. What is {important, relevant, useful} can change over time. Having only a narrow definition misses the point.
  - xpe 4y ago
    > Research is not supposed to just be a resume checkmark, and a PhD isn’t just supposed to be some structured degree program where you can get “on rails” and churn out papers to get a degree proving you have above-average intelligence. But that’s what it is, and it generates tons of noise, while cheapening the value of a PhD. Ah, this seems like the emotional core of your concern. I appreciate the honesty. You are referencing an ideal that you think has been lost. I've seen this get mentioned frequently, and I tend to believe it. Still, one has to admit this emphasis runs the risk of appearing elitist and/or egotistical. Foundational work is important, yes. So are applications. I'm seeing strong signs of the zero sum game battle for resources and recognition here in your comment, a quality that often makes academic institutions particularly nasty and self-defeating.
    - opportune 4y ago
      I do think there is a tinge of elitism for sure, but what I mean more practically speaking is that a lot of research is getting churned out that is not incentivized by actually wanting to do good research. I fundamentally believe you always get what you incentivize, and the incentivizes for research lean too far towards building careers and resumes outside of research, to deleterious effects. The elitism you are picking up is my sense that academia is losing its value by becoming a bureaucracy machine with low standards, rather than a way to produce novel insights. It doesn’t make sense to build a road to nowhere just because it’s funded, looks good on a resume, and makes it look like you know how to build roads. Can a meta-analysis be valuable? Sure. Should 1 million people be submitting meta-analyses to journals? No, because almost none of them will be read, and they are unlikely to be of high quality. My proposal is simply to make a scientist track that is not indexed on novelty but instead on conducting good research, a focus on the scientific method and applying it to a field, etc. This should satisfy the desire to produce PhDs for basic governmental and educational jobs that for whatever reason (even if artificial scarcity) need some kind of barrier to advancement or knowledge of how to conduct research. Most people going from academia to elsewhere will not be working on the exact niches they studied anyway. And it would at least keep the treadmill of people chasing the “highest degree” or random publications for their resumes at bay by removing the requirement for novelty.
deleted 4y ago
[deleted]

xpe 4y ago

I wish the extent to which a paper is well written (honest, clear, appropriate for the audience) was enough. Sure, I want scientist to "have a nose" for finding interesting results, but I don't like the framing that a solid piece of work is any less because it happened to not demonstrate a useful result. I'll add that I would like to see a LOT more work that synthesizes and analyzes other work; i.e. literature reviews and meta-analyses.

opportune 4y ago
That’s what most of those papers I am calling garbage are: they are not necessarily wrong but they are not interesting. A meta analysis is cheap and easy to do for even an undergrad, and doesn’t require any special insight or foresight. I think the bigger problem is we have too many people trying to chase after the highest tier of academic achievement relative to what that tier “should” be or was in the past. It’s benchmarked on novelty, but most people doing research are never going to produce any worthwhile novel results - in some cases it’s just bad luck but in most I think it is just lack of aptitude. Research is not supposed to just be a resume checkmark, and a PhD isn’t just supposed to be some structured degree program where you can get “on rails” and churn out papers to get a degree proving you have above-average intelligence. But that’s what it is, and it generates tons of noise, while cheapening the value of a PhD. If I were emperor of the world I’d split research into two tiers where one is more focused on basic science: investigative studies, verifying results, writing papers with solid structure, applying stats. This would be what most people get, and could soak up demand for people getting PhDs with no intention of becoming academics (ie to immigrate, to qualify for some kind of job). And then a second tier of research for the wickedly skilled researchers who are producing novel results and really moving the field forward. Right now that second group, in most science disciplines, is who progresses in academia anyway. But tons of people are “doing research” and getting advanced degrees based on research in a way that is more like a “basic scientist training program” rather than “moving forward a field of study.”
- xpe 4y ago
  > A meta analysis is cheap and easy to do for even an undergrad, and doesn’t require any special insight or foresight. This is too harsh, too narrow, and too judgmental. This only mentions the skill required, but neglects to account for the benefits to the body of knowledge (literature). I reject the idea that scientific papers impact should be judged on a curve according to how difficult the work was: 1. This is premised on a false notion of scarcity. 2. Breakthroughs can seem obvious to a lucky person in the right place and time. 3. Others may ascribe "genius-level" status to such lucky people, but this elevation is often unwarranted and not useful. 4. The time delay before impact and benefit is uncertain. Often work takes time to be synthesized, appreciated, applied, and so on. 5. What is {important, relevant, useful} can change over time. Having only a narrow definition misses the point.
- xpe 4y ago
  > Research is not supposed to just be a resume checkmark, and a PhD isn’t just supposed to be some structured degree program where you can get “on rails” and churn out papers to get a degree proving you have above-average intelligence. But that’s what it is, and it generates tons of noise, while cheapening the value of a PhD. Ah, this seems like the emotional core of your concern. I appreciate the honesty. You are referencing an ideal that you think has been lost. I've seen this get mentioned frequently, and I tend to believe it. Still, one has to admit this emphasis runs the risk of appearing elitist and/or egotistical. Foundational work is important, yes. So are applications. I'm seeing strong signs of the zero sum game battle for resources and recognition here in your comment, a quality that often makes academic institutions particularly nasty and self-defeating.
  - opportune 4y ago
    I do think there is a tinge of elitism for sure, but what I mean more practically speaking is that a lot of research is getting churned out that is not incentivized by actually wanting to do good research. I fundamentally believe you always get what you incentivize, and the incentivizes for research lean too far towards building careers and resumes outside of research, to deleterious effects. The elitism you are picking up is my sense that academia is losing its value by becoming a bureaucracy machine with low standards, rather than a way to produce novel insights. It doesn’t make sense to build a road to nowhere just because it’s funded, looks good on a resume, and makes it look like you know how to build roads. Can a meta-analysis be valuable? Sure. Should 1 million people be submitting meta-analyses to journals? No, because almost none of them will be read, and they are unlikely to be of high quality. My proposal is simply to make a scientist track that is not indexed on novelty but instead on conducting good research, a focus on the scientific method and applying it to a field, etc. This should satisfy the desire to produce PhDs for basic governmental and educational jobs that for whatever reason (even if artificial scarcity) need some kind of barrier to advancement or knowledge of how to conduct research. Most people going from academia to elsewhere will not be working on the exact niches they studied anyway. And it would at least keep the treadmill of people chasing the “highest degree” or random publications for their resumes at bay by removing the requirement for novelty.

deleted 4y ago

[deleted]