3 ms·

Academia does not value quality, but quantity. It selects for scientists who are the best at marketing and networking, not necessarily doing quality science, th

Academia does not value quality, but quantity. It selects for scientists who are the best at marketing and networking, not necessarily doing quality science, though they can also be. I have no idea how to fix this, but competition needs to be reduced, probably by more guaranteed funding for positions, not just projects, as grants are. This latest military aid package is 2x the entire NIH budget, so surely there is more money for science out there.

ska 3y ago
It's not really quality vs quantity. The root problem is that (particularly in R1's) the job of raising money to perform the science has devolved somewhat to the level of individual labs and PI's, which creates an incentive that rewards good fundraisers in a much more predictable way than good researchers. In theory this could be addressed by more rigor in the funding agencies review processes, but they aren't resourced to really handle that. It's like a baby (both in size and impact) version of the problem in US Congress & Senate.
- michaelrpeskin 3y ago
  But it's even worse than that. Since universities are funded by the grant overhead (30-40% of the grant goes directly to the university, sometimes higher), there's an incentive for "expensive" research. Why fund a theorist who needs a pencil and paper and maybe a fancy computer when you can fund an expensive lab full of state of the art lasers and optics or any other type of expensive technology. Do you want to come up with a ground breaking theory or do you want to turn the crank and measure some value a little more precise? There is value in both, but the universities really bias towards the latter because it's more expensive and needs more and bigger grants to get done.
  - ska 3y ago
    Not really, at least in my experience. Overhead is fungible, so they get roughly the same cut of everything that comes in (to a first approximation). That means the support the institution will put behind a shoot-for-the-stars research centre grant is way different than what a theorist looking to pay for 5 grad students will get, but the institution is happy to proportionately support that as well. Especially R1's that are trying to play the prestige game aggressively, they'll push for a "world class" faculty page pretty much across the board. But they don't all get the same offices. Institutions' reliance on overhead to fund operations varies wildly as well, which makes the calculus different.
    - ska 3y ago
      Oh I should note that capex and opex aren't treated the same in grant-land either, and funding agencies can put limits on university overhead for infrastructure grants etc. so that's not all apples to apples. The benefit to the university for soft salary or funding grad students etc. in both scenarios is the same, but getting money for a new computer cluster or a synchrotron or whatever isn't, typically.
  - queuebert 3y ago
    The overhead rate for R1s is closer to 70%.
  - cycomanic 3y ago
    It's more likely the other way around. For most grants I'm aware off universities can't charge overheads on equipment so a theoretician with lots of phd students brings in more overhead than an experimentalist that needs lots of equipment. Obviously the reality is much more complex, depending on country it's easier or more difficult to get funding as a theoretician (in the US it's supposedly much more difficult), theory groups are typically smaller, experimental research often results in more publicity... And all this really depends on the field. I think pitting theory against experiments does not address the issues. The big problem IMO is that the funding systems are so competitive and at the same time (initially) have a large luck component, that it incentives short term, low risk research.
  - Fomite 3y ago
    My department is actively pivoting away from more expensive researchers towards theorists, implementation scientists, etc. Partially because all that expensive lab equipment means expensive startup packages. And expensive grants get reviewed in the context of being expensive.
Djle 3y ago
This is more of a Thomas Kuhn moment than anything else, where the mainstream doesn't accept new theories that will upend their own work.
test77777 3y ago
I think the opposite, we need more raw labor aimed at replicating scientific results. Today our institutions are so tiny they can hardly afford what few projects are funded to completion.
StableAlkyne 3y ago
Part of it is that research grants are not used fully as a research funding source - a typical university administration will skim about a quarter of every grant for "administrative costs." It's not called out as corruption because it's the norm, but it does have the effect of reducing the amount of tax-funded R&D dollars that actually make it to R&D As a result, the people in charge of hiring and firing have a self-preserving interest to value grant-earners It'll never happen, but if funding agencies like the NSF or NIH put strings on the funding like "100% must go to the PI awarded this grant" with accounting requirements, it would help remove some of the financial incentive. It would also help lower some of the pressure to publish or perish, since a lot of that comes from the need to chase grants.
- ccooffee 3y ago
  > It'll never happen, but if funding agencies like the NSF or NIH put strings on the funding like "100% must go to the PI awarded this grant" with accounting requirements, it would help remove some of the financial incentive. From what I've seen in very limited searches, universities claim that the 30-60% overhead/administrative costs are to account for things like employee benefits, utility costs, building maintenance, and the like. The stated money pits all make sense to me, but I don't see how it actually comes up to those numbers. Do you know if these costs are ever itemized by universities? That's probably a necessary first step before NSF/NIH would consider a rule to avoid paying opaque overhead costs. (Though I fear it would lead to absurd equipment rental fees or something of the sort. "You want to use a test tube? $3 per day per tube!")
  - StableAlkyne 3y ago
    I don't know how much accounting is done on the university side to itemize research bills. What my advisor told me was when I was going through the ringer was that once you have the money, it's yours - you can do whatever you want with it short of embezzlement. All that matters is you make progress on the thing the funding agency granted you the money for, doesn't matter if you ran over budget or spent $100.
    - 3y ago
Etheryte 3y ago
While I see where you're coming from, I'm very conflicted on your approach to address this. I don't think throwing more money at the problem fixes the underlying issues, if anything, I would expect them to deepen. There was an article making the rounds some time ago on HN how something like three quarters of medical studies had either strong data analysis errors in them or had complete bogus data, to the point where it was impossible to tell whether the results had any grain of truth to them. That's an absurd ratio, and not something I would want to fund.
- queuebert 3y ago
  If the desire to inflate results comes from the intense competition, not incompetence, then more money would fix it. The fact that the top universities are suffering big fake research scandals would bolster this. Personally I've known many competent scientists who left for greener pastures not because they couldn't cut it but because they wanted higher pay and less stress.
- cycomanic 3y ago
  I think this attitude is actually a big part of the problem. Because research is largely government funded there is big political pressure to show that the research leads to measurable outcomes and that there is zero misuse or waste of funding. This is what has lead to the current system of big administrative bodies just focused on tracking how funding is used (and we can't really blame universities for the situation, reporting requirements on everything they do have increased dramatically). On the other hand nobody cares about the waste in private industry, we are perfectly fine with paying a certain amount for e.g. a Facebook ad even though they just wasted a huge amount of money on a big VR bet.. Maybe we should just admit that there will be wastage and we can't easily measure scientific outcomes and just say we are happy with the overall benefit we get from science/academia for the price we pay for it (which in the broad picture is quite small).
antognini 3y ago
My dad was a professor and would joke that the way a tenure committee made their decision was that they'd print out all your papers, put them in a folder, and then throw it down a stairwell. If your packet made it to the bottom, it would be an easy yes. If it made it halfway down it would be marginal. If it only made it down a few steps it would be a definite no. There's also the old aphorism that tenure committees can't read, but they can count.

ska 3y ago

It's not really quality vs quantity. The root problem is that (particularly in R1's) the job of raising money to perform the science has devolved somewhat to the level of individual labs and PI's, which creates an incentive that rewards good fundraisers in a much more predictable way than good researchers. In theory this could be addressed by more rigor in the funding agencies review processes, but they aren't resourced to really handle that. It's like a baby (both in size and impact) version of the problem in US Congress & Senate.

michaelrpeskin 3y ago
But it's even worse than that. Since universities are funded by the grant overhead (30-40% of the grant goes directly to the university, sometimes higher), there's an incentive for "expensive" research. Why fund a theorist who needs a pencil and paper and maybe a fancy computer when you can fund an expensive lab full of state of the art lasers and optics or any other type of expensive technology. Do you want to come up with a ground breaking theory or do you want to turn the crank and measure some value a little more precise? There is value in both, but the universities really bias towards the latter because it's more expensive and needs more and bigger grants to get done.
- ska 3y ago
  Not really, at least in my experience. Overhead is fungible, so they get roughly the same cut of everything that comes in (to a first approximation). That means the support the institution will put behind a shoot-for-the-stars research centre grant is way different than what a theorist looking to pay for 5 grad students will get, but the institution is happy to proportionately support that as well. Especially R1's that are trying to play the prestige game aggressively, they'll push for a "world class" faculty page pretty much across the board. But they don't all get the same offices. Institutions' reliance on overhead to fund operations varies wildly as well, which makes the calculus different.
  - ska 3y ago
    Oh I should note that capex and opex aren't treated the same in grant-land either, and funding agencies can put limits on university overhead for infrastructure grants etc. so that's not all apples to apples. The benefit to the university for soft salary or funding grad students etc. in both scenarios is the same, but getting money for a new computer cluster or a synchrotron or whatever isn't, typically.
- queuebert 3y ago
  The overhead rate for R1s is closer to 70%.
- cycomanic 3y ago
  It's more likely the other way around. For most grants I'm aware off universities can't charge overheads on equipment so a theoretician with lots of phd students brings in more overhead than an experimentalist that needs lots of equipment. Obviously the reality is much more complex, depending on country it's easier or more difficult to get funding as a theoretician (in the US it's supposedly much more difficult), theory groups are typically smaller, experimental research often results in more publicity... And all this really depends on the field. I think pitting theory against experiments does not address the issues. The big problem IMO is that the funding systems are so competitive and at the same time (initially) have a large luck component, that it incentives short term, low risk research.
- Fomite 3y ago
  My department is actively pivoting away from more expensive researchers towards theorists, implementation scientists, etc. Partially because all that expensive lab equipment means expensive startup packages. And expensive grants get reviewed in the context of being expensive.

Djle 3y ago

This is more of a Thomas Kuhn moment than anything else, where the mainstream doesn't accept new theories that will upend their own work.

test77777 3y ago

I think the opposite, we need more raw labor aimed at replicating scientific results. Today our institutions are so tiny they can hardly afford what few projects are funded to completion.

StableAlkyne 3y ago

Part of it is that research grants are not used fully as a research funding source - a typical university administration will skim about a quarter of every grant for "administrative costs." It's not called out as corruption because it's the norm, but it does have the effect of reducing the amount of tax-funded R&D dollars that actually make it to R&D As a result, the people in charge of hiring and firing have a self-preserving interest to value grant-earners It'll never happen, but if funding agencies like the NSF or NIH put strings on the funding like "100% must go to the PI awarded this grant" with accounting requirements, it would help remove some of the financial incentive. It would also help lower some of the pressure to publish or perish, since a lot of that comes from the need to chase grants.

ccooffee 3y ago
> It'll never happen, but if funding agencies like the NSF or NIH put strings on the funding like "100% must go to the PI awarded this grant" with accounting requirements, it would help remove some of the financial incentive. From what I've seen in very limited searches, universities claim that the 30-60% overhead/administrative costs are to account for things like employee benefits, utility costs, building maintenance, and the like. The stated money pits all make sense to me, but I don't see how it actually comes up to those numbers. Do you know if these costs are ever itemized by universities? That's probably a necessary first step before NSF/NIH would consider a rule to avoid paying opaque overhead costs. (Though I fear it would lead to absurd equipment rental fees or something of the sort. "You want to use a test tube? $3 per day per tube!")
- StableAlkyne 3y ago
  I don't know how much accounting is done on the university side to itemize research bills. What my advisor told me was when I was going through the ringer was that once you have the money, it's yours - you can do whatever you want with it short of embezzlement. All that matters is you make progress on the thing the funding agency granted you the money for, doesn't matter if you ran over budget or spent $100.
  - 3y ago

Etheryte 3y ago

While I see where you're coming from, I'm very conflicted on your approach to address this. I don't think throwing more money at the problem fixes the underlying issues, if anything, I would expect them to deepen. There was an article making the rounds some time ago on HN how something like three quarters of medical studies had either strong data analysis errors in them or had complete bogus data, to the point where it was impossible to tell whether the results had any grain of truth to them. That's an absurd ratio, and not something I would want to fund.

queuebert 3y ago
If the desire to inflate results comes from the intense competition, not incompetence, then more money would fix it. The fact that the top universities are suffering big fake research scandals would bolster this. Personally I've known many competent scientists who left for greener pastures not because they couldn't cut it but because they wanted higher pay and less stress.
cycomanic 3y ago
I think this attitude is actually a big part of the problem. Because research is largely government funded there is big political pressure to show that the research leads to measurable outcomes and that there is zero misuse or waste of funding. This is what has lead to the current system of big administrative bodies just focused on tracking how funding is used (and we can't really blame universities for the situation, reporting requirements on everything they do have increased dramatically). On the other hand nobody cares about the waste in private industry, we are perfectly fine with paying a certain amount for e.g. a Facebook ad even though they just wasted a huge amount of money on a big VR bet.. Maybe we should just admit that there will be wastage and we can't easily measure scientific outcomes and just say we are happy with the overall benefit we get from science/academia for the price we pay for it (which in the broad picture is quite small).

antognini 3y ago

My dad was a professor and would joke that the way a tenure committee made their decision was that they'd print out all your papers, put them in a folder, and then throw it down a stairwell. If your packet made it to the bottom, it would be an easy yes. If it made it halfway down it would be marginal. If it only made it down a few steps it would be a definite no. There's also the old aphorism that tenure committees can't read, but they can count.