5 ms·

Academia has become tremendously cutthroat but it still operates under the assumption that everyone acts honorably. Perhaps that’s no longer sustainable, but w

Academia has become tremendously cutthroat but it still operates under the assumption that everyone acts honorably. Perhaps that’s no longer sustainable, but what is the solution? Many of the problems with modern-day academic research can be traced back to increased scale and competition, neither of which is going away.

BurningFrog 6y ago
Seems like Peer Review is hopelessly broken. Conflicts of interest galore. Science did fine without it until ~40 years ago, and it can do so again. I hope.
- mhh__ 6y ago
  The solution is still within peer review, I feel. What ethical alternative is there? Required reproduction of the result could be a step forward, but only for cheap research.
  - BurningFrog 6y ago
    I don't think science was unethical before Peer Review. As I understand it, modern peer review came about as a 'crowd sourcing" solution to the hugely expanding number of articles to publish, in ever more specialized fields. The journals couldn't keep enough relevant expertise around, so letting the scientists review each other was the simple fix.
    - throwawayiionqz 6y ago
      This does not make sense. Before scientists started reviewing each other, who at the journal was judging the work of scientists to be published? Scientists?
      - BurningFrog 6y ago
        Well, science literate people working at the journal, I assume. The point is that they did not have a conflict of interest. In 2020 peer review, people are often asked to review the work of their rivals. So they're incentivized to (1) say that it's bad and demand many changes, and/or (2) steal their ideas.
  - wegs 6y ago
    arxiv and letting good stuff bubble up after-the-fact. Peer review in my field is a random process. It's not corrupt; it's just that reviewers give papers a 30 second skim. I've had one paper really reviewed in my whole career. What gets in and out is just random. NIPS did a nice study on this many years back. On the flip side, academic discourse gets delayed. People aim for Science/Nature, and then down the line. Journals / conferences aim for the prestige of high reject rates. Research goes out sometimes years too late. Peer review gives the impression of credibility, without the reality. It should be gone. Tiers of journals should be gone too.
- tnecniv 6y ago
  Peer review is much older than that. It dates back to the 18th century.
  - BurningFrog 6y ago
    Sure, but it didn't become mandatory and institutionalized until fairly recently.
effie 6y ago
Current U.S./Europe academia seems unable to change from within. It lives on government money which it itself controls and distributes to those who "play ball" - the others are pushed away. The only way a change can happen here if academia-independent people control and distribute money to individuals who showed talent/potential/results in the past. It will take massive outside interference to fix or supplant the corrupt system.
- softwaredoug 6y ago
  It does seem like there's sometimes too much inside baseball in grant review/awards. I wonder if there needs to be a notion of a 'stakeholder' in research that can measure/advocate for the general market of the research outcomes
  - effie 6y ago
    Yes, the public should be the main stakeholder if public is funding the project. The problem is, it takes an expert to judge the work of another expert. Wise people outside the local academia should represent the public in such things. Maybe one way is to hire and pay well some foreign professors to do the adversarial oversight/evaluation of our local science workforce. For example, a renowned German scientist hired and payed by French government for 5 years to criticize and oversee funding of French scientists close to his area of expertise. The system has to be more adversarial to weed out/defund the bad activities.
- cycomanic 6y ago
  That's not quite how it works and in my opinion not the main problem. Take for example DARPA grants, they are controlled by outside people and I don't think it works. You essentially have a "case worker" often with little understanding of the field pushing in some direction. Moreover the amount of admin around these grants is staggering. The root cause of the situation is that there is not enough money in the system and compared to previously more money is distributed by competitive grant processes. This results in a couple of things. 1. The cutoff (reviewer mark) where grants get funded or not is completely arbitrary, because it is in the flat part of a arctan or logit type curve, so small differences or luck can me you are able to do your research. 2. Competition therefore is fierce. 3. To balance the odds you have to write a lot of applications (academics now spend most of their time either writing grants or administering their grants, not doing research.) 4. It encourages incremental research, because non-incremental research implies high risk, but no one can afford the risk to fail, because having a significant reduction of publications reduces you chances of getting grants in the future (see 1.). Once you get some gap in your funding its almost impossible to get back. 5. This disadvantages women who want to have children.
BeetleB 6y ago
> Academia has become tremendously cutthroat but it still operates under the assumption that everyone acts honorably. I think the problem goes a bit beyond that, in that the notion of what constitutes acting honorably has changed.
seesawtron 6y ago
There are a lot of possible solutions that scientific community (Early Career Researchers) is working on. Open Science: There is a growing movement by ECRs to publish Journals that are open access or atleast offer the option to do so with slightly extra costs. This helps to dismantle the centralized consortia of publishing groups that have become too powerful. Open review: There are journals that allow the reviewers' and authors' comments to be published alongside the paper. This ensures that reviewers are reasonable in their conduct. See eLife for research in natural sciences[0] where once your paper is under review, it is published no matter what. The reviewers can claim that their issues wer not addressed if they feel so and that gets published along the paper's final version. Similary in the computer science community, there are conferences that follow the open review policies as well. Competition can be replaced with collaborations if the Group Leaders are open minded and are "raised" in such an environment. The next generation of researchers are today's graduate students. Publish or Perish: This archaic movement can be suppresed when the universities hire people by looking at CVs of applicants where they can only see the papers but not the Journals where they were published. H-index and traditional metrics can be replaced by Altmetrics [1]. The hiring committees need to do the extra work to go through the research of their applicants and not be biased by the journals in which they publish. These are only few things that are being done. See OpenScience movement variants[3] to explore more in this direction. [0] https://reviewer.elifesciences.org/author-guide/journal-policies https://reviewer.elifesciences.org/author-guide/journal-poli... [1] https://www.altmetric.com/about-altmetrics/what-are-altmetrics/ https://www.altmetric.com/about-altmetrics/what-are-altmetri... [3] https://oa2020.org/ https://oa2020.org/
wegs 6y ago
1) Reduce competition. Make jobs less desirable to where there might be able to win without cheating. More people at lower salaries would be good here. More research/teaching gets done, and you eliminate subclasses like adjuncts and lifelong postdocs. 2) Ban universities which take federal grants, federal student aid, 501(c)3 status, etc. from using NDA/non-disparage agreements. 3) Put in FOIA/public records-style transparency measures. 4) Open science / open data. Publications should be open too, as should source code. Anyone should be able to replicate scientific research unless it requires $$$ equipment or e.g. builds on personally-identifiable information. 5) Democratic governance processes within universities and salary caps (it doesn't get at this stuff, but it does at a lot of embezzlement). 6) Limit number of grad students per advisor. Basically, let enough sunshine in that the bad stuff can't stick around.

BurningFrog 6y ago

Seems like Peer Review is hopelessly broken. Conflicts of interest galore. Science did fine without it until ~40 years ago, and it can do so again. I hope.

mhh__ 6y ago
The solution is still within peer review, I feel. What ethical alternative is there? Required reproduction of the result could be a step forward, but only for cheap research.
- BurningFrog 6y ago
  I don't think science was unethical before Peer Review. As I understand it, modern peer review came about as a 'crowd sourcing" solution to the hugely expanding number of articles to publish, in ever more specialized fields. The journals couldn't keep enough relevant expertise around, so letting the scientists review each other was the simple fix.
  - throwawayiionqz 6y ago
    This does not make sense. Before scientists started reviewing each other, who at the journal was judging the work of scientists to be published? Scientists?
    - BurningFrog 6y ago
      Well, science literate people working at the journal, I assume. The point is that they did not have a conflict of interest. In 2020 peer review, people are often asked to review the work of their rivals. So they're incentivized to (1) say that it's bad and demand many changes, and/or (2) steal their ideas.
- wegs 6y ago
  arxiv and letting good stuff bubble up after-the-fact. Peer review in my field is a random process. It's not corrupt; it's just that reviewers give papers a 30 second skim. I've had one paper really reviewed in my whole career. What gets in and out is just random. NIPS did a nice study on this many years back. On the flip side, academic discourse gets delayed. People aim for Science/Nature, and then down the line. Journals / conferences aim for the prestige of high reject rates. Research goes out sometimes years too late. Peer review gives the impression of credibility, without the reality. It should be gone. Tiers of journals should be gone too.
tnecniv 6y ago
Peer review is much older than that. It dates back to the 18th century.
- BurningFrog 6y ago
  Sure, but it didn't become mandatory and institutionalized until fairly recently.

effie 6y ago

Current U.S./Europe academia seems unable to change from within. It lives on government money which it itself controls and distributes to those who "play ball" - the others are pushed away. The only way a change can happen here if academia-independent people control and distribute money to individuals who showed talent/potential/results in the past. It will take massive outside interference to fix or supplant the corrupt system.

softwaredoug 6y ago
It does seem like there's sometimes too much inside baseball in grant review/awards. I wonder if there needs to be a notion of a 'stakeholder' in research that can measure/advocate for the general market of the research outcomes
- effie 6y ago
  Yes, the public should be the main stakeholder if public is funding the project. The problem is, it takes an expert to judge the work of another expert. Wise people outside the local academia should represent the public in such things. Maybe one way is to hire and pay well some foreign professors to do the adversarial oversight/evaluation of our local science workforce. For example, a renowned German scientist hired and payed by French government for 5 years to criticize and oversee funding of French scientists close to his area of expertise. The system has to be more adversarial to weed out/defund the bad activities.
cycomanic 6y ago
That's not quite how it works and in my opinion not the main problem. Take for example DARPA grants, they are controlled by outside people and I don't think it works. You essentially have a "case worker" often with little understanding of the field pushing in some direction. Moreover the amount of admin around these grants is staggering. The root cause of the situation is that there is not enough money in the system and compared to previously more money is distributed by competitive grant processes. This results in a couple of things. 1. The cutoff (reviewer mark) where grants get funded or not is completely arbitrary, because it is in the flat part of a arctan or logit type curve, so small differences or luck can me you are able to do your research. 2. Competition therefore is fierce. 3. To balance the odds you have to write a lot of applications (academics now spend most of their time either writing grants or administering their grants, not doing research.) 4. It encourages incremental research, because non-incremental research implies high risk, but no one can afford the risk to fail, because having a significant reduction of publications reduces you chances of getting grants in the future (see 1.). Once you get some gap in your funding its almost impossible to get back. 5. This disadvantages women who want to have children.

BeetleB 6y ago

> Academia has become tremendously cutthroat but it still operates under the assumption that everyone acts honorably. I think the problem goes a bit beyond that, in that the notion of what constitutes acting honorably has changed.

seesawtron 6y ago

There are a lot of possible solutions that scientific community (Early Career Researchers) is working on. Open Science: There is a growing movement by ECRs to publish Journals that are open access or atleast offer the option to do so with slightly extra costs. This helps to dismantle the centralized consortia of publishing groups that have become too powerful. Open review: There are journals that allow the reviewers' and authors' comments to be published alongside the paper. This ensures that reviewers are reasonable in their conduct. See eLife for research in natural sciences[0] where once your paper is under review, it is published no matter what. The reviewers can claim that their issues wer not addressed if they feel so and that gets published along the paper's final version. Similary in the computer science community, there are conferences that follow the open review policies as well. Competition can be replaced with collaborations if the Group Leaders are open minded and are "raised" in such an environment. The next generation of researchers are today's graduate students. Publish or Perish: This archaic movement can be suppresed when the universities hire people by looking at CVs of applicants where they can only see the papers but not the Journals where they were published. H-index and traditional metrics can be replaced by Altmetrics [1]. The hiring committees need to do the extra work to go through the research of their applicants and not be biased by the journals in which they publish. These are only few things that are being done. See OpenScience movement variants[3] to explore more in this direction. [0] https://reviewer.elifesciences.org/author-guide/journal-policies https://reviewer.elifesciences.org/author-guide/journal-poli... [1] https://www.altmetric.com/about-altmetrics/what-are-altmetrics/ https://www.altmetric.com/about-altmetrics/what-are-altmetri... [3] https://oa2020.org/ https://oa2020.org/

wegs 6y ago

1) Reduce competition. Make jobs less desirable to where there might be able to win without cheating. More people at lower salaries would be good here. More research/teaching gets done, and you eliminate subclasses like adjuncts and lifelong postdocs. 2) Ban universities which take federal grants, federal student aid, 501(c)3 status, etc. from using NDA/non-disparage agreements. 3) Put in FOIA/public records-style transparency measures. 4) Open science / open data. Publications should be open too, as should source code. Anyone should be able to replicate scientific research unless it requires $$$ equipment or e.g. builds on personally-identifiable information. 5) Democratic governance processes within universities and salary caps (it doesn't get at this stuff, but it does at a lot of embezzlement). 6) Limit number of grad students per advisor. Basically, let enough sunshine in that the bad stuff can't stick around.