9 ms·

What do you suggest as an alternative?

cscurmudgeon 6y ago
Research publication, funding, and jobs should not be tied to outcomes of research. They should be tied to how systematic, logical, and well-documented the research is. We need a system wide change from funding bodies, job committees to publishing criteria.
- mlthoughts2018 6y ago
  Along the same lines, “what question to ask” or “what idea to try” should never generate credit or fame for individuals. This incentivizes Indiana Jones style research, results at all costs. As a researcher you should be desiring to get jobs or rewards based solely on the care of methodology, clarity of communication and ease of reproducibility. If you do those things well and the papers turn up negative results, that’s good archived knowledge for society. Turning up positive results should be viewed as an emergent property of a wide network of labs, agencies, universities and governments, and never a property of darling individuals.
  - dasudasu 6y ago
    It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions. Of course, the counterpart to this is that you see hype and fads developing very easily. But overall it is very easy to come up with experiments that any experienced researcher would have told you beforehand had no chance of producing interesting / impactful / positive results. The system must discriminates against researchers that are able to come up with interesting (positive) results with greater probability, since it represents every time an expense of finite resources. If your only contribution as a researcher is to be able to make conscientious and well realized experiments, then sadly your place is much more as an assistant than a PI. I've never seen it argued in serious research circles that grants should be given this way, but I did find the non-research community to be overly obsessed over negative results not being published or rewarded. It is simply a consequence of the set of possible experiments being infinite, the same way you can come up with an infinite number of startup ideas, but not all of them are equally good even before implementation. Should we reward startup founders with failed startups because they tried their best and really did the best they could given the circumstances?
    - mlthoughts2018 6y ago
      The comparison with startups isn’t valid - they seek an explicit profit outcome where good or bad ideas are judged by success in markets. Research is meant to be a public good, that furthers society’s at large repository of knowledge and culture. > “ It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions.” I don’t think this can be argued actually. This is mythology, usually applied to creditmongers who run labs and accumulate accolades that are actually due to a wide array of students and post-docs who are made to get reduced credit as a type of dues paying laced with rampant discrimination and sexism. I’d say mythologizing the idea of a crack sleuth who has a special knack for research intuition is extremely harmful on all fronts: depriving value to society because it’s false and depriving value to the wide network of lower level staff who are actually responsible for progress.
gwerbret 6y ago
1) I think the idea of funding "people, not projects" (the Howard Hughes Research Institute's byline; I'm not affiliated with them in any way) is a great one. It would probably involve changing the structure of research programs in other fundamental ways, such as by 2) Limiting the number of graduate students. The current relatively-low barrier of entry into grad school provides cheap, motivated labor for PIs who are trying to stretch their research dollars to the limit. The outcome today is far more PhD graduates than jobs, and for those who get jobs, far too many of them for the research funds that are available. The overall quality of research drops when PIs focus what is fundable, rather than what is important. 3) Shifting the burden of the bulk of research work from trainees to salaried research associates/assistants/lab techs. 4) Change the focus of research output from novelty and volume (of papers published) to quality and significance of work done. Good research is often slow and careful, and doesn't fit well with the demands of grant funding agencies. Fewer grad students means more resources available for their training, and better career prospects. Shifting the bulk of the research benchwork to salaried professionals removes the incentive to commit fraud. None of this has to come at the expense of research quality; as an example, the NIH/HHMI already have approaches to vetting research programs for quality, even if the PIs aren't competing for grant funding.
- an_opabinia 6y ago
  > Limiting the number of graduate students. Isn't a lot of the graduate school product providing a way for people to immigrate to the United States? The economy around it is pretty complicated and definitely there's way more pressure to supply more, not fewer, spots.
  - vkou 6y ago
    No, the problem is a bit more structural then that. A single professor trains dozens of grad students over the course of their career. Even if you remove every single foreigner, and every single person who does not want to pursue a career in academia, you still end up with dozens of graduates - per professor. The number of jobs available for those graduates? One - that professor's - when he or she retires. And until then, they get to burn the midnight oil, doing grunt work on their projects, in the hopes of competing their dozen collegaues for a shot at that one spot.
adminprof 6y ago
This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happening all the time. If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago. The ideas also lack nuance and when you get into the definitions of things (for example, p-hacking), then things become a lot more grey; are you allowed to look at a dataset that you spent 2 years collecting if your first hypothesis does not pan out? The clear cut cases are obvious to everyone, it's the grey area that takes 99% of the time to figure out. Imagine reading a thread where everyone is proposing "solutions" to software development. It'd go something like "software development is a cesspool and 80% of it fails (see voting systems, MySpace, electronic health records, Theranos. Here's what software companies need to do:" [yes I'm being intentionally stupid to demonstrate how annoying this is] 1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them. 2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs. [yes, see how triggering that is] 3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max. 4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing. These changes need to happen. derp derp
- nl 6y ago
  Not sure who downvoted this, but this is 100% on the money in my experience. People here are writing comments like that funding should be tied to "how systematic, logical, and well-documented the research is" as though these things are correlated to the existing criteria at all. Much of the criticism here is like someone who knows how to build roads criticising agile software development because it makes it look like no one knows what they are going to build in the end. It's frustrating, and wrong, but the errors are subtle and aggregative.

Research publication, funding, and jobs should not be tied to outcomes of research. They should be tied to how systematic, logical, and well-documented the research is. We need a system wide change from funding bodies, job committees to publishing criteria.

mlthoughts2018 6y ago
Along the same lines, “what question to ask” or “what idea to try” should never generate credit or fame for individuals. This incentivizes Indiana Jones style research, results at all costs. As a researcher you should be desiring to get jobs or rewards based solely on the care of methodology, clarity of communication and ease of reproducibility. If you do those things well and the papers turn up negative results, that’s good archived knowledge for society. Turning up positive results should be viewed as an emergent property of a wide network of labs, agencies, universities and governments, and never a property of darling individuals.
- dasudasu 6y ago
  It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions. Of course, the counterpart to this is that you see hype and fads developing very easily. But overall it is very easy to come up with experiments that any experienced researcher would have told you beforehand had no chance of producing interesting / impactful / positive results. The system must discriminates against researchers that are able to come up with interesting (positive) results with greater probability, since it represents every time an expense of finite resources. If your only contribution as a researcher is to be able to make conscientious and well realized experiments, then sadly your place is much more as an assistant than a PI. I've never seen it argued in serious research circles that grants should be given this way, but I did find the non-research community to be overly obsessed over negative results not being published or rewarded. It is simply a consequence of the set of possible experiments being infinite, the same way you can come up with an infinite number of startup ideas, but not all of them are equally good even before implementation. Should we reward startup founders with failed startups because they tried their best and really did the best they could given the circumstances?
  - mlthoughts2018 6y ago
    The comparison with startups isn’t valid - they seek an explicit profit outcome where good or bad ideas are judged by success in markets. Research is meant to be a public good, that furthers society’s at large repository of knowledge and culture. > “ It can be argued that part of being a good researcher is developing an intuition and taste for interesting and promising research questions.” I don’t think this can be argued actually. This is mythology, usually applied to creditmongers who run labs and accumulate accolades that are actually due to a wide array of students and post-docs who are made to get reduced credit as a type of dues paying laced with rampant discrimination and sexism. I’d say mythologizing the idea of a crack sleuth who has a special knack for research intuition is extremely harmful on all fronts: depriving value to society because it’s false and depriving value to the wide network of lower level staff who are actually responsible for progress.

gwerbret 6y ago

1) I think the idea of funding "people, not projects" (the Howard Hughes Research Institute's byline; I'm not affiliated with them in any way) is a great one. It would probably involve changing the structure of research programs in other fundamental ways, such as by 2) Limiting the number of graduate students. The current relatively-low barrier of entry into grad school provides cheap, motivated labor for PIs who are trying to stretch their research dollars to the limit. The outcome today is far more PhD graduates than jobs, and for those who get jobs, far too many of them for the research funds that are available. The overall quality of research drops when PIs focus what is fundable, rather than what is important. 3) Shifting the burden of the bulk of research work from trainees to salaried research associates/assistants/lab techs. 4) Change the focus of research output from novelty and volume (of papers published) to quality and significance of work done. Good research is often slow and careful, and doesn't fit well with the demands of grant funding agencies. Fewer grad students means more resources available for their training, and better career prospects. Shifting the bulk of the research benchwork to salaried professionals removes the incentive to commit fraud. None of this has to come at the expense of research quality; as an example, the NIH/HHMI already have approaches to vetting research programs for quality, even if the PIs aren't competing for grant funding.

an_opabinia 6y ago
> Limiting the number of graduate students. Isn't a lot of the graduate school product providing a way for people to immigrate to the United States? The economy around it is pretty complicated and definitely there's way more pressure to supply more, not fewer, spots.
- vkou 6y ago
  No, the problem is a bit more structural then that. A single professor trains dozens of grad students over the course of their career. Even if you remove every single foreigner, and every single person who does not want to pursue a career in academia, you still end up with dozens of graduates - per professor. The number of jobs available for those graduates? One - that professor's - when he or she retires. And until then, they get to burn the midnight oil, doing grunt work on their projects, in the hopes of competing their dozen collegaues for a shot at that one spot.

adminprof 6y ago

This entire thread is really frustrating for me to read. A lot of these ideas are one-liners from people who have had very little experience with different parts of academia, except maybe being a student. Every single idea here are debated to death by academics already (not just university academics, but funding agencies, academic societies, award panels, journal editors and conference organizers), with changes happening all the time. If it was this simple that a random person here could come up with how to "solve" academia, we'd have already done it decades ago. The ideas also lack nuance and when you get into the definitions of things (for example, p-hacking), then things become a lot more grey; are you allowed to look at a dataset that you spent 2 years collecting if your first hypothesis does not pan out? The clear cut cases are obvious to everyone, it's the grey area that takes 99% of the time to figure out. Imagine reading a thread where everyone is proposing "solutions" to software development. It'd go something like "software development is a cesspool and 80% of it fails (see voting systems, MySpace, electronic health records, Theranos. Here's what software companies need to do:" [yes I'm being intentionally stupid to demonstrate how annoying this is] 1) stop releasing before the bugs are fixed. Software and games are rushed out. Companies need to take their time to fix the bugs so the users don't have to encounter them. 2) no more technical debt. Programmers are sloppy and introduce technical debt because they are not incentivized to do high quality programs. [yes, see how triggering that is] 3) cap team sizes. Everyone knows that large teams fail more spectacularly. Gmail and Napster and the original version of Google were made by a group of 4 people. Software teams need to be 4-5 people max. 4) programmers must use a transparency scorecard. Software companies like Oracle and IBM charge ridiculous amounts for their work. They hide costs and cut corners. Programmers should be transparent about the work they are doing each day, what data they access, and which functions they are writing. These changes need to happen. derp derp

nl 6y ago
Not sure who downvoted this, but this is 100% on the money in my experience. People here are writing comments like that funding should be tied to "how systematic, logical, and well-documented the research is" as though these things are correlated to the existing criteria at all. Much of the criticism here is like someone who knows how to build roads criticising agile software development because it makes it look like no one knows what they are going to build in the end. It's frustrating, and wrong, but the errors are subtle and aggregative.