4 ms·

I was referring to "system broken" with respect to many PhDs with ambitions of an academic career having to leave academia because there are not enough tenure-t

I was referring to "system broken" with respect to many PhDs with ambitions of an academic career having to leave academia because there are not enough tenure-track positions available. With respect to that problem, it would be equivalent to say that "Hollywood is broken" because many talented aspiring actors cannot find stable work in films and tv shows. It is a supply and demand mismatch problem. I was not referring to the paper review and publication process, funding, grants, teaching etc. Your audience is someone with a PhD in the biological sciences, with more than 10 years of postdoc experience (before leaving for greener pastures a few years ago, after having realized that I was not interested in tenure-track positions anymore and in any case no position had been offered to me) in Europe and in the US, who won grants, awards, competitive scholarships, and have dozens of first-author papers published in some of the most prestigious disciplinary and interdisciplinary journals.

godelski 3y ago
> I was referring to ... many PhDs with ambitions of an academic career having to leave academia... I see. I don't think this makes the system broken and I don't think a large portion of people going into industry is a bad thing. At the end of the day, schooling, even at the PhD level, is training. I am not concerned that we do not have a high retention rate, though I am concerned that the retention is not targeting the best of the best (but that's a different conversation). I do not think this is what most academics are referring to when they mention a broken system. My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. This is the more frequent usage of this phrasing that I see, so I was quite surprised at the mention of your background. The problem I am more concerned about is the entire existential question of academia. It is being treated like a business but academia is explicitly about not being subject to such things like a short term return on investment. I think an important aspect of academia is being able to do risky and low level research where the returns are going to be years or decades. So I think we have a broken system when we are trying to perform high pace output with high impact. I simply do not think you can be fast and revolutionary at the same time (at least consistently). It seems that h-index is of greater concern and this is a compounding metric that does not requisite the highest quality work but rather is more dependent upon frequency and highly influenced by publicity. In my mind, the system is broken when the it is not encouraging academics to be the best scientists they can be. When the system encourages bad science and when a noisy process is held up as if it is the arbiter of truth (journals and conferences are even given the misnomer of "peer review" when this, as you know, takes place before and after venue publication and peers still review "preprint" works that never end up getting "published." The whole language is misleading when you break it down). I'm sure a lot of my feelings come from working in ML where the signal to noise ratio of venue publications is exceptionally low (nearly all CS domains target conferences rather than journals and so there is little to no rebuttal period. The number of submissions in each conference is well over 10k now and you cannot assume a reviewer has domain knowledge in your subfield), but I do believe that it still parallels other domains and that the problem is growing not shrinking. I think the system is broken when we discourage foundational research and the pursuit of knowledge for scientific value. Certainly something is broken when we hire a ton of administrative staff and the administrative tasks given to researchers only increases.
- borroka 3y ago
  > My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. We have to keep in mind that those incentives, targets, and metrics were set up by the academics themselves. It is the same in funding agencies and departments.
  - godelski 3y ago
    I'd be a bit hard pressed to agree with this. While I think it is technically correct I do not think it is meaningful. Because anyone working in the academic system is specifically considered an academic. But a researcher is not the same thing as an admin. The truth is that a dean is not the same thing as a researcher even if they previously were. I'd specify that the metrics were setup by the bureaucrats, which does include overlap between administrators and researchers. I'd also add that just because the metrics were made up by these groups doesn't mean they can't decide to move past them. In fact, it even makes the case of internally solving the problem stronger because we did it before so we can do it again.

godelski 3y ago

> I was referring to ... many PhDs with ambitions of an academic career having to leave academia... I see. I don't think this makes the system broken and I don't think a large portion of people going into industry is a bad thing. At the end of the day, schooling, even at the PhD level, is training. I am not concerned that we do not have a high retention rate, though I am concerned that the retention is not targeting the best of the best (but that's a different conversation). I do not think this is what most academics are referring to when they mention a broken system. My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. This is the more frequent usage of this phrasing that I see, so I was quite surprised at the mention of your background. The problem I am more concerned about is the entire existential question of academia. It is being treated like a business but academia is explicitly about not being subject to such things like a short term return on investment. I think an important aspect of academia is being able to do risky and low level research where the returns are going to be years or decades. So I think we have a broken system when we are trying to perform high pace output with high impact. I simply do not think you can be fast and revolutionary at the same time (at least consistently). It seems that h-index is of greater concern and this is a compounding metric that does not requisite the highest quality work but rather is more dependent upon frequency and highly influenced by publicity. In my mind, the system is broken when the it is not encouraging academics to be the best scientists they can be. When the system encourages bad science and when a noisy process is held up as if it is the arbiter of truth (journals and conferences are even given the misnomer of "peer review" when this, as you know, takes place before and after venue publication and peers still review "preprint" works that never end up getting "published." The whole language is misleading when you break it down). I'm sure a lot of my feelings come from working in ML where the signal to noise ratio of venue publications is exceptionally low (nearly all CS domains target conferences rather than journals and so there is little to no rebuttal period. The number of submissions in each conference is well over 10k now and you cannot assume a reviewer has domain knowledge in your subfield), but I do believe that it still parallels other domains and that the problem is growing not shrinking. I think the system is broken when we discourage foundational research and the pursuit of knowledge for scientific value. Certainly something is broken when we hire a ton of administrative staff and the administrative tasks given to researchers only increases.

borroka 3y ago
> My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. We have to keep in mind that those incentives, targets, and metrics were set up by the academics themselves. It is the same in funding agencies and departments.
- godelski 3y ago
  I'd be a bit hard pressed to agree with this. While I think it is technically correct I do not think it is meaningful. Because anyone working in the academic system is specifically considered an academic. But a researcher is not the same thing as an admin. The truth is that a dean is not the same thing as a researcher even if they previously were. I'd specify that the metrics were setup by the bureaucrats, which does include overlap between administrators and researchers. I'd also add that just because the metrics were made up by these groups doesn't mean they can't decide to move past them. In fact, it even makes the case of internally solving the problem stronger because we did it before so we can do it again.