6 ms·

First, a postdoc on the way out of academia is not going to fix academia. Second, how is the system broken? There are mismanaged resources, some nepotism (in t

First, a postdoc on the way out of academia is not going to fix academia. Second, how is the system broken? There are mismanaged resources, some nepotism (in the U.S., a lot of nepotism if we look at countries other than the U.S. or the like), some research questions that are (wrongly) favored, but it doesn't change the fact that many more PhDs with academic career ambitions are being produced than there are (and will be) positions available. And this is not just an academia problem: the same mismatch between supply and demand is found in many other creative and aspirational careers, think actors, singers, sports. Only a small percentage of those who want that life can get it, and those who are left out are often frustrated by the perceived unfairness of life. But that does not change the fact that many aspiring creatives, because of that massive mismatch between supply and demand, will not be able to pursue those careers. Sure, more permanent research positions could be thought of instead of forcing careers through the very narrow bottleneck of tenure track positions, but the vast majority of PhDs with academic ambitions (at least 8 out 10) will not have the opportunity to make that career, and they are (or are themselves) being bread-crumbed for years and years hoping that their dreams will, one day, come true. But they won't come true. And, as soon as they find another fulfilling occupation, they will find out it was not a real dream anyway, just a dream they thought they had.

godelski 3y ago
> Second, how is the system broken? I want to d my best to respond, but can you help me determine my audience? That way I can be clear? Are you coming from inside or outside academia? If inside, what field? If outside, have you gone through a graduate program? Which decade? No need to give highly specific answers, I'm just trying to get some additional context to best respond. There are problems more visible to those inside and often nuanced and problems that are invisible to outside.
- borroka 3y ago
  I was referring to "system broken" with respect to many PhDs with ambitions of an academic career having to leave academia because there are not enough tenure-track positions available. With respect to that problem, it would be equivalent to say that "Hollywood is broken" because many talented aspiring actors cannot find stable work in films and tv shows. It is a supply and demand mismatch problem. I was not referring to the paper review and publication process, funding, grants, teaching etc. Your audience is someone with a PhD in the biological sciences, with more than 10 years of postdoc experience (before leaving for greener pastures a few years ago, after having realized that I was not interested in tenure-track positions anymore and in any case no position had been offered to me) in Europe and in the US, who won grants, awards, competitive scholarships, and have dozens of first-author papers published in some of the most prestigious disciplinary and interdisciplinary journals.
  - godelski 3y ago
    > I was referring to ... many PhDs with ambitions of an academic career having to leave academia... I see. I don't think this makes the system broken and I don't think a large portion of people going into industry is a bad thing. At the end of the day, schooling, even at the PhD level, is training. I am not concerned that we do not have a high retention rate, though I am concerned that the retention is not targeting the best of the best (but that's a different conversation). I do not think this is what most academics are referring to when they mention a broken system. My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. This is the more frequent usage of this phrasing that I see, so I was quite surprised at the mention of your background. The problem I am more concerned about is the entire existential question of academia. It is being treated like a business but academia is explicitly about not being subject to such things like a short term return on investment. I think an important aspect of academia is being able to do risky and low level research where the returns are going to be years or decades. So I think we have a broken system when we are trying to perform high pace output with high impact. I simply do not think you can be fast and revolutionary at the same time (at least consistently). It seems that h-index is of greater concern and this is a compounding metric that does not requisite the highest quality work but rather is more dependent upon frequency and highly influenced by publicity. In my mind, the system is broken when the it is not encouraging academics to be the best scientists they can be. When the system encourages bad science and when a noisy process is held up as if it is the arbiter of truth (journals and conferences are even given the misnomer of "peer review" when this, as you know, takes place before and after venue publication and peers still review "preprint" works that never end up getting "published." The whole language is misleading when you break it down). I'm sure a lot of my feelings come from working in ML where the signal to noise ratio of venue publications is exceptionally low (nearly all CS domains target conferences rather than journals and so there is little to no rebuttal period. The number of submissions in each conference is well over 10k now and you cannot assume a reviewer has domain knowledge in your subfield), but I do believe that it still parallels other domains and that the problem is growing not shrinking. I think the system is broken when we discourage foundational research and the pursuit of knowledge for scientific value. Certainly something is broken when we hire a ton of administrative staff and the administrative tasks given to researchers only increases.
aleph_minus_one 3y ago
> First, a postdoc on the way out of academia is not going to fix academia. I disagree: just found an edtech startup for tertiary education or (post-)graduate level research.
- godelski 3y ago
  Honestly if I knew how to do this I would do it. How do I convince a bunch of billionaires to hand me money to do very fundamental research? Dream would be to have it be very open too and hire people from multiple domains but keep a high focus on being nuance and detail oriented. Honestly, I don't think companies like OpenAI are going to end up being the first to AGI because they get caught in the profit stages and so just pivot to say that more data and more compute is all you need. Maybe it is true, but I doubt it. And most certainly there are other ways considering a 3 pound piece of meat does it using less than a kW per day.
  - aleph_minus_one 3y ago
    > Honestly if I knew how to do this I would do it. Do an MVP in your spare time. I guess if you have a normal programming job in most countries (even in countries where programmers are not paid that well), you will likely be able to pay the bills for, say, server renting, domains, ... For things that you will do likely badly by yourself like design drafts (if you are bad at visual design), ask some friends whether they would be willing to do it for you in exchange for something that you could do for them (so that you don't have to spend money). Start with kinds of research that can be done with little money, but have a possible insanely high impact. Iterate until your platform cannot be ignored anymore.
    - godelski 3y ago
      > Do an MVP in your spare time. I think you gravely misread my comment if this is your first suggestion. I am essentially saying that what I would want to do is closer to starting something akin to DeepMind or what OpenAI looked like in the beginning. There is no clear minimal viable product because the goal is to create general intelligence. There are many demonstrations of narrow cases of generalizability and I even have work that demonstrates this, but the stronger sense and actually trying to build sentient machines. I would make the argument that there is an advantage in what I'm proposing. Since the existing labs that are trying to perform a similar goal are all following similar paths. The chance of beating them using similar methods is quite low. But if you believe (as many researchers do) that the conventional approach is not correct or not efficient, then it reasons that you should take an unconventional approach. I'd also say that my preferred method can be great publicity for them if we do allow for a fully open research platform (open training/models/communication) and given the goals the computational costs are not quite what they are for OpenAI nor DeepMind. As I believe I could do a lot of work with a small team with only a handful of A100 or H100 nodes. That's pocket change to the Musk types and especially if split between multiple entities. It is a high risk high reward situation but I think also low cost.
jebarker 3y ago
> And this is not just an academia problem: the same mismatch between supply and demand is found in many other creative and aspirational careers, think actors, singers, sports I don't often hear people complaining that sports, music, theatre are broken because of this supply/demand mismatch. (People certainly complain about other problems, e.g. streaming music royalties). It's generally accepted that it's a rare and special thing to make a good living in those industries. Why do we view academia differently? Is it because it seems more like "work"?
- borroka 3y ago
  I think it's because people who do research are perceived as someone who wants, out of good heart, to provide a benefit to humanity. "How can you not want me to work on this enzyme pathway that can explain how this very rare and aggressive form of cancer develops?" sounds different from "How can you not want me to star in this silly sitcom?". I used extreme examples, but still.
  - jebarker 3y ago
    Interesting. That perception underestimates how much researchers like doing research I think. The demand shortage and price paid is just function of how much society values research versus entertainment.
    - borroka 3y ago
      That may be so. But it is the same when we talk about surgeons and teachers, who, in the collective subconscious, are seen as saintly figures, people who have dedicated themselves to the betterment of society. But the vast majority of surgeons and teachers love to be surgeons and teachers, love to cut and repair and explain things to young people. In fact, people who teach, for example, sales to employees of a company basically do the same job as high school teachers, but we do not consider them as worthy of our admiration. And as for the price paid by society for research, the public knows approximately nothing about research. And when they think about scientists they think about Albert Einstein, people of genius frequently lost in their thoughts solving equations. But if they knew the reality of most universities, departments, and research groups, they would be willing to pay way less than the state/government is paying scientists and other researchers for their, I dare to say, often meaningless investigations.

godelski 3y ago

> Second, how is the system broken? I want to d my best to respond, but can you help me determine my audience? That way I can be clear? Are you coming from inside or outside academia? If inside, what field? If outside, have you gone through a graduate program? Which decade? No need to give highly specific answers, I'm just trying to get some additional context to best respond. There are problems more visible to those inside and often nuanced and problems that are invisible to outside.

borroka 3y ago
I was referring to "system broken" with respect to many PhDs with ambitions of an academic career having to leave academia because there are not enough tenure-track positions available. With respect to that problem, it would be equivalent to say that "Hollywood is broken" because many talented aspiring actors cannot find stable work in films and tv shows. It is a supply and demand mismatch problem. I was not referring to the paper review and publication process, funding, grants, teaching etc. Your audience is someone with a PhD in the biological sciences, with more than 10 years of postdoc experience (before leaving for greener pastures a few years ago, after having realized that I was not interested in tenure-track positions anymore and in any case no position had been offered to me) in Europe and in the US, who won grants, awards, competitive scholarships, and have dozens of first-author papers published in some of the most prestigious disciplinary and interdisciplinary journals.
- godelski 3y ago
  > I was referring to ... many PhDs with ambitions of an academic career having to leave academia... I see. I don't think this makes the system broken and I don't think a large portion of people going into industry is a bad thing. At the end of the day, schooling, even at the PhD level, is training. I am not concerned that we do not have a high retention rate, though I am concerned that the retention is not targeting the best of the best (but that's a different conversation). I do not think this is what most academics are referring to when they mention a broken system. My complaint about the system being broken is more about the incentive structures and metrics that are used within the academic setting. This is the more frequent usage of this phrasing that I see, so I was quite surprised at the mention of your background. The problem I am more concerned about is the entire existential question of academia. It is being treated like a business but academia is explicitly about not being subject to such things like a short term return on investment. I think an important aspect of academia is being able to do risky and low level research where the returns are going to be years or decades. So I think we have a broken system when we are trying to perform high pace output with high impact. I simply do not think you can be fast and revolutionary at the same time (at least consistently). It seems that h-index is of greater concern and this is a compounding metric that does not requisite the highest quality work but rather is more dependent upon frequency and highly influenced by publicity. In my mind, the system is broken when the it is not encouraging academics to be the best scientists they can be. When the system encourages bad science and when a noisy process is held up as if it is the arbiter of truth (journals and conferences are even given the misnomer of "peer review" when this, as you know, takes place before and after venue publication and peers still review "preprint" works that never end up getting "published." The whole language is misleading when you break it down). I'm sure a lot of my feelings come from working in ML where the signal to noise ratio of venue publications is exceptionally low (nearly all CS domains target conferences rather than journals and so there is little to no rebuttal period. The number of submissions in each conference is well over 10k now and you cannot assume a reviewer has domain knowledge in your subfield), but I do believe that it still parallels other domains and that the problem is growing not shrinking. I think the system is broken when we discourage foundational research and the pursuit of knowledge for scientific value. Certainly something is broken when we hire a ton of administrative staff and the administrative tasks given to researchers only increases.

aleph_minus_one 3y ago

> First, a postdoc on the way out of academia is not going to fix academia. I disagree: just found an edtech startup for tertiary education or (post-)graduate level research.

godelski 3y ago
Honestly if I knew how to do this I would do it. How do I convince a bunch of billionaires to hand me money to do very fundamental research? Dream would be to have it be very open too and hire people from multiple domains but keep a high focus on being nuance and detail oriented. Honestly, I don't think companies like OpenAI are going to end up being the first to AGI because they get caught in the profit stages and so just pivot to say that more data and more compute is all you need. Maybe it is true, but I doubt it. And most certainly there are other ways considering a 3 pound piece of meat does it using less than a kW per day.
- aleph_minus_one 3y ago
  > Honestly if I knew how to do this I would do it. Do an MVP in your spare time. I guess if you have a normal programming job in most countries (even in countries where programmers are not paid that well), you will likely be able to pay the bills for, say, server renting, domains, ... For things that you will do likely badly by yourself like design drafts (if you are bad at visual design), ask some friends whether they would be willing to do it for you in exchange for something that you could do for them (so that you don't have to spend money). Start with kinds of research that can be done with little money, but have a possible insanely high impact. Iterate until your platform cannot be ignored anymore.
  - godelski 3y ago
    > Do an MVP in your spare time. I think you gravely misread my comment if this is your first suggestion. I am essentially saying that what I would want to do is closer to starting something akin to DeepMind or what OpenAI looked like in the beginning. There is no clear minimal viable product because the goal is to create general intelligence. There are many demonstrations of narrow cases of generalizability and I even have work that demonstrates this, but the stronger sense and actually trying to build sentient machines. I would make the argument that there is an advantage in what I'm proposing. Since the existing labs that are trying to perform a similar goal are all following similar paths. The chance of beating them using similar methods is quite low. But if you believe (as many researchers do) that the conventional approach is not correct or not efficient, then it reasons that you should take an unconventional approach. I'd also say that my preferred method can be great publicity for them if we do allow for a fully open research platform (open training/models/communication) and given the goals the computational costs are not quite what they are for OpenAI nor DeepMind. As I believe I could do a lot of work with a small team with only a handful of A100 or H100 nodes. That's pocket change to the Musk types and especially if split between multiple entities. It is a high risk high reward situation but I think also low cost.

jebarker 3y ago

> And this is not just an academia problem: the same mismatch between supply and demand is found in many other creative and aspirational careers, think actors, singers, sports I don't often hear people complaining that sports, music, theatre are broken because of this supply/demand mismatch. (People certainly complain about other problems, e.g. streaming music royalties). It's generally accepted that it's a rare and special thing to make a good living in those industries. Why do we view academia differently? Is it because it seems more like "work"?

borroka 3y ago
I think it's because people who do research are perceived as someone who wants, out of good heart, to provide a benefit to humanity. "How can you not want me to work on this enzyme pathway that can explain how this very rare and aggressive form of cancer develops?" sounds different from "How can you not want me to star in this silly sitcom?". I used extreme examples, but still.
- jebarker 3y ago
  Interesting. That perception underestimates how much researchers like doing research I think. The demand shortage and price paid is just function of how much society values research versus entertainment.
  - borroka 3y ago
    That may be so. But it is the same when we talk about surgeons and teachers, who, in the collective subconscious, are seen as saintly figures, people who have dedicated themselves to the betterment of society. But the vast majority of surgeons and teachers love to be surgeons and teachers, love to cut and repair and explain things to young people. In fact, people who teach, for example, sales to employees of a company basically do the same job as high school teachers, but we do not consider them as worthy of our admiration. And as for the price paid by society for research, the public knows approximately nothing about research. And when they think about scientists they think about Albert Einstein, people of genius frequently lost in their thoughts solving equations. But if they knew the reality of most universities, departments, and research groups, they would be willing to pay way less than the state/government is paying scientists and other researchers for their, I dare to say, often meaningless investigations.