5 ms·

If you were an NLP researcher at a university whose past years of experience is facing existential threat due to this rapid innovation causing your area to beco

by oars 4y ago

If you were an NLP researcher at a university whose past years of experience is facing existential threat due to this rapid innovation causing your area to become obsolete, what would be some good areas to pivot to or refocus on?

echelon 4y ago
Get out of academia and into industry. Why the hell stay in in academia? This is clearly the next technological wave, and you shouldn't sleep on it. Especially when you're so well positioned to take advantage of your experience. You can make $500,000/yr (maybe more with all the new startups and options) and be on the bleeding edge. If you want to go back to academia later, you can comfortably do so. Most don't, but that doesn't mean it isn't an option.
- matthewdgreen 4y ago
  If you go into industry you’ll be given a chance to deploy these models and rush them into products. You’ll also make good money. If you go into academia (or research, whether it’s in academia or industry) you’ll be given the chance to try to understand what they’re doing. I can see the appeal of making money and rushing products out. But it wouldn’t even begin to compete with my curiosity. Makes me wish I was younger and could start my research career over. ETA: And though it may take longer, people who understand these models will eventually be in possession of the most valuable skill there is. Perhaps one of the last valuable human skills, if things go a certain direction.
  - CuriouslyC 4y ago
    Plot twist: as these models increase in function, complexity and size, behaviors given activations will be as inscrutable to us as our behaviors are given gene and neuron activations.
  - siva7 4y ago
    This is as likely to happen as that someone will fully understand how the brain works. I don't think you're missing much out in academia
    - matthewdgreen 4y ago
      We can’t isolate individual neurons in a functioning brain or train custom models (“probes”) inside of a living human brain that lets us see what they’re feeling on specific inputs. The scope to understand how these models work is incredible: the more intelligent they get, the more we can learn about intelligence works.
  - thwayunion 4y ago
    Do both. Getting your hands dirty is the best way to understand how something works. Think about all the useless SE and PL work that gets done by folks who never programmed for a living, and how often faculty members in those fields with 10 yoe in industry spend their first few years back in academia just slamming ball after ball way out of the park. More importantly, $500K gross is $300K net. Times 5 is $1.5, or time 10 is $3M. That's pretty good "fuck you" money. On top which some industry street cred allows new faculty to opt out of a lot of the ridiculous BS that happens in academia. Seen this time and again. I think the easiest and best path for a fresh NLP phd grad can do right now is find the highest paying industry position, stick it out 5-10 years, then return as a profess of practice and tear it up pre-tenure (or just say f u to the tenure track because who needs tenure when you've got a flush brokerage account?)
    - ilarum 4y ago
      What does "profess of practice and tear it up pre-tenure" mean?
  - akavi 4y ago
    The danger is that the opportunity academia is giving you is something more like "you’ll be given the chance to try to understand what they were doing 5 years ago".
- Beaver117 4y ago
  $500,000 is not a lot after all the inflation we had. $100,000 in 1970 is worth almost $800,000 today. Yes, downvote me all you want. But if you're an NLP expert thinking of working for a company that will make billions off your work, you can and should demand millions at least.
tokai 4y ago
NLP is nowhere near being solved.
- mach1ne 4y ago
  Depending on definition, it is solved.
  - deleted 4y ago
    [deleted]
  - chartpath 4y ago
    Even if that were true, LLMs don't give any kind of "handles" on the semantics. You just get what you get and have to hope it is tuned for your domain. This is 100% fine for generic consumer-facing services where the training data is representative, but for specialized and jargon-filled domains where there has to be a very opinionated interpretation of words, classical NLU is really the only ethical choice IMHO.
  - gattilorenz 4y ago
    You're using the wrong definition, then. /s Where is some evidence that NLP is 'solved'? What does it even mean? OpenAI itself acknowledges the fundamental limitations of ChatGPT and the method of training it, but apparently everybody is happily sweeping them under the rug: "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows." (from https://openai.com/blog/chatgpt https://openai.com/blog/chatgpt ) Certainly ChatGPT/GPT-4 are impressive accomplishments, and it doesn't mean they won't be useful, but we were pretty sure in the past that we had "solved" AI or that we were just about to crack it, just give it a few years... except there's always a new rabbit hole to fall into waiting for you.
    - gonzo41 4y ago

by oars 4y ago

echelon 4y ago

Get out of academia and into industry. Why the hell stay in in academia? This is clearly the next technological wave, and you shouldn't sleep on it. Especially when you're so well positioned to take advantage of your experience. You can make $500,000/yr (maybe more with all the new startups and options) and be on the bleeding edge. If you want to go back to academia later, you can comfortably do so. Most don't, but that doesn't mean it isn't an option.

matthewdgreen 4y ago
If you go into industry you’ll be given a chance to deploy these models and rush them into products. You’ll also make good money. If you go into academia (or research, whether it’s in academia or industry) you’ll be given the chance to try to understand what they’re doing. I can see the appeal of making money and rushing products out. But it wouldn’t even begin to compete with my curiosity. Makes me wish I was younger and could start my research career over. ETA: And though it may take longer, people who understand these models will eventually be in possession of the most valuable skill there is. Perhaps one of the last valuable human skills, if things go a certain direction.
- CuriouslyC 4y ago
  Plot twist: as these models increase in function, complexity and size, behaviors given activations will be as inscrutable to us as our behaviors are given gene and neuron activations.
- siva7 4y ago
  This is as likely to happen as that someone will fully understand how the brain works. I don't think you're missing much out in academia
  - matthewdgreen 4y ago
    We can’t isolate individual neurons in a functioning brain or train custom models (“probes”) inside of a living human brain that lets us see what they’re feeling on specific inputs. The scope to understand how these models work is incredible: the more intelligent they get, the more we can learn about intelligence works.
- thwayunion 4y ago
  Do both. Getting your hands dirty is the best way to understand how something works. Think about all the useless SE and PL work that gets done by folks who never programmed for a living, and how often faculty members in those fields with 10 yoe in industry spend their first few years back in academia just slamming ball after ball way out of the park. More importantly, $500K gross is $300K net. Times 5 is $1.5, or time 10 is $3M. That's pretty good "fuck you" money. On top which some industry street cred allows new faculty to opt out of a lot of the ridiculous BS that happens in academia. Seen this time and again. I think the easiest and best path for a fresh NLP phd grad can do right now is find the highest paying industry position, stick it out 5-10 years, then return as a profess of practice and tear it up pre-tenure (or just say f u to the tenure track because who needs tenure when you've got a flush brokerage account?)
  - ilarum 4y ago
    What does "profess of practice and tear it up pre-tenure" mean?
- akavi 4y ago
  The danger is that the opportunity academia is giving you is something more like "you’ll be given the chance to try to understand what they were doing 5 years ago".
Beaver117 4y ago
$500,000 is not a lot after all the inflation we had. $100,000 in 1970 is worth almost $800,000 today. Yes, downvote me all you want. But if you're an NLP expert thinking of working for a company that will make billions off your work, you can and should demand millions at least.

tokai 4y ago

NLP is nowhere near being solved.

mach1ne 4y ago
Depending on definition, it is solved.
- deleted 4y ago
  [deleted]
- chartpath 4y ago
  Even if that were true, LLMs don't give any kind of "handles" on the semantics. You just get what you get and have to hope it is tuned for your domain. This is 100% fine for generic consumer-facing services where the training data is representative, but for specialized and jargon-filled domains where there has to be a very opinionated interpretation of words, classical NLU is really the only ethical choice IMHO.
- gattilorenz 4y ago
  You're using the wrong definition, then. /s Where is some evidence that NLP is 'solved'? What does it even mean? OpenAI itself acknowledges the fundamental limitations of ChatGPT and the method of training it, but apparently everybody is happily sweeping them under the rug: "ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows." (from https://openai.com/blog/chatgpt https://openai.com/blog/chatgpt ) Certainly ChatGPT/GPT-4 are impressive accomplishments, and it doesn't mean they won't be useful, but we were pretty sure in the past that we had "solved" AI or that we were just about to crack it, just give it a few years... except there's always a new rabbit hole to fall into waiting for you.
  - gonzo41 4y ago