3 ms·

As an NLP professor, yes, I think we're mostly screwed - saying LLMs are a dead end or not a big deal, like some of the interviewed say, is just wishful thinkin

As an NLP professor, yes, I think we're mostly screwed - saying LLMs are a dead end or not a big deal, like some of the interviewed say, is just wishful thinking. A lot of NLP tasks that were subject of active research for decades have just been wiped out. Ironically, the tasks that still aren't solved well by LLMs and can still have a few years of life in them are the most low-level ones, that had become unfashionable in the last 15 years or so - part-of-speech tagging, syntactic parsing, NER. Of course, they have lost a lot of importance as well: you no longer need them for user-oriented downstream tasks. But they may still get some use: for example NER for its own sake is used in biomedical domains, and parsing can be useful for scientific studies of language (say, language universals related to syntax, language evolution, etc.). Which is more than you can say about translation or summarization, which have been pretty much obsoleted by LLMs. Still, even with these tasks, NLP will go from broad applicability to niche. I'm not too worried for my livelihood at the moment (partly because I have tenure, and partly because the academic system works in such a way that zombie fields keep walking for quite long - there are still journals and conferences on the semantic web, which has been a zombie for who knows how long). But it's a pity: I got into this because it was fun and made an impact and now it seems most of my work is going to be irrelevant, like those semantic web researchers I used to look down at. I guess my consolation is that computers that really understand human language was the dream that got me into this in the first place, and it has been realized early. I can play with it and enjoy it while I sink into irrelevant research, I guess :/ Or try to pivot into discrete math or something.

larodi 1y ago
Let's admit it - overnight it became much much harder to be a convincing professor, given each student can use GPTs of all sorts to contradict or otherwise intimidate you. Only a seasoned professor knows the feeling of being bullied by a smart-ass student. Which brings down the total value, the incentive, to teach, and also given the avoidance GPTs silently imprint in students. I mean - why write a program/paper/research when the GPT can do it for you, and save you the suffering. The whole area of algorithms suddenly became more challenging, as now you also have to understand folding multi-dimensional spaces, and retell this all as a nice story for students to remember. We are very likely heading into some dark uncharted era for academia, which will very likely lead to academia shrinking massively. And given the talk of 'all science now happens in big corpos'... I can expect the universities to go back to the original state they started from - monasteries. Saying this all having spent 20+ years as part-time contributor to one such monastery.
- simianwords 1y ago
  Not totally related but I have wondered how someone who thinks they are an expert in a field may deal with contradictions presented by GPT. For example, you may consider yourself an expert on some niche philosophy like say Orientalism. A student can now contradict any theories you can come up with using GPT and the scary thing is that they will be sensible contradictions. I feel like the bar to consider yourself an expert is much higher - you not only have to beat your students but also know enough to beat GPT.
  - horsh1 1y ago
    Why in this story students use GPT and professors don't? If you are an expert -- sit down and start working with GPT on your own. See what it can and what it can not do. See where it helps. See where it hands down lose.
    - simianwords 1y ago
      You are right - this is a new activity professors have to pick up. A latent point in my previous comment was that maybe some professors have not as much expertise as may be required. Now that this expertise is sort of democratised there's more pressure on professors to get better.
    - chunkmonke99 1y ago
      Yeah I agree! I use GPTs (Gemini 2 series and o3 etc.) and they are excellent! But even they sometimes don't quite get the nuance of things or subtly misses the point! There are certain "meta-cognitive" limitations ... I would never bet against the AGI industry and I can only assume that these issues will eventually be solved after we build planets of computronium and force every word and thought any human ever utters to be recorded and trained on. But for now there remain some limitations.
    - larodi 1y ago
      I sure does use it to prepare exams etc. Problem is students don’t seem to see the reason for not using GPT to solve them ;) for me it perfectly makes sense to prepare exams and labs with some AI help… but this questions both my role and the role of in-person education. Really I consider stopping doing it at all…
karel-3d 1y ago
I used to study NLP but before transformers, and now I don't work with NLP/ML/LLMs at all. Can you explain to me this view? LLMs are NLP? We have a model that works, and that works great for many people and many different usages, shouldn't NLP be at its top now? LLMs are not conceptionally that different to other models? I worked with GIZA/MOSES statistical MT back in the day during my studies, it's at the end of the day just matrices that you don't really understand, same as with LLMs?
- justanotherjoe 1y ago
  Its not the same at all. Llm is big yes and thats part of it. But llm is small compared to equivalent performance machine with something like ngram statistical models. You'd need the whole universe. Or something prohibitive like that. And itd still be worse. People don't like it but LLMs 'understand' texts in a very real meaning of the word. Because that's the most compressive way to do the task it's trained to. Is it the same as human understanding? Most likely not, but that complain is cheating.
- Al-Khwarizmi 1y ago
  NLP is indeed at its top, NLP professors aren't :) Imagine if you had stayed in academia and kept working in MT for the last two decades. First of all, now you would see how LLMs render all your work pretty much obsolete. That's already hard for many people. Not so much for me, as I consider myself to be rather adaptable, and maybe not for you either - you can embrace the new thing and start working on it yourself, right? But the problem is that you can't. The main issue is not lack of explainability, but the sheer resources needed. In academia we can't run, let alone train, anything within even one or two orders of magnitude of ChatGPT. We can work with toy models, knowing that we won't get even remotely near the state of the art (many are now doing this, with the excuse of "green AI", sustainability and such, but that won't even hold much longer). Or we can work with the likes of ChatGPT as plain users, but then we are studying the responses of an "oracle" that we don't even have control of, and it starts looking more like religion than science. Ten years ago an academic could beat the state of the art in MT and many other NLP tasks, now for many tasks that's just impossible (unless we count coming up with some clever prompt engineering, but again, religion). For those of us who were in this field because we liked it, not only to make a living, research has become quite unfulfilling.
chunkmonke99 1y ago
[flagged]

larodi 1y ago

Let's admit it - overnight it became much much harder to be a convincing professor, given each student can use GPTs of all sorts to contradict or otherwise intimidate you. Only a seasoned professor knows the feeling of being bullied by a smart-ass student. Which brings down the total value, the incentive, to teach, and also given the avoidance GPTs silently imprint in students. I mean - why write a program/paper/research when the GPT can do it for you, and save you the suffering. The whole area of algorithms suddenly became more challenging, as now you also have to understand folding multi-dimensional spaces, and retell this all as a nice story for students to remember. We are very likely heading into some dark uncharted era for academia, which will very likely lead to academia shrinking massively. And given the talk of 'all science now happens in big corpos'... I can expect the universities to go back to the original state they started from - monasteries. Saying this all having spent 20+ years as part-time contributor to one such monastery.

simianwords 1y ago
Not totally related but I have wondered how someone who thinks they are an expert in a field may deal with contradictions presented by GPT. For example, you may consider yourself an expert on some niche philosophy like say Orientalism. A student can now contradict any theories you can come up with using GPT and the scary thing is that they will be sensible contradictions. I feel like the bar to consider yourself an expert is much higher - you not only have to beat your students but also know enough to beat GPT.
- horsh1 1y ago
  Why in this story students use GPT and professors don't? If you are an expert -- sit down and start working with GPT on your own. See what it can and what it can not do. See where it helps. See where it hands down lose.
  - simianwords 1y ago
    You are right - this is a new activity professors have to pick up. A latent point in my previous comment was that maybe some professors have not as much expertise as may be required. Now that this expertise is sort of democratised there's more pressure on professors to get better.
  - chunkmonke99 1y ago
    Yeah I agree! I use GPTs (Gemini 2 series and o3 etc.) and they are excellent! But even they sometimes don't quite get the nuance of things or subtly misses the point! There are certain "meta-cognitive" limitations ... I would never bet against the AGI industry and I can only assume that these issues will eventually be solved after we build planets of computronium and force every word and thought any human ever utters to be recorded and trained on. But for now there remain some limitations.
  - larodi 1y ago
    I sure does use it to prepare exams etc. Problem is students don’t seem to see the reason for not using GPT to solve them ;) for me it perfectly makes sense to prepare exams and labs with some AI help… but this questions both my role and the role of in-person education. Really I consider stopping doing it at all…

karel-3d 1y ago

I used to study NLP but before transformers, and now I don't work with NLP/ML/LLMs at all. Can you explain to me this view? LLMs are NLP? We have a model that works, and that works great for many people and many different usages, shouldn't NLP be at its top now? LLMs are not conceptionally that different to other models? I worked with GIZA/MOSES statistical MT back in the day during my studies, it's at the end of the day just matrices that you don't really understand, same as with LLMs?

justanotherjoe 1y ago
Its not the same at all. Llm is big yes and thats part of it. But llm is small compared to equivalent performance machine with something like ngram statistical models. You'd need the whole universe. Or something prohibitive like that. And itd still be worse. People don't like it but LLMs 'understand' texts in a very real meaning of the word. Because that's the most compressive way to do the task it's trained to. Is it the same as human understanding? Most likely not, but that complain is cheating.
Al-Khwarizmi 1y ago
NLP is indeed at its top, NLP professors aren't :) Imagine if you had stayed in academia and kept working in MT for the last two decades. First of all, now you would see how LLMs render all your work pretty much obsolete. That's already hard for many people. Not so much for me, as I consider myself to be rather adaptable, and maybe not for you either - you can embrace the new thing and start working on it yourself, right? But the problem is that you can't. The main issue is not lack of explainability, but the sheer resources needed. In academia we can't run, let alone train, anything within even one or two orders of magnitude of ChatGPT. We can work with toy models, knowing that we won't get even remotely near the state of the art (many are now doing this, with the excuse of "green AI", sustainability and such, but that won't even hold much longer). Or we can work with the likes of ChatGPT as plain users, but then we are studying the responses of an "oracle" that we don't even have control of, and it starts looking more like religion than science. Ten years ago an academic could beat the state of the art in MT and many other NLP tasks, now for many tasks that's just impossible (unless we count coming up with some clever prompt engineering, but again, religion). For those of us who were in this field because we liked it, not only to make a living, research has become quite unfulfilling.

chunkmonke99 1y ago

[flagged]