4 ms·
Everyone who did NLP research or product discovery in the past 5 years had to pivot real hard to salvage their shit post-transformers. They're very disruptively
by isoprophlex 1y ago
Everyone who did NLP research or product discovery in the past 5 years had to pivot real hard to salvage their shit post-transformers. They're very disruptively good at most NLP task
edit: post-transformers meaning "in the era after transformers were widely adopted" not some mystical new wave of hypothetical tech to disrupt transformers themselves.
- rootnod3 1y agoSo, unless this went r/woosh over my head....how is current AI better than shit post-transformers? If all....old shit post-transformers are at least deterministic or open and not a randomized shitbox. Unless I misinterpreted the post, render me confused.
- dgacmu 1y agoI think you're misinterpreting: "with the advent of transformers, (many) people doing NLP with pre-transformers techniques had to salvage their shit"
- rootnod3 1y agoI guess. That's why I added the "unless I am mis-interpreting", still got downvoted for it because I guess it was against AI. The wording was confusing but so was my understanding of it as a non-native speaker. Shit happens.
- dgacmu 1y agoI agree that the wording was a bit confusing (as a native speaker).
- isoprophlex 1y agoI wasn't too clear, I think. Apologies if the wording was confusing. People who started their NLP work (PhDs etc; industry research projects) before the LLM / transformer craze had to adapt to the new world. (Hence 'post-mass-uptake-of-transformers')
- rootnod3 1y agoAh ok, then I did misunderstand a lot. That makes sense. And I do like the non AI based NLP work.
- numpad0 1y agoThere's no post-transformer tech. There are lots of NLP tasks that you can now, just, prompt an LLM to do.
- isoprophlex 1y agoYeah unclear wording; see the sibling comment also. I meant "the tech we have now", in the era after "attention is all you need"
- dingnuts 1y agoSorry but you didn't really answer the question. The original claim was that transformers changed a whole bunch of fields, and you listed literally the one thing language models are directly useful for.. modeling language. I think this might be the ONLY example that doesn't back up the original claim, because of course an advancement in language processing is an advancement in language processing -- that's tautological! every new technology is an advancement in its domain; what's claimed to be special about transformers is that they are allegedly disruptive OUTSIDE of NLP. "Which fields have been transformed?" means ASIDE FROM language processing. other than disrupting users by forcing "AI" features they don't want on them... what examples of transformers being revolutionary exist outside of NLP? Claude Code? lol
- deleted 1y ago[deleted]
- iknowstuff 1y agohttps://x.com/aelluswamy/status/1981760576591393203 https://x.com/aelluswamy/status/1981760576591393203 saving lives
- dingnuts 1y agoI'm not watching a video on Twitter about self driving from the company who told us twelve years ago that completely autonomous vehicles were a year away as a rebuttal to the point I made. If you have something relevant to say, you can summarize for the class & include links to your receipts.
- iknowstuff 1y agoyour choice, I don't really care about your opinion
- lawlessone 1y agoThen why reply to them?
- GuB-42 1y agoThey are now trying to understand why transformers are so good. That's the thing with deep learning in general, people don't really understand what they are doing. It is a game of throwing stuff at the wall and see what sticks. NLP researchers are trying to open up these neural networks and try to understand where the familiar structures of language form. I think it is important research. Both for improving models and to better understand language. Traditional NLP research is seen as obsolete by some but I think it is more relevant than ever. We can think of transformer-based LLMs as a life form we have created by accident and NLP researchers as biologists studying it, where companies like OpenAI and DeepSeek are more like breeders.