4 ms·
People have to stop treating LLMs and transformer models like they're the solution to everything. Never before had I seen so much jump on a hype by academics (I
by behnamoh 3y ago
People have to stop treating LLMs and transformer models like they're the solution to everything. Never before had I seen so much jump on a hype by academics (I'm an academic too). Honestly it feels like mostly clout seeking and a way to improve citation numbers.
- anon291 3y agoI would agree, but on the other hand, Transformer (or attention really) based models seem to be the first time that computers are generating ad hoc human text on a variety of topics, so I do believe the hype is justified. I mean... people have spent entire careers in pursuit of this goal, and it's here... as long as what you want to talk about is less than 4096 / some K tokens. Given how little progress (relatively) was made until transformers, it seems totally reasonable to pursue att ention models.
- necovek 3y agoAnd as long as it is English. How well do they work for other languages with large corpuses? I know they suck for Serbian, but I wonder what kind of corpus they need to become useful?
- blovescoffee 3y agoFwiw I talk to gpt 3.5 in Spanish frequently and there’s no problem that doesn’t exist in the English version.
- necovek 3y agoInteresting: I do wonder about slightly more complex languages which have declensions and verb "gender" (eg. in Serbian "pevala" means "(a female) sang", whereas "pevao" means that a male did. Or nouns and adjectives can be in 7 declensions: "plavom olovkom" means "with a blue pen", whereas "a blue pen" is just "plava olovka". ChatGPT always mixes these up, hallucinates a bunch of words (inappropriate prefixes, declensions etc and is very happy to explain the meaning of these imaginary words), and I can imagine smaller, more complex languages like Serbian needing even larger corpuses than English, yet that's exactly the hard part: there is simply less content to go off of.
- infecto 3y agoI agree they are probably being over-applied but I for one am fascinated and excited for the future and all the possibilities. It’s amazing how much effort is being applied across the board and failures to me just progress other research.
- Scipio_Afri 3y agoThe branding on this is a bit much, it’s not an operating system. However LLMs are the real deal, some papers claim they are achieving SOTA or significant breakthroughs in various domains. Surely they’re computationally intensive, but if you have read the most of the papers out of Berkeley, Microsoft, Meta, Google/DeepMind/Waymo… I think you’d have an change in opinion.
- deleted 3y ago[deleted]
- TerrifiedMouse 3y ago> However LLMs are the real deal Weird thing is it was designed to model language. It’s surprising that it returns sound answers as often as it does. But that’s also kind of the problem, it’s “surprising”, i.e. we don’t really know what happened. You wouldn’t fly on a jetliner that’s “surprising it flies without disintegrating midair”.
- beoberha 3y agoUnless you’re using an LLM to fly a plane, your analogy is a woefully bad comparison.
- TerrifiedMouse 3y agoI guess my point is, if we don’t understand it, we don’t know its failure scenarios.
- omarfarooq 3y agoThere's an LLM for that.
- jackgolding 3y agoThis is the case with all neural nets/black box AI models and a lot are used in various industries.
- 3y ago
- avmich 3y agoJudge it by its merits. If the object of hype adds useful novelty, the interest could be justified. If, as it's often the case, it is not quite known - it's a question to figure out. Granted, intuition is worth something, but it's still not a certainty, so somebody having a different opinion still could see something useful here.
- pylua 3y agoIt deserves to be explored. A lot of research is just applying existing techniques to problems with a few tweaks and documenting the results. I did not read the paper, but it could be that the authors find it is not effective.
- keyle 3y agoI'm a software developer, I live in the concrete world. To put it in layman's terms, LLMs are to software developers what power tools were for tradesmen. Sure you can still use your old screw driver, and for some work, it's not worth getting your electric drill out; but it's a game changer. Sometimes, the hype is justified. I believe at the OS layers, LLM support would make sense. It's full of old mental constructs and "I have to remember how to do this".
- deleted 3y ago[deleted]
- cle 3y agoI'm struggling to find where these authors are treating LLMs and transformers like they're the solution to everything, or where they are just following hype around? These are long-time AI researchers at Berkeley who've developed a framework for dealing with limited context windows, drawing inspiration from operating systems.
- behnamoh 3y ago"LLMs as Operating Systems"... —long-time AI researchers at Berkeley
- danenania 3y agoIn the paper, they're talking about managing LLM context windows the way operating systems manage memory and files. They're not saying LLMs should be used as operating systems.
- dstanko 3y agothey should work on their communication skills.
- another_story 3y agoThis is unhelpful. People should be expected to read past headlines. The paper's abstract takes 30 seconds to read.
- dstanko 3y agoI'm sorry it's unhelpful, but it is how I feel about the article and click-baity title. downwote all you want.
- deleted 3y ago[deleted]
- 3y ago
- IOT_Apprentice 3y agoSure if the initial blockchain hype I remember people wanted to build a OS.
- yieldcrv 3y agoSome of those “OS”es did launch, but theyve been rebranded to SDKs
- greatpostman 3y agoLLMs are a step function change in computers
- gmerc 3y agoTransformers are a primitive we have not fully figured out yet. So it’s perfectly fine to see where this primitive fits. For example, this week several papers on time series forecasts indicate they may have use there. For example they seem to do better job on translation than previous approaches. For example they seem to do a better job at transcription than previous approaches. Probably will do better on OCR than previous approaches. Probably has flaws that limits scenarios requiring high precision we may or may not overcome Possibly will better on autonomous decision making (Does x include a privacy leak should be investigated?) than previous approaches (keyword scanning). We are in a technological wave of discovery and experimentation, calls for restraint of curiosity and research betray fears
- sigmoid10 3y agoIt goes way beyond that. Transformers are the first practical, scalable, general-purpose differentiable (i.e. trainable with gradient descent) algorithm. We haven't come close to seeing the limits of what they can do, because everything so far points to the fact that their only limit is our current hardware. And hardware is improving at a much faster and steadier rate than algorithms in computer science these days.
- esafak 3y agoI'm not sure about the last assertion. How would you even compare them? It is easier to deploy newer algorithms (software) than hardware.
- sigmoid10 3y agoThere's two ways you can solve currently intractable problems: Find better algorithms or improve the hardware. It's actually insanely hard to come up with new algorithms, that is why machine learning and AI were lagging behind most of computer science for decades.
- deleted 3y ago[deleted]
- rvz 3y agoPrecisely this. This doesn't solve anything and appears to be absurd techno-solutionism at this point. Every single problem does not need to be solved using an LLM, which this paper tells us the amount of desperation in this hype cycle. Those reading this paper and giving it credibility have fallen for this nonsense and are probably gullible enough to believe in this non use-case.
- Philpax 3y agoDid you read the paper? What exactly do you think it does?
- danielbln 3y ago> Every single problem does not need to be solved using an LLM How do you know if a problem is suitable for transformers/LLMs unless you try? We have this great generalized architecture, I would hope people throw everything at it and see what sticks, because what's the downside? Less focus on bespoke predictive models? Oh no.
- nyolfen 3y agothey are very clearly the most powerful technological innovation i've witnessed in my adult life. we don't even know what they're capable of yet, we're like one year into discovering practical usecases and they are improving at a constant dizzying rate. this is the time for exploration and experimentation.
- hlfshell 3y agoWe have found a new hammer and everything very much looks like a nail. It's perfectly natural and allows us to figure out what does and doesn't work, even if it means sometimes we have to deal with empty hype projects.
- danielbln 3y agoThis wave of AI hype really brought out the curmudgeons, didn't it? Here we are, with this exciting architecture that is surprisingly applicable to a wide range of problem domains, yet we get "this is like crypto all over again!!1" and "oh everything looks like a nail now, huh?". I mean dang.
- deleted 3y ago[deleted]