7 ms·
MemGPT: Towards LLMs as Operating Systems
- behnamoh 3y agoPeople have to stop treating LLMs and transformer models like they're the solution to everything. Never before had I seen so much jump on a hype by academics (I'm an academic too). Honestly it feels like mostly clout seeking and a way to improve citation numbers.
- anon291 3y agoI would agree, but on the other hand, Transformer (or attention really) based models seem to be the first time that computers are generating ad hoc human text on a variety of topics, so I do believe the hype is justified. I mean... people have spent entire careers in pursuit of this goal, and it's here... as long as what you want to talk about is less than 4096 / some K tokens. Given how little progress (relatively) was made until transformers, it seems totally reasonable to pursue att ention models.
- necovek 3y agoAnd as long as it is English. How well do they work for other languages with large corpuses? I know they suck for Serbian, but I wonder what kind of corpus they need to become useful?
- blovescoffee 3y agoFwiw I talk to gpt 3.5 in Spanish frequently and there’s no problem that doesn’t exist in the English version.
- necovek 3y agoInteresting: I do wonder about slightly more complex languages which have declensions and verb "gender" (eg. in Serbian "pevala" means "(a female) sang", whereas "pevao" means that a male did. Or nouns and adjectives can be in 7 declensions: "plavom olovkom" means "with a blue pen", whereas "a blue pen" is just "plava olovka". ChatGPT always mixes these up, hallucinates a bunch of words (inappropriate prefixes, declensions etc and is very happy to explain the meaning of these imaginary words), and I can imagine smaller, more complex languages like Serbian needing even larger corpuses than English, yet that's exactly the hard part: there is simply less content to go off of.
- infecto 3y agoI agree they are probably being over-applied but I for one am fascinated and excited for the future and all the possibilities. It’s amazing how much effort is being applied across the board and failures to me just progress other research.
- Scipio_Afri 3y agoThe branding on this is a bit much, it’s not an operating system. However LLMs are the real deal, some papers claim they are achieving SOTA or significant breakthroughs in various domains. Surely they’re computationally intensive, but if you have read the most of the papers out of Berkeley, Microsoft, Meta, Google/DeepMind/Waymo… I think you’d have an change in opinion.
- deleted 3y ago[deleted]
- TerrifiedMouse 3y ago> However LLMs are the real deal Weird thing is it was designed to model language. It’s surprising that it returns sound answers as often as it does. But that’s also kind of the problem, it’s “surprising”, i.e. we don’t really know what happened. You wouldn’t fly on a jetliner that’s “surprising it flies without disintegrating midair”.
- beoberha 3y agoUnless you’re using an LLM to fly a plane, your analogy is a woefully bad comparison.
- TerrifiedMouse 3y agoI guess my point is, if we don’t understand it, we don’t know its failure scenarios.
- omarfarooq 3y agoThere's an LLM for that.
- jackgolding 3y agoThis is the case with all neural nets/black box AI models and a lot are used in various industries.
- 3y ago
- avmich 3y agoJudge it by its merits. If the object of hype adds useful novelty, the interest could be justified. If, as it's often the case, it is not quite known - it's a question to figure out. Granted, intuition is worth something, but it's still not a certainty, so somebody having a different opinion still could see something useful here.
- pylua 3y agoIt deserves to be explored. A lot of research is just applying existing techniques to problems with a few tweaks and documenting the results. I did not read the paper, but it could be that the authors find it is not effective.
- keyle 3y agoI'm a software developer, I live in the concrete world. To put it in layman's terms, LLMs are to software developers what power tools were for tradesmen. Sure you can still use your old screw driver, and for some work, it's not worth getting your electric drill out; but it's a game changer. Sometimes, the hype is justified. I believe at the OS layers, LLM support would make sense. It's full of old mental constructs and "I have to remember how to do this".
- deleted 3y ago[deleted]
- cle 3y agoI'm struggling to find where these authors are treating LLMs and transformers like they're the solution to everything, or where they are just following hype around? These are long-time AI researchers at Berkeley who've developed a framework for dealing with limited context windows, drawing inspiration from operating systems.
- behnamoh 3y ago"LLMs as Operating Systems"... —long-time AI researchers at Berkeley
- danenania 3y agoIn the paper, they're talking about managing LLM context windows the way operating systems manage memory and files. They're not saying LLMs should be used as operating systems.
- dstanko 3y agothey should work on their communication skills.
- another_story 3y agoThis is unhelpful. People should be expected to read past headlines. The paper's abstract takes 30 seconds to read.
- dstanko 3y agoI'm sorry it's unhelpful, but it is how I feel about the article and click-baity title. downwote all you want.
- deleted 3y ago[deleted]
- 3y ago
- IOT_Apprentice 3y agoSure if the initial blockchain hype I remember people wanted to build a OS.
- yieldcrv 3y agoSome of those “OS”es did launch, but theyve been rebranded to SDKs
- greatpostman 3y agoLLMs are a step function change in computers
- gmerc 3y agoTransformers are a primitive we have not fully figured out yet. So it’s perfectly fine to see where this primitive fits. For example, this week several papers on time series forecasts indicate they may have use there. For example they seem to do better job on translation than previous approaches. For example they seem to do a better job at transcription than previous approaches. Probably will do better on OCR than previous approaches. Probably has flaws that limits scenarios requiring high precision we may or may not overcome Possibly will better on autonomous decision making (Does x include a privacy leak should be investigated?) than previous approaches (keyword scanning). We are in a technological wave of discovery and experimentation, calls for restraint of curiosity and research betray fears
- sigmoid10 3y agoIt goes way beyond that. Transformers are the first practical, scalable, general-purpose differentiable (i.e. trainable with gradient descent) algorithm. We haven't come close to seeing the limits of what they can do, because everything so far points to the fact that their only limit is our current hardware. And hardware is improving at a much faster and steadier rate than algorithms in computer science these days.
- esafak 3y agoI'm not sure about the last assertion. How would you even compare them? It is easier to deploy newer algorithms (software) than hardware.
- sigmoid10 3y agoThere's two ways you can solve currently intractable problems: Find better algorithms or improve the hardware. It's actually insanely hard to come up with new algorithms, that is why machine learning and AI were lagging behind most of computer science for decades.
- deleted 3y ago[deleted]
- rvz 3y agoPrecisely this. This doesn't solve anything and appears to be absurd techno-solutionism at this point. Every single problem does not need to be solved using an LLM, which this paper tells us the amount of desperation in this hype cycle. Those reading this paper and giving it credibility have fallen for this nonsense and are probably gullible enough to believe in this non use-case.
- Philpax 3y agoDid you read the paper? What exactly do you think it does?
- danielbln 3y ago> Every single problem does not need to be solved using an LLM How do you know if a problem is suitable for transformers/LLMs unless you try? We have this great generalized architecture, I would hope people throw everything at it and see what sticks, because what's the downside? Less focus on bespoke predictive models? Oh no.
- nyolfen 3y agothey are very clearly the most powerful technological innovation i've witnessed in my adult life. we don't even know what they're capable of yet, we're like one year into discovering practical usecases and they are improving at a constant dizzying rate. this is the time for exploration and experimentation.
- hlfshell 3y agoWe have found a new hammer and everything very much looks like a nail. It's perfectly natural and allows us to figure out what does and doesn't work, even if it means sometimes we have to deal with empty hype projects.
- danielbln 3y agoThis wave of AI hype really brought out the curmudgeons, didn't it? Here we are, with this exciting architecture that is surprisingly applicable to a wide range of problem domains, yet we get "this is like crypto all over again!!1" and "oh everything looks like a nail now, huh?". I mean dang.
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- lyapunova 3y agoI think the problem with stuff like this is that in many cases the authors have an idea like "LLMS + ???? = Operating System !" which is not that hard of an idea to come up with. Generally in each instance of LLM + ???? = "thing", the first papers that come out to fill the ???? with an answer do that in a rush to get on arxiv and so naturally the work is lackluster (since they have barely had time to think about an actual good solution for the ????).
- SV_BubbleTime 3y agoDid you think these people just stopped existing when crypto took a dump? Blockchain all the things… LLM all the things! At least LLMs have a very obvious use from day1… but an OS is not one of them.
- lacrimacida 3y agoYou’re probably thinking of a traditional OS. This could be an interesting project though.
- blovescoffee 3y agoDid you read the paper before commenting?
- BasilPH 3y agoMy initial reaction when reading the title was the same. Their abstract does a better job at explaining what they actually do, and where the connection with an OS comes in: > To enable using context beyond limited context windows, we propose virtual context management, a technique drawing inspiration from hierarchical memory systems in traditional operating systems that provide the appearance of large memory resources through data movement between fast and slow memory.
- pacjam 3y agoHey OP, thank you for the post. In this work, we are looking at one particular functionality of an OS - memory management. The project started off motivated by trying to extend context length for a project we were working on. The analogy to OSes came naturally as the project progressed and there was growing similarity to the caching and memory hierarchy relation.
- pacjam 3y agoHey all, I’m the lead author on this work. Thanks to the OP for posting, and for all the interest and feedback. To clarify a few points: the goal of this work is to investigate the extent to which an LLM can manage memory and different memory hierarchies, applying lessons from operating systems to extend effective context lengths. All of our code is open sourced at https://github.com/cpacker/MemGPT https://github.com/cpacker/MemGPT so you can try it out yourselves!
- a1j9o94 3y agoThis is a fascinating approach! I posted this as a separate comment but would be curious about your response directly. What do you think about the tradeoff of having an agent manage its own memory as opposed to having a separate agent whose job it is to manage the memory and the other agent just focuses on the conversation?
- sabareesh 3y agowell basically this is what i have been doing since functions introduced part of chatgpt api
- tayo42 3y agoThis comment section is wild right now btw, no one is responding to the content, just the, i guess, misleading title. The OS in the title is making a comparison to how page caches work. Relevant data is moved into the context the llm needs to answer chat questions. When it runs out of memory it moves a condensed, searchable version of the content to another data store. Its trying to solve the problem of relatively low max token limits. I wonder if this makes it easier to run on small devices?
- pacjam 3y agoThanks for checking out our work! Yeah, great point, this is even more critical in those scenarios when you have very limited memory. You can play around with different context sizes using the code on GitHub: https://github.com/cpacker/MemGPT/blob/main/memgpt/constants.py#L16 https://github.com/cpacker/MemGPT/blob/main/memgpt/constants...
- tayo42 3y agoThats cool, thanks for sharing the code!
- deleted 3y ago[deleted]
- danielmarkbruce 3y agoThis paper could be interesting but the title and analogy are so absurd sounding that most people won't read it.
- deleted 3y ago[deleted]
- skrebbel 3y agoSorry if this is a bit meta but why is it so popular and accepted to Just Say Stuff in AI land? Sometimes it seems to me that AI people have begun hallucinating as much as the models they’re training. This article seems pretty cool but in no way has it anything to do with its title. “Operating system” has a clearly defined meaning and it’s not “a thing that has memory”. To me these kinds of grandiose claims with no substance undermine the credibility of the authors. I mean what’s wrong with “Tiered memory layers to provide extended AI context windows”?
- xamde 3y agoThis ist really a succint description: "Sometimes it seems to me that AI people have begun hallucinating as much as the models they’re training."
- isoprophlex 3y agoThe NFT/web3/shitcoin grifters had their bubble burst, so now they're gravitating towards AI.
- kalium-xyz 3y agoThey already owned the video cards. Might as Well
- HPsquared 3y agoNvidia seems to be doing pretty well, at least.
- TeMPOraL 3y agoWe always thought dangerous AIs will be funded by the military, or out of internal R&D budgets of evil multinational megacorps-that-do-everything. Turns out instead it's all been funded by grifters and scammers, both large and small. Literally. It's not DARPA or some Umbrella Corporation, but the advertising companies that did all the AI R&D, while shitcoiners spent a decade unintentionally funding the hardware AI needs. Strange reality we live in.
- mgaunard 3y agoAs a software engineer, the last thing I need from my operating system is to be a LLM. Thankfully, this paper is not that; the name is just misleading.
- deleted 3y ago[deleted]
- isatty 3y agoIt’s an interesting paper by credible academics but the name is just downright terrible. I suppose I’m extra annoyed because I don’t really have an interest in LLMs to begin with but I do in operating systems.
- trenchgun 3y agoIt is a horrible name, but probably gets more attention this way. Paper itself seems good.
- dang 3y agoAs there is another thread about this currently on the front page, I've merged all the comments that are not just bickering about the title into that one: https://news.ycombinator.com/item?id=37901902 https://news.ycombinator.com/item?id=37901902
- arxiv_papers 3y ago[dead]