5 ms·

Sorry if this is a bit meta but why is it so popular and accepted to Just Say Stuff in AI land? Sometimes it seems to me that AI people have begun hallucinating

Sorry if this is a bit meta but why is it so popular and accepted to Just Say Stuff in AI land? Sometimes it seems to me that AI people have begun hallucinating as much as the models they’re training. This article seems pretty cool but in no way has it anything to do with its title. “Operating system” has a clearly defined meaning and it’s not “a thing that has memory”. To me these kinds of grandiose claims with no substance undermine the credibility of the authors. I mean what’s wrong with “Tiered memory layers to provide extended AI context windows”?

xamde 3y ago
This ist really a succint description: "Sometimes it seems to me that AI people have begun hallucinating as much as the models they’re training."
isoprophlex 3y ago
The NFT/web3/shitcoin grifters had their bubble burst, so now they're gravitating towards AI.
- kalium-xyz 3y ago
  They already owned the video cards. Might as Well
  - HPsquared 3y ago
    Nvidia seems to be doing pretty well, at least.
    - TeMPOraL 3y ago
      We always thought dangerous AIs will be funded by the military, or out of internal R&D budgets of evil multinational megacorps-that-do-everything. Turns out instead it's all been funded by grifters and scammers, both large and small. Literally. It's not DARPA or some Umbrella Corporation, but the advertising companies that did all the AI R&D, while shitcoiners spent a decade unintentionally funding the hardware AI needs. Strange reality we live in.
      - yowlingcat 3y ago
        If you told me a decade ago that this is where we would land, I would tell you to at least make the lie a bit more believable. And yet here we are! Strange times indeed.
      - ElevenLathe 3y ago
        Agreed, it feels very Stross-ian.
    - classified 3y ago
      Selling shovels makes you the most money in a gold rush.
      - cbm-vic-20 3y ago
        I've never seen the historic mansions of shovel sellers. I've seen many of railroad owners.
        muzani 3y ago
        Well, check out the insane profits (not just revenue) NVIDIA has made this year. That profit margin can fit quite a few mansions as well as the biggest AI companies. People are joking about the crypto bros funding it, but the leap in profit suggests it's not crypto fueled at all.
        kridsdale3 3y ago
        A railroad is on the continuum that has shovels in it. The job is to get ore from the mine in to a processor. 1. Use you hands 2. Use a pick or shovel in to a bucket and carry it 3. Use a wheelbarrow 4. Use a mine cart 5. Put a steam engine on your minecart
- danielbln 3y ago
  I hate this take, no matter how often it's repeated. Yes there is hype fever pitch around AI, yes it attracts grifters (as does every hype), but in comparison to crypto, at the very least gen AI provides tractable benefits in a wide array of usecases. What has nft/web3/x-coins ever given us apart from speculative investments (aka gambling)?
  - thesuperbigfrog 3y ago
    >> What has nft/web3/x-coins ever given us apart from speculative investments (aka gambling)? Cryptocurrencies modernized the malware industry. Before cryptocurrencies, ransomware used to require victims to pay using gift cards or other ridiculous means. After cryptocurrencies, victims could now pay their "captors" directly over the Internet in cryptocurrency. Before cryptocurrencies, malware entities and actors acted largely independently unless they were part of a state-sponsored group or larger organized crime "business". After cryptocurrencies, "ransomware as a service" emerged where different groups specialize in particular components and services of ransomware: tools to establish footholds, tools to take over and encrypt victim systems, services to demand and take payments, and services to wash and launder payments. Payments between these groups are made in cryptocurrency.
  - swalsh 3y ago
    The last bull saw the rise of all kinds of new technologies which will enable the use cases people were talking about (Avalanche is my favorite example, but theres some new stuff like SUI coming). The thing web3 did absolutely right is create a true interchangeable format (ie tokens) the downside is this technology has been developing sooooooo slowly. Whenever people start to get hope the technology might be ready for use, the gambling use cases rise to the top and dominate. I think crypto is also dominated by a certain hardline techy. Or at least it's critics are. Crypto should be working side by side with traditional finance. Instead they view it as their enemy. FTX US is an example of how regulation can be good (the funds on FTX-US were more or less unmolested) yet most crypto people seem to argue against any regulation at all. It will be their demise.
pacjam 3y ago
Thanks for your feedback @skrebbel. We called it MemGPT - in reference to teaching LLMs how to manage their own memory hierarchy. The idea and title evolved when we realized we were teaching the LLM to read/write, and this enabled perpetual chatbots, doc QA, etc. Various capabilities by a few simple abstractions, inspired by the original unix paper's 4 key abstractions - read/write/open/close. Further, in the title we say "Towards", indicating a qualified step in that direction.
- __void 3y ago
  still this is not an "operating system" nor does it in any way related to them; please use the right words when you have to communicate, otherwise you just sound like scammers who want to sell pots with holes by passing them off as colanders
- skrebbel 3y ago
  Perpetual chatbots are a super cool achievement but I fail to see how that resembles an OS, a thing that handles files and lets you run programs, in any way. If MemGPT is as spectacular as you suggest then I think an honest, down to earth title would have gotten you a lot more attention. It seems to me that limited context windows are one of the key roadblocks to wider real-world LLM applicability, beyond demos that do well on Twitter I mean.
  - selestify 3y ago
    > If MemGPT is as spectacular as you suggest then I think an honest, down to earth title would have gotten you a lot more attention. The fact that the current title got onto the HN front page suggests otherwise when it comes to incentives for picking submission titles.
- abdellah123 3y ago
  > we were teaching the LLM to read/write Very interesting.
- ludwik 3y ago
  I think a title like "MemGPT: Operating System inspired memory management for LLMs" would be okay. The current title makes it seem as if you are building an OS based on the LLM technology, which is quite misleading.
deleted 3y ago
[deleted]
lgessler 3y ago
IMO, as an NLP researcher, it's a product of the "fast science" culture of AI where there are more papers than anyone could possibly read even in a given subfield at a particular conference, and an "old" paper is anything that was published over two years ago. (Who knew Bollywood movies and AI publications would have so much in common?) Maybe there are good reasons why publications should move so much more quickly in AI as opposed to, say, scholastic philosophy, but it's undeniable that it has become a struggle to get others to even read an abstract of your work, and as such we should not be surprised when sensationalization verges on becoming a requirement for your work to get engagement. So to answer your question--perhaps cynically--“Tiered memory layers to provide extended AI context windows” is not the best title you could have for an AI paper because it has all the rizz of a shipping manifest. If you want to maximize citations, you need to market more. I also considered whether to blame the field's very young reviewer population for not having the proper disdain for sensationalization that I'd expect from an older researcher who would surely have more restraint than to speak so much more of the sizzle than of the steak, but then I remembered that the paper which introduced the Transformer model was titled "Attention Is All You Need".
- EncomLab 3y ago
  I love this response because it's at least honest about the reality of "scientific publishing" today - who cares about the accuracy, verifiability or ability to replicate - you just need to maximize your citations via hype!!"
- althea_tx 3y ago
  This isn’t a slight against OP’s paper, but it’s worth noting that arXiv is not peer reviewed.
quickthrower2 3y ago
Attention is all you need. People’s attention that is :-)
amelius 3y ago
This is why we need to get AI (or DL) out of the realm of CS, quickly. Similar to how psychology is a different field than physics even if the former is built on the latter.
fritzo 3y ago
In defense of the authors, I find it helpful to explore analogies between the nascent paradigm of LLM programming and other much older conceptual frames. This paper isn't just tiered memory, but also the analogy between an LLM and a CPU, interrupts, event processing etc.
ultra_nick 3y ago
Are you new to software? The bugs in programs aren't actual bugs either, windows aren't actual windows, and many would argue that computer science is still more of an art than a science.

xamde 3y ago

This ist really a succint description: "Sometimes it seems to me that AI people have begun hallucinating as much as the models they’re training."

isoprophlex 3y ago

The NFT/web3/shitcoin grifters had their bubble burst, so now they're gravitating towards AI.

kalium-xyz 3y ago
They already owned the video cards. Might as Well
- HPsquared 3y ago
  Nvidia seems to be doing pretty well, at least.
  - TeMPOraL 3y ago
    We always thought dangerous AIs will be funded by the military, or out of internal R&D budgets of evil multinational megacorps-that-do-everything. Turns out instead it's all been funded by grifters and scammers, both large and small. Literally. It's not DARPA or some Umbrella Corporation, but the advertising companies that did all the AI R&D, while shitcoiners spent a decade unintentionally funding the hardware AI needs. Strange reality we live in.
    - yowlingcat 3y ago
      If you told me a decade ago that this is where we would land, I would tell you to at least make the lie a bit more believable. And yet here we are! Strange times indeed.
    - ElevenLathe 3y ago
      Agreed, it feels very Stross-ian.
  - classified 3y ago
    Selling shovels makes you the most money in a gold rush.
    - cbm-vic-20 3y ago
      I've never seen the historic mansions of shovel sellers. I've seen many of railroad owners.
      - muzani 3y ago
        Well, check out the insane profits (not just revenue) NVIDIA has made this year. That profit margin can fit quite a few mansions as well as the biggest AI companies. People are joking about the crypto bros funding it, but the leap in profit suggests it's not crypto fueled at all.
      - kridsdale3 3y ago
        A railroad is on the continuum that has shovels in it. The job is to get ore from the mine in to a processor. 1. Use you hands 2. Use a pick or shovel in to a bucket and carry it 3. Use a wheelbarrow 4. Use a mine cart 5. Put a steam engine on your minecart
danielbln 3y ago
I hate this take, no matter how often it's repeated. Yes there is hype fever pitch around AI, yes it attracts grifters (as does every hype), but in comparison to crypto, at the very least gen AI provides tractable benefits in a wide array of usecases. What has nft/web3/x-coins ever given us apart from speculative investments (aka gambling)?
- thesuperbigfrog 3y ago
  >> What has nft/web3/x-coins ever given us apart from speculative investments (aka gambling)? Cryptocurrencies modernized the malware industry. Before cryptocurrencies, ransomware used to require victims to pay using gift cards or other ridiculous means. After cryptocurrencies, victims could now pay their "captors" directly over the Internet in cryptocurrency. Before cryptocurrencies, malware entities and actors acted largely independently unless they were part of a state-sponsored group or larger organized crime "business". After cryptocurrencies, "ransomware as a service" emerged where different groups specialize in particular components and services of ransomware: tools to establish footholds, tools to take over and encrypt victim systems, services to demand and take payments, and services to wash and launder payments. Payments between these groups are made in cryptocurrency.
- swalsh 3y ago
  The last bull saw the rise of all kinds of new technologies which will enable the use cases people were talking about (Avalanche is my favorite example, but theres some new stuff like SUI coming). The thing web3 did absolutely right is create a true interchangeable format (ie tokens) the downside is this technology has been developing sooooooo slowly. Whenever people start to get hope the technology might be ready for use, the gambling use cases rise to the top and dominate. I think crypto is also dominated by a certain hardline techy. Or at least it's critics are. Crypto should be working side by side with traditional finance. Instead they view it as their enemy. FTX US is an example of how regulation can be good (the funds on FTX-US were more or less unmolested) yet most crypto people seem to argue against any regulation at all. It will be their demise.

pacjam 3y ago

Thanks for your feedback @skrebbel. We called it MemGPT - in reference to teaching LLMs how to manage their own memory hierarchy. The idea and title evolved when we realized we were teaching the LLM to read/write, and this enabled perpetual chatbots, doc QA, etc. Various capabilities by a few simple abstractions, inspired by the original unix paper's 4 key abstractions - read/write/open/close. Further, in the title we say "Towards", indicating a qualified step in that direction.

__void 3y ago
still this is not an "operating system" nor does it in any way related to them; please use the right words when you have to communicate, otherwise you just sound like scammers who want to sell pots with holes by passing them off as colanders
skrebbel 3y ago
Perpetual chatbots are a super cool achievement but I fail to see how that resembles an OS, a thing that handles files and lets you run programs, in any way. If MemGPT is as spectacular as you suggest then I think an honest, down to earth title would have gotten you a lot more attention. It seems to me that limited context windows are one of the key roadblocks to wider real-world LLM applicability, beyond demos that do well on Twitter I mean.
- selestify 3y ago
  > If MemGPT is as spectacular as you suggest then I think an honest, down to earth title would have gotten you a lot more attention. The fact that the current title got onto the HN front page suggests otherwise when it comes to incentives for picking submission titles.
abdellah123 3y ago
> we were teaching the LLM to read/write Very interesting.
ludwik 3y ago
I think a title like "MemGPT: Operating System inspired memory management for LLMs" would be okay. The current title makes it seem as if you are building an OS based on the LLM technology, which is quite misleading.

deleted 3y ago

[deleted]

lgessler 3y ago

IMO, as an NLP researcher, it's a product of the "fast science" culture of AI where there are more papers than anyone could possibly read even in a given subfield at a particular conference, and an "old" paper is anything that was published over two years ago. (Who knew Bollywood movies and AI publications would have so much in common?) Maybe there are good reasons why publications should move so much more quickly in AI as opposed to, say, scholastic philosophy, but it's undeniable that it has become a struggle to get others to even read an abstract of your work, and as such we should not be surprised when sensationalization verges on becoming a requirement for your work to get engagement. So to answer your question--perhaps cynically--“Tiered memory layers to provide extended AI context windows” is not the best title you could have for an AI paper because it has all the rizz of a shipping manifest. If you want to maximize citations, you need to market more. I also considered whether to blame the field's very young reviewer population for not having the proper disdain for sensationalization that I'd expect from an older researcher who would surely have more restraint than to speak so much more of the sizzle than of the steak, but then I remembered that the paper which introduced the Transformer model was titled "Attention Is All You Need".

EncomLab 3y ago
I love this response because it's at least honest about the reality of "scientific publishing" today - who cares about the accuracy, verifiability or ability to replicate - you just need to maximize your citations via hype!!"
althea_tx 3y ago
This isn’t a slight against OP’s paper, but it’s worth noting that arXiv is not peer reviewed.

quickthrower2 3y ago

Attention is all you need. People’s attention that is :-)

amelius 3y ago

This is why we need to get AI (or DL) out of the realm of CS, quickly. Similar to how psychology is a different field than physics even if the former is built on the latter.

fritzo 3y ago

In defense of the authors, I find it helpful to explore analogies between the nascent paradigm of LLM programming and other much older conceptual frames. This paper isn't just tiered memory, but also the analogy between an LLM and a CPU, interrupts, event processing etc.

ultra_nick 3y ago

Are you new to software? The bugs in programs aren't actual bugs either, windows aren't actual windows, and many would argue that computer science is still more of an art than a science.