4 ms·

Letta: Letta is a framework for creating LLM services with memory

121 points by sebg 2y ago

jxjnskkzxxhx 2y ago
What's an "LLM with memory"? Is it an agent that knows how to insert into, and query, a database?
- blooalien 2y ago
  https://docs.letta.com/letta-platform https://docs.letta.com/letta-platform
- pacjam 2y ago
  In Letta that's referring to an LLM agent that has persistent memory (eg keep running the agent far beyond its context window limit, but maintain some persistent state to avoid derailment / enable indefinite use / learning over long horizons). If you implement this persistent memory via tool use, then it's related to inserting/querying databases, because some agent has to read/write to a data store (potentially backed by a database) to maintain memories. -charles from letta
rcarmo 2y ago
I find it somewhat amusing that Lethe was the river of forgetfulness, and that the new name for the project is so close to that.
- teleforce 2y ago
  Thanks for the info, never heard of Lethe before [1]. Fun facts in Arabic, "insan" meaning human being or mankind. This word takes its root from the word "nisyan", which literally means “forgetfulness”. [1] Lethe: https://en.wikipedia.org/wiki/Lethe https://en.wikipedia.org/wiki/Lethe [2] Arabic for forgetfulness: https://arabic.fi/words/3766 https://arabic.fi/words/3766
  - andsoitis 2y ago
    > This word takes its root from the word "nisyan", which literally means “forgetfulness”. There are two linguistic theories: the one you mention, and the other which is from “uns” (أنس) which means “companionship” or “affection.” While both explanations have historical significance, the stronger linguistic evidence points to the latter explanation (companionship or sociability) by Classical Arabic linguists. The former (forget) is a more recent interpretation linked to philosophical and theological discussions rather than libguistic roots.
  - Xmd5a 2y ago
    Man, mind, to remain, to remind: they all originate from the same intertwined proto-indo-european roots "men" (thinking) and "mon/man" (human). Edit: just found this etymological trail for insan: >From Proto-Semitic *ʔināš- (“man, person”). Has the form of a verbal noun , as if from أَنِسَ (ʔanisa, “to sense (visually or auditorily)”).
chrisvalleybay 2y ago
I didn't find any thing on Letta and MCP. Any thoughts on integrating MCP into the platform?
- anon373839 2y ago
  Totally serious question: is MCP basically an in-joke right now? I mean, I know what it is. I just don’t know why I’m seeing it everywhere in the last 3 days?
  - imtringued 2y ago
    Maybe this will help you figure it out: https://github.com/modelcontextprotocol/servers https://github.com/modelcontextprotocol/servers
    - yard2010 2y ago
      Thank you for the mind-blowing 2 hr rabbit hole. I'm going to use this in my personal helper bot.
  - chrisvalleybay 2y ago
    Probably people are excited about the prospect of having this problem be solved so that we can focus more on getting the agents to work rather than all be reimplementing the same data fetchers/parsers etc.
    - anon373839 2y ago
      Sure, but I mean why so suddenly? This has been out a while...
      - _neil 2y ago
        I think it’s just reached critical mass (and has been picked up by the hype peddlers). It’s been gaining momentum for months.
  - 2y ago
redwood 2y ago
This was the berkeley team that previously released MemGPT
- pacjam 2y ago
  indeed - hello there
dangbeetle 2y ago
My father is suffering from increasingly bad short term memory. This has been a slow process that has taken many decades. My mom was worried about his memory 20 years ago. Now he lives alone and has nobody to help remind him of things. He is highly functional, able to multiply two digit numbers in his head and run heavy equipment on his farm. But he'll ask you the same question, five minutes after he asked it the last time. He'll forget what neighbor just visited him the day before, or whether he called the gas company to refill his propane barrel. I believe that a voice interactive LLM with memory could be extremely helpful to him and to other elderly people. He could use it as a daily log, telling it things that have occurred, and then interrogate it later. In truth, who among us could not benefit from such a system, even in our younger years? This framework looks interesting to use for such a project. Are there any other people working on similar things?
- pacjam 2y ago
  > "voice interactive LLM with memory" Our team has been working really hard on a low-latency voice integration that enables (open source) "advanced voice mode", but with long-term memory! If you're interested definitely hop in our Discord to keep track of dev progress (you can also monitor our docs). It's all OSS of course, so you can use it with any model you want and run it all locally (your agent data stays on device). Once available you can definitely use Letta for this usecase - basically provision a stateful Letta agent for your dad, with instructions / prompting that your dad has memory loss issues, and the agent should be proactive in recording memories (and potentially jogging your dad's memory too). All the memory/state will be stored in postgres, so you can expose it to other tools as well, eg could even collate the memories into a nextjs/mobile app where you dad can browse them. -charles from letta

121 points by sebg 2y ago

jxjnskkzxxhx 2y ago

What's an "LLM with memory"? Is it an agent that knows how to insert into, and query, a database?

blooalien 2y ago
https://docs.letta.com/letta-platform https://docs.letta.com/letta-platform
pacjam 2y ago
In Letta that's referring to an LLM agent that has persistent memory (eg keep running the agent far beyond its context window limit, but maintain some persistent state to avoid derailment / enable indefinite use / learning over long horizons). If you implement this persistent memory via tool use, then it's related to inserting/querying databases, because some agent has to read/write to a data store (potentially backed by a database) to maintain memories. -charles from letta

rcarmo 2y ago

I find it somewhat amusing that Lethe was the river of forgetfulness, and that the new name for the project is so close to that.

teleforce 2y ago
Thanks for the info, never heard of Lethe before [1]. Fun facts in Arabic, "insan" meaning human being or mankind. This word takes its root from the word "nisyan", which literally means “forgetfulness”. [1] Lethe: https://en.wikipedia.org/wiki/Lethe https://en.wikipedia.org/wiki/Lethe [2] Arabic for forgetfulness: https://arabic.fi/words/3766 https://arabic.fi/words/3766
- andsoitis 2y ago
  > This word takes its root from the word "nisyan", which literally means “forgetfulness”. There are two linguistic theories: the one you mention, and the other which is from “uns” (أنس) which means “companionship” or “affection.” While both explanations have historical significance, the stronger linguistic evidence points to the latter explanation (companionship or sociability) by Classical Arabic linguists. The former (forget) is a more recent interpretation linked to philosophical and theological discussions rather than libguistic roots.
- Xmd5a 2y ago
  Man, mind, to remain, to remind: they all originate from the same intertwined proto-indo-european roots "men" (thinking) and "mon/man" (human). Edit: just found this etymological trail for insan: >From Proto-Semitic *ʔināš- (“man, person”). Has the form of a verbal noun , as if from أَنِسَ (ʔanisa, “to sense (visually or auditorily)”).

chrisvalleybay 2y ago

I didn't find any thing on Letta and MCP. Any thoughts on integrating MCP into the platform?

anon373839 2y ago
Totally serious question: is MCP basically an in-joke right now? I mean, I know what it is. I just don’t know why I’m seeing it everywhere in the last 3 days?
- imtringued 2y ago
  Maybe this will help you figure it out: https://github.com/modelcontextprotocol/servers https://github.com/modelcontextprotocol/servers
  - yard2010 2y ago
    Thank you for the mind-blowing 2 hr rabbit hole. I'm going to use this in my personal helper bot.
- chrisvalleybay 2y ago
  Probably people are excited about the prospect of having this problem be solved so that we can focus more on getting the agents to work rather than all be reimplementing the same data fetchers/parsers etc.
  - anon373839 2y ago
    Sure, but I mean why so suddenly? This has been out a while...
    - _neil 2y ago
      I think it’s just reached critical mass (and has been picked up by the hype peddlers). It’s been gaining momentum for months.
- 2y ago

redwood 2y ago

This was the berkeley team that previously released MemGPT

pacjam 2y ago
indeed - hello there

dangbeetle 2y ago

My father is suffering from increasingly bad short term memory. This has been a slow process that has taken many decades. My mom was worried about his memory 20 years ago. Now he lives alone and has nobody to help remind him of things. He is highly functional, able to multiply two digit numbers in his head and run heavy equipment on his farm. But he'll ask you the same question, five minutes after he asked it the last time. He'll forget what neighbor just visited him the day before, or whether he called the gas company to refill his propane barrel. I believe that a voice interactive LLM with memory could be extremely helpful to him and to other elderly people. He could use it as a daily log, telling it things that have occurred, and then interrogate it later. In truth, who among us could not benefit from such a system, even in our younger years? This framework looks interesting to use for such a project. Are there any other people working on similar things?

pacjam 2y ago
> "voice interactive LLM with memory" Our team has been working really hard on a low-latency voice integration that enables (open source) "advanced voice mode", but with long-term memory! If you're interested definitely hop in our Discord to keep track of dev progress (you can also monitor our docs). It's all OSS of course, so you can use it with any model you want and run it all locally (your agent data stays on device). Once available you can definitely use Letta for this usecase - basically provision a stateful Letta agent for your dad, with instructions / prompting that your dad has memory loss issues, and the agent should be proactive in recording memories (and potentially jogging your dad's memory too). All the memory/state will be stored in postgres, so you can expose it to other tools as well, eg could even collate the memories into a nextjs/mobile app where you dad can browse them. -charles from letta