3 ms·

It's becoming increasingly clear that memory and context are the bottlenecks in advancing usage of AI. I can't help but feel there needs to be a general, perhap

It's becoming increasingly clear that memory and context are the bottlenecks in advancing usage of AI. I can't help but feel there needs to be a general, perhaps even built into the model, solution for this - everyone seems to be building something on top that is roughly the same thing.

ramoz 1y ago
Karpathy had a similar interesting take the other day https://x.com/karpathy/status/1921368644069765486 https://x.com/karpathy/status/1921368644069765486
- FisherKK 1y ago
  Skill Library!
- hnuser123456 1y ago
  I'm starting up experiments with having agents write system prompts for sub-agents. Specifically, have the LLM build, test, and validate a small, simple tool, and once validated, add it to its own system prompt listing available tools. Anyone else experimenting with letting LLMs generate their own or sub-agent system prompts?
hnuser123456 1y ago
Fine tuning should be combined with inference in some way. However this requires keeping the model loaded at high enough precision for backprop to work. Instead of hundreds of thousands of us downloading the latest and greatest model that won't fundamentally update one bit until we're graced with the next one, I would think we should all be able to fine-tune the weights so that it can naturally memorize new additional info and preferences without using up context length.
pacjam 1y ago
check out Letta - the OSS codebase (https://github.com/letta-ai/letta https://github.com/letta-ai/letta) is basically focused on solving the memory/context problem in a generalized way (via "agentic context management"). if you're more interested in papers, we also worked on MemGPT and more recently sleep-time compute (https://arxiv.org/abs/2504.13171 https://arxiv.org/abs/2504.13171)
- edunteman 1y ago
  Love your sleep time stuff! It's an inspiration for Muscle Mem
tom_m 1y ago
Absolutely. The "intelligence" isn't complete without a memory. In fact there's a whole lot more to it than that. The LLM is one component, a logic factory, but there's so much more than the LLM and the memory. In fact, systems should be LLM agnostic or use different models for different needs. I don't believe building something into the model will ever be the solution though. It is interesting what Google is trying to do with model caching but at the end of the day I believe the strength of agents here will rely heavily upon modularity.

ramoz 1y ago

Karpathy had a similar interesting take the other day https://x.com/karpathy/status/1921368644069765486 https://x.com/karpathy/status/1921368644069765486

FisherKK 1y ago
Skill Library!
hnuser123456 1y ago
I'm starting up experiments with having agents write system prompts for sub-agents. Specifically, have the LLM build, test, and validate a small, simple tool, and once validated, add it to its own system prompt listing available tools. Anyone else experimenting with letting LLMs generate their own or sub-agent system prompts?

hnuser123456 1y ago

Fine tuning should be combined with inference in some way. However this requires keeping the model loaded at high enough precision for backprop to work. Instead of hundreds of thousands of us downloading the latest and greatest model that won't fundamentally update one bit until we're graced with the next one, I would think we should all be able to fine-tune the weights so that it can naturally memorize new additional info and preferences without using up context length.

pacjam 1y ago

check out Letta - the OSS codebase (https://github.com/letta-ai/letta https://github.com/letta-ai/letta) is basically focused on solving the memory/context problem in a generalized way (via "agentic context management"). if you're more interested in papers, we also worked on MemGPT and more recently sleep-time compute (https://arxiv.org/abs/2504.13171 https://arxiv.org/abs/2504.13171)

edunteman 1y ago
Love your sleep time stuff! It's an inspiration for Muscle Mem

tom_m 1y ago

Absolutely. The "intelligence" isn't complete without a memory. In fact there's a whole lot more to it than that. The LLM is one component, a logic factory, but there's so much more than the LLM and the memory. In fact, systems should be LLM agnostic or use different models for different needs. I don't believe building something into the model will ever be the solution though. It is interesting what Google is trying to do with model caching but at the end of the day I believe the strength of agents here will rely heavily upon modularity.