5 ms·

So, this is cool and a per-user memory is obviously relevant for effective LLM use. And major props for the temporal focus. However, keeping a tight, constrain

by lukev 1y ago

So, this is cool and a per-user memory is obviously relevant for effective LLM use. And major props for the temporal focus. However, keeping a tight, constrained context turns out to actually be pretty important for correct LLM results (https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-ho...). Do you have a take on how we reconcile the tension between these objectives? How to make sure the model has access to relevant info, while explicitly excluding irrelevant or confounding factors from the context?

Manoj58 1y ago
hey, thanks for the article reference. i read it. that's the exact problem we've been solving! Context bloat vs. memory depth is the core challenge. our approach tackles this by being selective, not comprehensive. We don't dump everything into context - instead, we: - use graph structure to identify truly relevant facts (not just keyword matches) - leverage temporal tracking to prioritize current information and filter out outdated beliefs - structure memories as discrete statements that can be included/excluded individually the big advantage? Instead of retrieving entire conversations or documents, we can pull just the specific facts and relevant episodes needed for a given query. it's like having a good assistant who knows when to remind you about something relevant without overwhelming you with every tangentially related memory. the graph structure also gives users more transparency - they can see exactly which memories are influencing responses and why, rather than a black-box retrieval system. ps: one of the authors of CORE
- stingraycharles 1y ago
  One of the challenges I was facing with other memory MCP servers is to get the LLM clients to actually use it to recall relevant information when they need it. Implementing to MCP tools is one thing, getting LLM clients to invoke them at the right time is another. How do you solve that problem?
  - harshithmul 1y ago
    We faced the same challenge while building SOL (https://github.com/RedPlanetHQ/sol https://github.com/RedPlanetHQ/sol) — a personal assistant that relies heavily on memory for context and continuity. Getting LLMs to invoke memory tools at the right time is definitely trickier than just wiring up MCP correctly. We're still refining it, but we've made good progress by explicitly guiding the assistant within the system prompt on when and how to use memory. You can see an example of how we structure this in SOL here: Prompt instructions for memory usage (https://github.com/RedPlanetHQ/sol/blob/964ed23c885910e040bdca56b572f942275c7868/apps/server/src/triggers/agents/chat/prompt.ts#L120C1-L161C10 https://github.com/RedPlanetHQ/sol/blob/964ed23c885910e040bd...) Using something on similar lines as rules in claude/cursor etc has been working better. It’s not perfect yet, but this combination of prompt engineering and structured tool exposure has been moving us in the right direction. ps - one of the authors of CORE
- IXCoach 1y ago
  This is helpful. Love to see a demo of how tight you got the context window injection against a query. Thats where theres always 70% bloat in my previous systems. I solved this by building holonically, same structure as you have it seems roughly, so I actually, through a ui can grab a holon and inject it into context including its children ( holon ~ nested heirarchy ), And I usually use semantic search so Ill add that in as well. I have not added agentic memory flows yet, like when a model asks itself if it has what it needs and allows itself to look deeper.. have you? I have agentic flows with other things, about 15 cascading steps between user and ai response, but have not done so with memory yet. Im appreciating what you put together here. Jonathan - Next AI Labs and IX Coach

by lukev 1y ago

Manoj58 1y ago

hey, thanks for the article reference. i read it. that's the exact problem we've been solving! Context bloat vs. memory depth is the core challenge. our approach tackles this by being selective, not comprehensive. We don't dump everything into context - instead, we: - use graph structure to identify truly relevant facts (not just keyword matches) - leverage temporal tracking to prioritize current information and filter out outdated beliefs - structure memories as discrete statements that can be included/excluded individually the big advantage? Instead of retrieving entire conversations or documents, we can pull just the specific facts and relevant episodes needed for a given query. it's like having a good assistant who knows when to remind you about something relevant without overwhelming you with every tangentially related memory. the graph structure also gives users more transparency - they can see exactly which memories are influencing responses and why, rather than a black-box retrieval system. ps: one of the authors of CORE

stingraycharles 1y ago
One of the challenges I was facing with other memory MCP servers is to get the LLM clients to actually use it to recall relevant information when they need it. Implementing to MCP tools is one thing, getting LLM clients to invoke them at the right time is another. How do you solve that problem?
- harshithmul 1y ago
  We faced the same challenge while building SOL (https://github.com/RedPlanetHQ/sol https://github.com/RedPlanetHQ/sol) — a personal assistant that relies heavily on memory for context and continuity. Getting LLMs to invoke memory tools at the right time is definitely trickier than just wiring up MCP correctly. We're still refining it, but we've made good progress by explicitly guiding the assistant within the system prompt on when and how to use memory. You can see an example of how we structure this in SOL here: Prompt instructions for memory usage (https://github.com/RedPlanetHQ/sol/blob/964ed23c885910e040bdca56b572f942275c7868/apps/server/src/triggers/agents/chat/prompt.ts#L120C1-L161C10 https://github.com/RedPlanetHQ/sol/blob/964ed23c885910e040bd...) Using something on similar lines as rules in claude/cursor etc has been working better. It’s not perfect yet, but this combination of prompt engineering and structured tool exposure has been moving us in the right direction. ps - one of the authors of CORE
IXCoach 1y ago
This is helpful. Love to see a demo of how tight you got the context window injection against a query. Thats where theres always 70% bloat in my previous systems. I solved this by building holonically, same structure as you have it seems roughly, so I actually, through a ui can grab a holon and inject it into context including its children ( holon ~ nested heirarchy ), And I usually use semantic search so Ill add that in as well. I have not added agentic memory flows yet, like when a model asks itself if it has what it needs and allows itself to look deeper.. have you? I have agentic flows with other things, about 15 cascading steps between user and ai response, but have not done so with memory yet. Im appreciating what you put together here. Jonathan - Next AI Labs and IX Coach