4 ms·
I scrolled straight to section 2.1 "Definition and Characteristics of LLM agents" to find out which of the many definitions of "agent" they are using here. The
by simonw 2y ago
I scrolled straight to section 2.1 "Definition and Characteristics of LLM agents" to find out which of the many definitions of "agent" they are using here.
They went for LLM + short-term and long-term memory + planning + tool using + action execution.
Presumably "planning" here is covered by any LLM that can do "think step by step" reasonably well?
It wasn't clear to me what the difference between "tool using" and "action execution" was.
I haven't seen a definition that specifically encompasses both short- and long-term memory before. They say:
> This dual memory system allows agents to maintain conversation continuity while building knowledge over time.
So presumably, this is the standard LLM chat conversation log plus a tool that can decide to stash extra information in a permanent store - similar to how ChatGPT's memory feature worked up until about four weeks ago.
- swyx 2y ago> Presumably "planning" here is covered by any LLM that can do "think step by step" reasonably well? mild disagree. 1) externalizing the plan and letting the user audit/edit the plan while its working is "tool use", yes, but a very specialcase kind of tool use that, for example, operator and deep research use Temporal for. ofc we also saw this with Devin/Manus and i kinda think they're better 2) there is a form of primitive tree search that people are doing where they can spam out several different paths and run it a few steps ahead to gain information about optimal planning. You will see this with morph's launch at AIE. 3) plan meta reflection and reuse - again a form of tool use, but the devin and allhands folks have worked on this a lot more than most. my criticism of many agent definitions is that they generally do not take memory, planning, and auth seriously enough, and i think those 3 areas are my current bets for "alpha" in 2025. > I haven't seen a definition that specifically encompasses both short- and long-term memory before. here - https://docs.mem0.ai/core-concepts/memory-types#short-term-memory https://docs.mem0.ai/core-concepts/memory-types#short-term-m... - https://x.com/swyx/status/1915128966203236571 https://x.com/swyx/status/1915128966203236571
- Tokumei-no-hito 2y agowhat is morphs launch at AIE?
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- nkko 2y agoMorph.so is an ai sandbox thing that has branching from state, prob doing a talk in SF on this at AI Engineer conf
- Tokumei-no-hito 2y agothanks
- tough 2y ago> It wasn't clear to me what the difference between "tool using" and "action execution" was. doing a lot of inference here, but could be a separation between -read- tool kinda actions, and -write/execute- (like running code/sending an email, etc) a bit weird from a coding perspective but idk
- sebastiennight 2y ago> similar to how ChatGPT's memory feature worked up until about four weeks ago What happened four weeks ago?
- swyx 2y agothey updated it opaquely - we no longer know anything about how it works https://www.latent.space/p/clippy-v-anton https://www.latent.space/p/clippy-v-anton
- wunderwuzzi23 2y agoSee my response to Simon above on some insights on how it works - I'll write up in detail in a blog post also when I get to it.
- simonw 2y agoThey added a feature where it can reference content from any conversation you've had with it in the past. It's not clear how this works - probably some kind of vector search? Prior to that change "memory" was a tool call: https://simonwillison.net/2024/Oct/15/chatgpt-horoscopes/#how-memory-works https://simonwillison.net/2024/Oct/15/chatgpt-horoscopes/#ho...
- wunderwuzzi23 2y agoHere is what I have been able to reverse engineer for o3... At high level it maintains about ~40 conversations in system prompt under a section called "recent conversation content". It only contains what the user typed, not assistant responses (probably due to prompt injection) - there a few corner cases though. :) There are other sections in the system prompt now that contain aggregated info, so recent conversations turn into user insights over time I believe. It can't actually "search" history afaik - that part I'm still wondering, as it was my first thought on how it might work... I also found a way to exfiltrate the recent content - so hopefully that will be fixed soon... Overall, this feature creates a lot of confusion and response quality declines at times too - and anything someone posts now online (like weird behavior or hallucinations,...) is likely influenced by their past conversations! So it will make it more difficult to understand what's really happening. I think it would be cool if "projects" would be entirely isolated with their own memories and history etc. or have different "profiles"
- jsemrau 2y agoAccurate memory access across memory types is still not really solved. That is the issue. Most agent frameworks from the main model providers are still quite spotty. Image of a table outlining where the major frameworks are: https://substackcdn.com/image/fetch/w_1272,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F76107516-eba3-4a17-ba85-5ccd76180d29_960x540.png https://substackcdn.com/image/fetch/w_1272,c_limit,f_webp,q_... Here is also an article I wrote last year on the different types of memory: https://open.substack.com/pub/jdsemrau/p/memory-and-knowledge-for-agents?r=7fx0t&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true https://open.substack.com/pub/jdsemrau/p/memory-and-knowledg...