3 ms·
On a much simpler level, llm frameworks could re-summarize their context to keep relevant, use-case-specific facts, cleanup and also organize long and short ter
by dr_hooo 7mo ago
On a much simpler level, llm frameworks could re-summarize their context to keep relevant, use-case-specific facts, cleanup and also organize long and short term memory on some local storage, etc. So kind of like sleep. I think these examples are low hanging fruit to improve the perceived intelligence of LLM systems (so probably they're already used somewhere).
- sigmoid10 7mo agoWe already have that for a while. It works to some degree, but context tokens simply don't offer the level of compression that model weights do. At least with current approaches that keep the context human-readable.