I've always considered the human ability to abstract things away - the capacity to say "this is like that" - as a kind of data compression mechanism. Instead of

I've always considered the human ability to abstract things away - the capacity to say "this is like that" - as a kind of data compression mechanism. Instead of memorizing everything, we remember the first few things of that kind that we encountered, and then the differences between them and everything else that's sort of like them. The core memories being the root, and the differences being steps along the path of a trie -- both space and time efficient in storage and retrieval. I think the addition of medium-term memory to an LLM system would mean an individualization / giving that system instance the means to become unique from its common starting point. We can already consider the context window as a short-term memory of sorts, but it's lobotomized every time. Having a persistent medium-term memory window (eg, via collecting summaries / tries of abstraction "keyframes" + diffs) would begin to pose some serious questions. If a mechanism was further developed to persist these medium-term data summaries into "long term memory" (adjusting weights and parameters), we would have continuous learning. It's pretty wild stuff.