Approaches like RETRO tack vector search over a fixed database onto an attention based model - that's how you get long term memory. People are already working o

Approaches like RETRO tack vector search over a fixed database onto an attention based model - that's how you get long term memory. People are already working on it for the current crop of LLMs.