Personally I think multi-year scale memory is possible with currently available research, if we just put it all together. What happens if we combine very long c

Personally I think multi-year scale memory is possible with currently available research, if we just put it all together. What happens if we combine very long context lengths, dedicated summarising LLMs, RAG, MemGPT, sparse MoE, and a perennial Constitutional AI overseer? I don't see any reason why these systems couldn't work together. It just hasn't really been that much time, particularly considering that large training runs can take months, and if you want the best performance you often need to train with your specific architecture in mind. As well as time, it's not like all of these advances are coming from the same group of people, they're from AI researchers spread across the globe. Get them all in a room with a CERN-level budget for compute resources and I think they could do it.