Have you looked at Titan and MIRAS where they use online/updating associative memory that happens to be read out via next-token prediction? https://research.go

Have you looked at Titan and MIRAS where they use online/updating associative memory that happens to be read out via next-token prediction? https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/ https://research.google/blog/titans-miras-helping-ai-have-lo... https://arxiv.org/abs/2501.00663 https://arxiv.org/abs/2501.00663 https://arxiv.org/pdf/2504.13173 https://arxiv.org/pdf/2504.13173 Much research is going into these directions, but I'm more interested in mind-wandering tangents, involving both attentional control and additional mechanisms (memory retrieval, self-referential processing).