For current transformer architectures, the weights don't change but the context window is where "it" "lives", so the "only" thing missing for continuous learnin

For current transformer architectures, the weights don't change but the context window is where "it" "lives", so the "only" thing missing for continuous learning is either infinite context or another persistence mechanism.. something akin to "dreaming" has been proposed f.e. .. Pretty likely somethign will make it into roll-out within a year, probably crossing more thresholds into AGI territory.. Of course that will trigger more goal-post moving, but I don't see how even sceptics would come to the conclusion in 2029 AGI hasn't been reached.