The concept of mostly static weights holding the bulk of base intuition/knowledge (foundation if you will ;)) seems like a good bet, since it's how the mammalia

The concept of mostly static weights holding the bulk of base intuition/knowledge (foundation if you will ;)) seems like a good bet, since it's how the mammalian brain works (with updates of those long term weights mostly happening while you sleep [1]). I very naively assume the "easy" path will be similar: a very different system that's bolted on/references the foundation models, to enable the realtime/novel reasoning (outside the fixed latent space) bit that isn't possible now. [1] https://animalcare.umich.edu/our-impact/our-impact-monitoring-sleeping-rats-may-solve-puzzle-behind-long-term-memory-processing/ https://animalcare.umich.edu/our-impact/our-impact-monitorin...