I wouldn't be surprised if AI research moves towards training models to use bolt-on peripherals like scratchpad memory. Transformers showed us an interface for

I wouldn't be surprised if AI research moves towards training models to use bolt-on peripherals like scratchpad memory. Transformers showed us an interface for addressing, so I wouldn't be surprised if someone figures out a way to use a similar addressing scheme and made it read/write.