To paraphrase The Graduate, "Recurrence, my child—recurrence!" I'd take the bet that in the medium term (whatever that means today), the Transformer architectu

To paraphrase The Graduate, "Recurrence, my child—recurrence!" I'd take the bet that in the medium term (whatever that means today), the Transformer architecture as implemented today will be replaced by designs which revive recurrent, non-directed architectures, with a few specific properties: • cycles will be allowed • equilibrium (over time) will serve as state ("output") • computationally this is a couple orders more expensive than current approaches, both in training and in use • investigating emergent internal encoding and computation will similarly be orders more expensive to investigate, and require nearly as expensive tools as subjects This is The Way. NB non-coincidentally this looks much more like an animal brain.