I've been learning how transformers work, and also reading about RNNs. It seems that transformers have succeeded because they can be parallelized, and solve ma

I've been learning how transformers work, and also reading about RNNs. It seems that transformers have succeeded because they can be parallelized, and solve many of the problems that RNNs have. This has lead to the advancements we've seen recently with GTP. Big companies are able to throw their data centers at the problem and train impressive and massive models. Part of me hopes that a non-parallelizable AI architecture might be discovered which performs even better. Perhaps the problems with RNNs could be solved in a way that doesn't parallelize? We would be so fortunate if a desktop computer could run an AI that's half as good as Microsoft's or Amazon's best AI. I would love to see the advantage of the data center removed. Philosophically, this does make some sense. The wisdom behind sayings such as "adding more people makes the project even later" exist because the greatest intellects we're aware of (ourselves) do not parallelize well.