The current best-in-class performance is from transformer models so of course research is focusing on this. You're creating a false dichotomy between "work on t

The current best-in-class performance is from transformer models so of course research is focusing on this. You're creating a false dichotomy between "work on transformers" and "work on something new and fundamentally better." All of the current work on optimizing transformer architectures is exploring and better understanding the space, and so it IS leading us to better architectures. This is the most fertile ground for new insights that will outperform the current approaches. New technologies aren't generally just created out of whole cloth by some genius, they're build up of layers of incremental improvements, each of which is modest in its own right but which when taken together are groundbreaking.