Interesting, that it goes against the grain. Since the seminal paper 'Attention is all you need', we went from RNN type neural network to pure attention based

Interesting, that it goes against the grain. Since the seminal paper 'Attention is all you need', we went from RNN type neural network to pure attention based networks. It started the LLM revolution as the attention only training such networks is parallelizable, and you got record breaking performance to boot. Now we learn, that going back to the old RNN paradigm is actually better. It even advertises itself as totally 'attention-free'!