> The paper shows that the model can compete with (but not decimate) vanilla Transformers on the scale of ~1B parameters in long-sequence prediction tasks. They

> The paper shows that the model can compete with (but not decimate) vanilla Transformers on the scale of ~1B parameters in long-sequence prediction tasks. They did not "unleash" anything, and scalability to very large parameter counts and datasets still has not been tested. One thing to keep in mind in particular is that the vast majority of transformer alternatives/improvements/optimizations that initially showed promise ultimately ended up scaling less well than the baseline transformer architecture. The transformer has been weirdly hard to improve/dethrone.