Eh, I'd say the attention paper lived up to the hype of its title. In 2017 LLM architectures were complex beasts that fiddled with many different different str

Eh, I'd say the attention paper lived up to the hype of its title. In 2017 LLM architectures were complex beasts that fiddled with many different different structures of layers. These days they're all just giant stacks of attention layers.