3 ms·
There's this tendency among AI researchers to write clickbait titles (like Attention is All You Need), partly due to extreme competition in the publishing/confe
by grepLeigh 3y ago
There's this tendency among AI researchers to write clickbait titles (like Attention is All You Need), partly due to extreme competition in the publishing/conference environment. If the odds of your paper being accepted is 20%-ish (NEURIPS), I can see why teams opt for attention-grabbing titles.
"Attention is at least X% more performant on selected benchmarks than a selected sample of recurrent networks, with ablation, thus proving attention is all you might need until a non-exponential architecture is developed" doesn't have the same catchy ring to it.
- PoignardAzur 3y agoEh, I'd say the attention paper lived up to the hype of its title. In 2017 LLM architectures were complex beasts that fiddled with many different different structures of layers. These days they're all just giant stacks of attention layers.