4 ms·
I have the opposite: my most hated illustration. It's the standard diagram of how Transformer language model works (https://www.researchgate.net/figure/Transfo
by eterevsky 4y ago
I have the opposite: my most hated illustration.
It's the standard diagram of how Transformer language model works (https://www.researchgate.net/figure/Transformer-Language-Model-Structure-Vaswani-et-al-2017-3_fig3_357660800 https://www.researchgate.net/figure/Transformer-Language-Mod...). When I tried to figure out transformers, I saw it in every single paper, and it didn't help almost at all. I think I finally got a good understanding only when I looked at a few implementations.
- YetAnotherNick 4y agoAlso it is wrong. The paper has add and norm, but the official implementation and all other good implementation has pre norm architecture: https://twitter.com/francoisfleuret/status/1467135366503219205?lang=en https://twitter.com/francoisfleuret/status/14671353665032192...