3 ms·
If you'd rather prefer something readable and explicit, instead of empty handwaving and uml-like diagrams, read "The Transformer model in equations" [0] by John
by enriquto 3y ago
If you'd rather prefer something readable and explicit, instead of empty handwaving and uml-like diagrams, read "The Transformer model in equations" [0] by John Thickstun [1].
[0] https://johnthickstun.com/docs/transformers.pdf https://johnthickstun.com/docs/transformers.pdf
[1] https://johnthickstun.com/docs/ https://johnthickstun.com/docs/
- robotresearcher 3y agoSee also the original paper: https://arxiv.org/abs/1706.03762 https://arxiv.org/abs/1706.03762
- epistasis 3y agoThe original paper is quite deceptive and hard to understand, IMHO. It relies on jumping between several different figures and mapping between shapes, in addition to guessing at what the unlabeled inputs are. Just a few more labels, making the implicit explicit, would make it far more intelligible. Plus, last time I went through it Im pretty sure that there's either a swap on the order of the three inputs between different figures, or that it's incorrectly diagrammed.