3 ms·
In case you find Attention (and specially transformers) interesting I have some saved links introducing the topic: http://www.peterbloem.nl/blog/transformers h
by kuu 7y ago
In case you find Attention (and specially transformers) interesting I have some saved links introducing the topic:
http://www.peterbloem.nl/blog/transformers http://www.peterbloem.nl/blog/transformers
https://nostalgebraist.tumblr.com/post/185326092369/the-transformer-explained https://nostalgebraist.tumblr.com/post/185326092369/the-tran...
https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf https://papers.nips.cc/paper/7181-attention-is-all-you-need....
http://jalammar.github.io/illustrated-transformer/ http://jalammar.github.io/illustrated-transformer/
https://arxiv.org/pdf/1807.03819.pdf https://arxiv.org/pdf/1807.03819.pdf
- BiasRegularizer 7y agoWhile Transformer's Self Attention(SA) is great, there are many applications where SA doesn't apply. For a more comprehensive overview of attention mechanisms, I often find myself coming back to Lilian Weng's post: https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html https://lilianweng.github.io/lil-log/2018/06/24/attention-at...