4 ms·
Try this one: https://www.youtube.com/watch?v=yGTUuEx3GkA https://www.youtube.com/watch?v=yGTUuEx3GkA This series of video explains how the core mechanism wor
by scotty79 4y ago
Try this one:
https://www.youtube.com/watch?v=yGTUuEx3GkA https://www.youtube.com/watch?v=yGTUuEx3GkA
This series of video explains how the core mechanism works. There are few details omitted like how to get good initial token embedding or how exactly positional encoding works.
High level overview is that main insight of transformers is just figuring out how to partition huge basic neural network and hardcode some intuitively beneficial operations into the structure of the network iteself and draw some connections between (not very) distant layers so that gradient doesn't get eaten up too soon during backpropagation.
It all makes the whole thing parallelizable so you can train it on the huge amount of data despite it having enough neurons altogether to infer pretty complex associations.