3 ms·
I'm excited to see attention layers animated like this. I feel like I'm this close to grasping them.
by api 3y ago
I'm excited to see attention layers animated like this. I feel like I'm this close to grasping them.
- Animated-AI 3y agoHi! I'm the creator of the site. Good news: I'm currently working on animations and an explainer video on transformers and self-attention. The best way to be notified is probably to subscribe to my YouTube channel and hit the bell icon for notifications.
- imjonse 3y agoYou mean you would be excited to see attention animations? The page presents convolutions not attention.
- Greenpants 3y agoI still haven't found "that one visualisation" that makes the attention concept in Transformers as easily understood as these CNNs. If someone here on HN has a link to a page that has helped them get to the Eureka-point of fully grasping attention layers, feel free to share!
- juliangoldsmith 3y agoI found this video helpful for understanding transformers in general, but it covers attention too: https://www.youtube.com/watch?v=kWLed8o5M2Y https://www.youtube.com/watch?v=kWLed8o5M2Y The short version (as I understand it) is that you use a neural network to weight pairs of inputs by their importance to each other. That lets you get rid of unimportant information while keeping what actually is important.