3 ms·
This is great. Just yesterday I was wondering how exactly transformers/attention and LLMs work. I'd worked through how back-propagation works in a deep RNN a lo
by karmakaze 2y ago
This is great. Just yesterday I was wondering how exactly transformers/attention and LLMs work. I'd worked through how back-propagation works in a deep RNN a long while ago and thought it would be interesting to see the rest.
- aDyslecticCrow 2y agoFor some intuition, 3b1b has some videos that explain it nicely. But he doesn't go that far into the nitty gritty.