4 ms·
Makes me wonder if there might be any advantages to modulating Softmax attention in deep artificial neural nets, with some kind of mechanism for recomputing att
by cs702 8y ago
Makes me wonder if there might be any advantages to modulating Softmax attention in deep artificial neural nets, with some kind of mechanism for recomputing attention based on each previous attention.
- ctoth 8y agoI came here hoping to read this. I'm not clever enough to implement it, but my first thought was that someone would see a connection here.