4 ms·
Attention is, after all, not what you need.
by _hl_ 3y ago
Attention is, after all, not what you need.
- jeremycochoy 3y agoWell, RWKV is using some linear (non-quadratic) form of attention so... strictly speaking... you still need a bit of attention :D