4 ms·
Well, RWKV is using some linear (non-quadratic) form of attention so... strictly speaking... you still need a bit of attention :D
by jeremycochoy 3y ago
Well, RWKV is using some linear (non-quadratic) form of attention so... strictly speaking... you still need a bit of attention :D