4 ms·
So maybe RWKV [1] is the next step. It parallelizes even better and seems to have no sequence limit. [1] https://github.com/BlinkDL/RWKV-LM https://github.com/
by naasking 4y ago
So maybe RWKV [1] is the next step. It parallelizes even better and seems to have no sequence limit.
[1] https://github.com/BlinkDL/RWKV-LM https://github.com/BlinkDL/RWKV-LM