3 ms·
Other dumb architectures don't parallelize as well. Other architectures that parallelize at similar levels (RNN-RWKV, H3, S4, etc.) do perform well at similar p
by whimsicalism 3y ago
Other dumb architectures don't parallelize as well. Other architectures that parallelize at similar levels (RNN-RWKV, H3, S4, etc.) do perform well at similar parameter counts and data sizes.
- homarp 3y agoRNN-RWKV - https://news.ycombinator.com/item?id=36038868 https://news.ycombinator.com/item?id=36038868 H3 - https://news.ycombinator.com/item?id=34673535 https://news.ycombinator.com/item?id=34673535 S4 - https://srush.github.io/annotated-s4/ https://srush.github.io/annotated-s4/