4 ms·
Now just need an autoregressive transformer <==> RNN isomorphism paper and we're golden
by baq 2y ago
Now just need an autoregressive transformer <==> RNN isomorphism paper and we're golden
- logicchains 2y agoPlain RNNs are theoretically weaker than transformers with COT: https://arxiv.org/abs/2402.18510 https://arxiv.org/abs/2402.18510 .
- tossandthrow 2y agoThe paper says transformers perform better than RNNs, which is not surprising. However, they are both, theoretically, Turing complete computers. So they are equally expressive.