4 ms·
> We have mathematically proven that transformers can solve any problem, provided they are allowed to generate as many intermediate reasoning tokens as needed.
by tossandthrow 2y ago
> We have mathematically proven that transformers can solve any problem, provided they are allowed to generate as many intermediate reasoning tokens as needed.
This is also the case with plain and regular RNNs
- baq 2y agoNow just need an autoregressive transformer <==> RNN isomorphism paper and we're golden
- logicchains 2y agoPlain RNNs are theoretically weaker than transformers with COT: https://arxiv.org/abs/2402.18510 https://arxiv.org/abs/2402.18510 .
- tossandthrow 2y agoThe paper says transformers perform better than RNNs, which is not surprising. However, they are both, theoretically, Turing complete computers. So they are equally expressive.
- Intralexical 2y agoIsn't it also expected to be the case with RNGs?