4 ms·
Except it proves nothing about TC, it just refers to a paper that demonstrated RNNs were TC. The authors decision to name their creation a "Neural Turing Machi
by nyrikki 3y ago
Except it proves nothing about TC, it just refers to a paper that demonstrated RNNs were TC.
The authors decision to name their creation a "Neural Turing Machine" doesn't make it computationally equivalent.
The fact that it explicitly states:
> ...analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-
end, allowing it to be efficiently trained with gradient descent.
Actually demonstrates it is not equivalent.
Note that I am not saying their approach is wrong, but that their overloading of terms suggests more than they claim.
While 'general-purpose computer' is ambiguous, a Turning machine is not as opaque.
LLMs can be TC with infinite space for input and infinite width.
For finite input, size, and depth, LLMs aren't even equivalent to primitive recursive functions IIRC.
With consent depth threshold circuits being a recent claim as the limits of LLMs.