4 ms·
For those unaware of Alex Graves' work, check out the NTM paper: https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401 This actually does what this b
by typon 2y ago
For those unaware of Alex Graves' work, check out the NTM paper: https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401
This actually does what this blog post pretends to
- nyrikki 2y agoExcept it proves nothing about TC, it just refers to a paper that demonstrated RNNs were TC. The authors decision to name their creation a "Neural Turing Machine" doesn't make it computationally equivalent. The fact that it explicitly states: > ...analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to- end, allowing it to be efficiently trained with gradient descent. Actually demonstrates it is not equivalent. Note that I am not saying their approach is wrong, but that their overloading of terms suggests more than they claim. While 'general-purpose computer' is ambiguous, a Turning machine is not as opaque. LLMs can be TC with infinite space for input and infinite width. For finite input, size, and depth, LLMs aren't even equivalent to primitive recursive functions IIRC. With consent depth threshold circuits being a recent claim as the limits of LLMs.