4 ms·
The feedforward nature does mean it's not computationally complete for a single iteration. But a single iteration only generates a single token, which gets fed
by phire 4y ago
The feedforward nature does mean it's not computationally complete for a single iteration.
But a single iteration only generates a single token, which gets fed back in to the next iteration. If you let it loop forever, generating unlimited tokens, then the theoretical computational complexity is unbounded.
At least theoretically.
In practice, it's not going to split computation over that many tokens, and you run into token limits.
- qsort 4y agoWell, okay, but you're running into the same problem with memory, and I could invoke the space variant of the theorem. You could say that an infinite family of GPT-like models with increasing context size collectively form a Turing-complete computational model and I would have no objections, but you're stretching the definition a bit...
- cypress66 4y agoIn the real world nothing has infinite memory, so no computer would be turing complete. Therefore this requirement is ignored.