I think part of what I suspect is going on here too is more computation and finiteness. It seems correct that LLM architectures cannot perform too much computat

I think part of what I suspect is going on here too is more computation and finiteness. It seems correct that LLM architectures cannot perform too much computation (unless you unroll it in the context). On the other hand you can look at statistical model identification in, say, nonlinear control. This can absolutely lead to unboundedly long predictions.