I think that's an oversimplification. LLMs have a limited context window of tokens. But that isn't necessarily a limitation: it's been proven that a LLM can sim

I think that's an oversimplification. LLMs have a limited context window of tokens. But that isn't necessarily a limitation: it's been proven that a LLM can simulate any algorithm even with a limited context window (https://arxiv.org/abs/2410.03170 https://arxiv.org/abs/2410.03170), making it computationally universal. Even if that weren't true, the context windows can be quite large and will get bigger as people figure out how to optimize LLMs. For example Gemini 1.5 has a context of 2 million tokens. A book is typically around 120,000 words, so that's almost 20 books. So one could argue with a context this big they could construct reasoning chains involving far more disparate pieces of information than humans typically work with simultaneously, and arguably that demonstrates intelligence as well.