> This is just that scaled up An LLM is not a Markov process. They are fundamentally different. An LLM conditions the next token prediction on the entire conte

> This is just that scaled up An LLM is not a Markov process. They are fundamentally different. An LLM conditions the next token prediction on the entire context window (via the attention mechanism), not just the previous token. Besides the token history window it also maintains a cache of neural activations which is updated at every step. Otherwise you could use the same reasoning to argue that a human is a Markov process, which is absurd, but vacuously true if "state" means the quantum level configuration of every atom in the body.