That's true. The fact that an LLM is a pure function of (all previous tokens) -> (next token), with internal state like KV cache only existing for optimisation

That's true. The fact that an LLM is a pure function of (all previous tokens) -> (next token), with internal state like KV cache only existing for optimisation purposes, is pretty mind-blowing. I guess it was more the "predictor" part I had issue with. There's a tendency to reach for statistical or probabilistic terminology to describe things that aren't usefully understood in those terms. For example in the "Speed Always Wins" LLM technical survey (https://arxiv.org/pdf/2508.09834 https://arxiv.org/pdf/2508.09834): > The gate is a crucial component to bring sparsity in MoE models. For a batch of input token representations X ∈ RT×D, the gate function G determines the probabilities of dispatching token xi to each expert e ...which is nonsense: the gate simply, directly, selects the experts. There's nothing probabilistic about it.