I'm not sure you're right you know. I think that the way that an LLM maintains a conversation is to have the conversational thread fed into an instance of it at

I'm not sure you're right you know. I think that the way that an LLM maintains a conversation is to have the conversational thread fed into an instance of it at every step. You can see this if you do a conversation step by step and then take all of it (including the LLM responses) apart from the final outcome and paste that into a new thread: https://chatgpt.com/share/6814e827-81cc-8001-a75f-64ed6df5fc4c https://chatgpt.com/share/6814e827-81cc-8001-a75f-64ed6df5fc... https://chatgpt.com/share/6814e7fb-f4d0-8001-a503-9c991df832aa https://chatgpt.com/share/6814e7fb-f4d0-8001-a503-9c991df832... if you think about how these things work as services you can see that this makes sense. The model weights are several gb, so caching the model weights for utilisation by a particular customer is impractical. So if the forward pass does update the model then that's instantly discarded, what's retained is the conversational text, and that's the bit that's uploaded to the model on each iteration for a new reply. There are hundreds of requests pinging through the data center where the models are used every second, all of these use the same models. But if you believe that there is a reasoning process taking place in the text then fair enough.