> Every time you run inference on the model, the client gets to compose the context in any way they want; there are more things than just concatenating prompts

> Every time you run inference on the model, the client gets to compose the context in any way they want; there are more things than just concatenating prompts and LLM ouputs. You can always launch a subagent with a fresh context. There are further things that you could do by tweaking the underlying transformer model (such as "joining" any number of independently cached contexts together on an equal basis, without having to rerun prefill on the "later" contexts) but this is quite general already.