The system prompt will always match in the prefix cache. I just meant it could be prefilled before any user queries on completely different hardware. Then you a

The system prompt will always match in the prefix cache. I just meant it could be prefilled before any user queries on completely different hardware. Then you are only dealing with the n^2 only for the actual user prompt. We're in agreeance I think.