4 ms·

The system prompt part is surely cached across all users.

• dcrazy 2mo ago
  For Anthropic models, yes. But OP was using Claude Code with GPT.
• tosh 2mo ago
  the system prompt still takes up useful space in the context window and steers the model into unnecessary actions and over-thinking patterns
• yojo 2mo ago
  You still pay cache token costs on API calls. Cache cost/token are 90% lower, but you pay it every single turn. I’m not sure if they let you skip the cache write cost on the first turn. That would imply cross-user caching infrastructure or special casing the default system prompt to give you a discount. Maybe? Away from the computer but you could try a “hello” in a fresh session and see what was billed.
  • cyanydeez 2mo ago
    Its inexpensive to reserve KV cache for the first turn and would benefit users, given the first turn already requires costly locating and allocating a model slot for a user. So yeah, when they banned tgird party harnesses there was a technical and $ case to have.