3 ms·
They charge you for cache writes already. There's a price assigned to it. Even if you're on subscription, it counts against your usage (very easy to observe wit
by dannyw 3mo ago
They charge you for cache writes already. There's a price assigned to it. Even if you're on subscription, it counts against your usage (very easy to observe with a big cache miss).
Anthropic (and now OpenAI too for 5.6) prompt caching is not free.
- searealist 3mo agoHolding your cache in VRAM for 5 minutes or 3 hours have very different costs to them. "They already charge me to park my car, why can't I leave it there for a year for the same price as 1 week?"