3 ms·
The real elephant in the room is pricing for KV cache writes and reads. That makes all the difference for tasks with large context.
by jnwatson 3mo ago
The real elephant in the room is pricing for KV cache writes and reads. That makes all the difference for tasks with large context.
- ianberdin 3mo ago[dead]
- petercooper 3mo agoCache writes/reads are the majority of the picture for agentic development. When you see these people saying a coding run "used 200 million tokens" or whatever, most of that is cache reads, so it should be the headline price IMO (and is one reason why DeepSeek API is so striking with its minuscule cache pricing).