23 ms·
Why wouldn't you turn on prompt caching? There must be a reason why it's a toggle rather than just being on for everything.
by colordrops 1y ago
Why wouldn't you turn on prompt caching? There must be a reason why it's a toggle rather than just being on for everything.
- TimMoore 1y agoWriting to the cache is more expensive than a request with caching disabled. So it only makes economic sense to do it when you know you're going to use the cached results. See https://docs.claude.com/en/docs/build-with-claude/prompt-caching#pricing https://docs.claude.com/en/docs/build-with-claude/prompt-cac...
- deleted 1y ago[deleted]
- adastra22 1y agoWhen you know the context is a one-and-done. Caching costs more than just running the prompt, but less than running the prompt twice.