3 ms·
It’s useful to consider the impact this has on conversations. In an ongoing conversation with a model you end up re-submitting the full test of the previous co
by simonw 2y ago
It’s useful to consider the impact this has on conversations.
In an ongoing conversation with a model you end up re-submitting the full test of the previous conversation - both prompts and responses - at every step. This means the cost per prompt in that conversation increases each time.
Claude prompt caching can start saving you money even with just a single user having a conversation, provided each of their replies is within five minutes of the previous reply.