3 ms·
Is Claude caching by whole message only? Pretty sure OpenAI caches up to the first differing character.
by svachalek 1y ago
Is Claude caching by whole message only? Pretty sure OpenAI caches up to the first differing character.
- jasonjmcghee 1y agoInteresting. Claude places breakpoints. Afaik - no way to do mid message. I believe (but not positive) there are 4 breakpoints. 1. End of tool definitions 2. End of system prompt 3. End of messages thread 4. (Least sure) 50% of the way through messages thread? This is how I've seen it done in open source things / seems optimal based on constraints of anthropic API (max 4 breakpoints)