3 ms·
Glad this exists. It will force anthropic to fix their flawed cache mechanism.
by ATMLOTTOBEER 2mo ago
Glad this exists. It will force anthropic to fix their flawed cache mechanism.
- cortesoft 2mo agoHow should they fix it?
- s0ck_r4w 2mo agoMake the cache TTL more adaptive. Have more tiers than just 5m and 1h. Long-running subagents become increasingly ubiquitous. There's no reason for Anthropic to not do a better job for this use case.
- addaon 2mo agoThere are a lot of workflows where cache is likely to be consumed exactly once (e.g. anything chat-like where a single thread is extended; if a prefix cache hit is found, and a set of new longer prefixes is inserted in the cache, it is unlikely that the original shorter prefix will be consumed in later turns). I could see allowing each session key a small number (1?) of 24-hour cache entries, where inserting a new cache entry (perhaps the maximum-length prefix, perhaps an API-tagged specified-length prefix) consumes that slot and either boots the rest of the items from cache, or demotes them to 5 minutes or something. Basically: workflow awareness, not uniform handling.
- randomblock1 2mo agoDefault to 1h. Allow setting it to longer or shorter, granularly. Add /pause to mark it for eviction, for a token refund.
- SwellJoe 2mo agoWhile Anthropic is far behind on caching and efficiency (made more dramatically apparent by how much cheaper it is to use GPT 5.6 Sol at API rates than even Opus 4.8, much less Fable), a bunch of people forcing their way to the front of the queue at the expense of everyone else isn't going to solve that.
- skeledrew 2mo agoThey definitely aren't. Over 98% of my tokens over the last 6 month have been cache reads (did an analysis just a couple days ago), which has made my Pro plan highly valuable.
- SwellJoe 2mo agoI certainly don't see much in the way of cached token rates when I use the API, whereas when I use DeepSeek via API, it's mostly at the cached token rate.