2 ms·
> However, we also try to leverage prompt-caching as much as possible to lower costs and improve latency. Interesting! That does have 5 minute expiry on Claude
by handfuloflight 2y ago
> However, we also try to leverage prompt-caching as much as possible to lower costs and improve latency.
Interesting! That does have 5 minute expiry on Claude, and your users can use Codebuff in an unoptimal way. Do you have plans in aligning your users towards using the tool in a way that makes the most use of prompt caches?
- jahooma 2y agoThat's a really great point. Since we manage the context, we should clear the old files if it's been > 5 minutes. Thanks for the idea!