3 ms·
Does their prompt caching work with semantically equivalent queries, or do they match lexically?
by esafak 2y ago
Does their prompt caching work with semantically equivalent queries, or do they match lexically?
- simonw 2y agoYou have to mark exactly which parts of the prompt you want to cache. The only vendor I've seen with "automatic" prompt caching so far is DeepSeek https://platform.deepseek.com/api-docs/news/news0802/ https://platform.deepseek.com/api-docs/news/news0802/
- deleted 2y ago[deleted]