2 ms·
Not exactly true ... KV and prompt caching is a thing
by Jabrov 1y ago
Not exactly true ... KV and prompt caching is a thing
- yahoozoo 1y agoAssuming you include the same prompts in the new request that were cached in the previous ones.
- throw310822 1y agoAs far as I understand, the entire chat is the prompt. So at the each round, the previous chat up to that point could already be cached. If I'm not wrong, Claude APIs require an explicit request to cache the prompt, while OpenAI's handle this automatically.
- iamnotagenius 1y ago[dead]
- littlestymaar 1y agoI don't understand how you are downvoted…