4 ms·
Prompt caching only works based on the prefix. Let’s call your output Y and the low reasoning prompt A and a medium reasoning prompt B. Previously you were at
by janalsncm 2mo ago
Prompt caching only works based on the prefix. Let’s call your output Y and the low reasoning prompt A and a medium reasoning prompt B.
Previously you were at A+Y. Switching to medium reasoning makes it B+Y. There’s no prefix which can be cached, so the entire B+Y needs to be reprocessed.
- foota 2mo agoWhy can't effort just be at the end of the prompt? My understanding is that that is how modes and other "dynamic updates" like the time are handled.