2 ms·
Hmmm... Why wouldn't this be handled like other end of prompt things like the current mode?
by foota 2mo ago
Hmmm... Why wouldn't this be handled like other end of prompt things like the current mode?
- janalsncm 2mo agoPrompt caching only works based on the prefix. Let’s call your output Y and the low reasoning prompt A and a medium reasoning prompt B. Previously you were at A+Y. Switching to medium reasoning makes it B+Y. There’s no prefix which can be cached, so the entire B+Y needs to be reprocessed.
- foota 1mo agoWhy can't effort just be at the end of the prompt? My understanding is that that is how modes and other "dynamic updates" like the time are handled.