2 ms·
Hmmm... Why wouldn't this be handled like other end of prompt things like the current mode?
by foota 3mo ago
Hmmm... Why wouldn't this be handled like other end of prompt things like the current mode?
- janalsncm 3mo agoPrompt caching only works based on the prefix. Let’s call your output Y and the low reasoning prompt A and a medium reasoning prompt B. Previously you were at A+Y. Switching to medium reasoning makes it B+Y. There’s no prefix which can be cached, so the entire B+Y needs to be reprocessed.
- foota 3mo agoWhy can't effort just be at the end of the prompt? My understanding is that that is how modes and other "dynamic updates" like the time are handled.