3 ms·
For Deepseek V4, The main effect of raising the reasoning effort is to add a little section[1] to the end of the system prompt that says "BTW make sure to think
by Centigonal 4mo ago
For Deepseek V4, The main effect of raising the reasoning effort is to add a little section[1] to the end of the system prompt that says "BTW make sure to think really hard! :)"
If Anthropic's models work the same way, then changing reasoning effort would break the cache because the API has to modify the system prompt given at the very start of the context and rerun the whole thing through the inference server.
This kind of limitation is one reason Opus 4.8's mid-conversation system messages[2] are actually a pretty big deal (if they actually work).
[1] https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/main/encoding/encoding_dsv4.py#L64 https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash/blob/ma...
[2] https://platform.claude.com/docs/en/build-with-claude/mid-conversation-system-messages https://platform.claude.com/docs/en/build-with-claude/mid-co...
- Chu4eeno 4mo ago> This kind of limitation is one reason Opus 4.8's mid-conversation system messages[2] are actually a pretty big deal (if they actually work). Didn't they start injecting system messages telling Claude to calm his tits in overly long and emotional (iirc it triggered on some keywords) chat contexts last year?
- simianwords 4mo agoI don't get it!? Why not just add the "BTW make sure to think really hard!" at the end in the new message? Is it harder to post-train in such a way?
- Centigonal 4mo agoThe model is trained to treat system messages differently from regular user/assistant messages. Most models are trained to only expect system messages at the start of the conversation. This is changing now.
- deleted 4mo ago[deleted]
- deleted 4mo ago[deleted]
- firemelt 4mo agolmao i used to do that manually is that means i raising the effort?