4 ms·

For models that can call tools, just giving them a think tool where they can write their thoughts, can improve performance. Even reasoning models, surprisingly

For models that can call tools, just giving them a think tool where they can write their thoughts, can improve performance. Even reasoning models, surprisingly enough. https://www.anthropic.com/engineering/claude-think-tool https://www.anthropic.com/engineering/claude-think-tool

jermaustin1 2y ago
I did something similar for a "game engine" let the NPCs remember things from other NPCs' and the PC's interaction with them. It wasn't perfect, but the player could negotiate a cheaper price on a dagger for instance if they promised to owe the NPC a larger payout next time they returned to the shop. And it worked... most of the time, the shop owner remembered the debt and inquired about it on the next interaction - but not always, which I guess is kind of "human".
rahimnathwani 2y ago
This is similar to this, right? https://github.com/modelcontextprotocol/servers/tree/main/src/sequentialthinking https://github.com/modelcontextprotocol/servers/tree/main/sr...
- eightysixfour 2y ago
  No, the approach in the Anthropic blog has no structure. It is literally just a place for the model to “write its thoughts.” If you took the server above and stripped everything except the “thought” tool. With reasoning models you don’t carry forward the reasoning tokens in the context for the next call, so my hypothesis is that calling a tool/responding lets the model ditch a lot of built up context and “refocus.”
  - rahimnathwani 2y ago
    Although the sequential thinking MCP asks for structure, I don't think it persists anything.
    - eightysixfour 2y ago
      You can see in the code it persists the thinking history and branches until the MCP server is shut off.
      - rahimnathwani 2y ago
        Right, but there's no method for the client to retrieve the content of previous thoughts. So it's analogous to a human writing notes for themselves as part of the thinking process, but never reading what they've written.

jermaustin1 2y ago

I did something similar for a "game engine" let the NPCs remember things from other NPCs' and the PC's interaction with them. It wasn't perfect, but the player could negotiate a cheaper price on a dagger for instance if they promised to owe the NPC a larger payout next time they returned to the shop. And it worked... most of the time, the shop owner remembered the debt and inquired about it on the next interaction - but not always, which I guess is kind of "human".

rahimnathwani 2y ago

This is similar to this, right? https://github.com/modelcontextprotocol/servers/tree/main/src/sequentialthinking https://github.com/modelcontextprotocol/servers/tree/main/sr...

eightysixfour 2y ago
No, the approach in the Anthropic blog has no structure. It is literally just a place for the model to “write its thoughts.” If you took the server above and stripped everything except the “thought” tool. With reasoning models you don’t carry forward the reasoning tokens in the context for the next call, so my hypothesis is that calling a tool/responding lets the model ditch a lot of built up context and “refocus.”
- rahimnathwani 2y ago
  Although the sequential thinking MCP asks for structure, I don't think it persists anything.
  - eightysixfour 2y ago
    You can see in the code it persists the thinking history and branches until the MCP server is shut off.
    - rahimnathwani 2y ago
      Right, but there's no method for the client to retrieve the content of previous thoughts. So it's analogous to a human writing notes for themselves as part of the thinking process, but never reading what they've written.