3 ms·

How have I told it there's a calculator? All I've given it is the ability to search for tools and enable ones it wants. > Of course the model will use the calc

by IanCal 2y ago

How have I told it there's a calculator? All I've given it is the ability to search for tools and enable ones it wants. > Of course the model will use the calculator you've explicitly informed it of I didn't. I also gave it no system prompt pushing it to always use tools or anything. It searches for tools with a query "calculator math root" and is given a list of things that includes a calculator. It picks the calculator, then it uses it. The code and trace are right there.

crackrook 2y ago
What do you imagine `tools` is doing if not a system prompt? I suggest reading the documentation: https://docs.anthropic.com/en/docs/build-with-claude/tool-use#tool-use-system-prompt https://docs.anthropic.com/en/docs/build-with-claude/tool-us... The vast majority of features that Anthropic and OpenAI ship are just clever ways of building system prompts
- IanCal 2y ago
  It's told it can search for tools and it can enable them. It is not told initially that there's a calculator, it asks for one.
  - crackrook 2y ago
    I see, that clarifies things for me, it's not quite like the example I gave then. Even so, doesn't informing the model of the fact that some "tools" are available, immediately before asking it a math problem (that would be virtually impossible for a human to answer precisely), seem like a pretty big hint that it should inquire if a calculator is available? Here's what I get from sonnet in response to the plain user-prompt "What is the eighth root of 4819387574?" """ Let me solve this step by step. To find the 8th root of 4819387574: 1) The 8th root of 4819387574 means finding x where x⁸ = 4819387574 2) This is a large number, but it's a perfect 8th power. 3) One way to approach this is to find factors: 4819387574 = 13⁸ 4) Therefore, the 8th root of 4819387574 is 13. To verify: 13⁸ = 13 × 13 × 13 × 13 × 13 × 13 × 13 × 13 = 4819387574 The answer is 13. """

by IanCal 2y ago

crackrook 2y ago

What do you imagine `tools` is doing if not a system prompt? I suggest reading the documentation: https://docs.anthropic.com/en/docs/build-with-claude/tool-use#tool-use-system-prompt https://docs.anthropic.com/en/docs/build-with-claude/tool-us... The vast majority of features that Anthropic and OpenAI ship are just clever ways of building system prompts

IanCal 2y ago
It's told it can search for tools and it can enable them. It is not told initially that there's a calculator, it asks for one.
- crackrook 2y ago
  I see, that clarifies things for me, it's not quite like the example I gave then. Even so, doesn't informing the model of the fact that some "tools" are available, immediately before asking it a math problem (that would be virtually impossible for a human to answer precisely), seem like a pretty big hint that it should inquire if a calculator is available? Here's what I get from sonnet in response to the plain user-prompt "What is the eighth root of 4819387574?" """ Let me solve this step by step. To find the 8th root of 4819387574: 1) The 8th root of 4819387574 means finding x where x⁸ = 4819387574 2) This is a large number, but it's a perfect 8th power. 3) One way to approach this is to find factors: 4819387574 = 13⁸ 4) Therefore, the 8th root of 4819387574 is 13. To verify: 13⁸ = 13 × 13 × 13 × 13 × 13 × 13 × 13 × 13 = 4819387574 The answer is 13. """