4 ms·

It's so easy to google this and find that they all do exactly this. Gemini: https://ai.google.dev/gemini-api/docs/code-execution https://ai.google.dev/gemini-a

It's so easy to google this and find that they all do exactly this. Gemini: https://ai.google.dev/gemini-api/docs/code-execution https://ai.google.dev/gemini-api/docs/code-execution ChatGPT: https://help.openai.com/en/articles/8437071-data-analysis-with-chatgpt https://help.openai.com/en/articles/8437071-data-analysis-wi... Claude: https://claude.com/blog/analysis-tool https://claude.com/blog/analysis-tool Reasoning only gets you so far, even humans write code or use spreadsheets, calculators, etc, to get their answers to problems.

simianwords 8mo ago
you have just linked the fact that they have code executions but not proved that it is needed for strawberry problem. there are multiple ways to disprove this 1. GPT o1 was released and it never supported the tools and it easily solved the strawberry problem - it was named strawberry internally 2. you can run GPT 5.2-thinking in the API right now and deny access to any tools, it will still work 3. you can run deepseek locally without tools and run it, it will still work Overall this idea that LLM's cant reason and need tools to do that is misleading and false and easily disproven.
- danpalmer 8mo ago
  Oh right you're very focused on specifically the strawberry problem. I just gave that as a throwaway example. It's a solution but not necessarily the solution for something that simple. My point was much more general, that code execution is a key part of these models ability to perform maths, analysis, and provide precise answers. It's not the only way, but a key way that's very efficient compared to more inference for CoT.
  - simianwords 8mo ago
    I agree that tool usage dramatically improves the utility of LLM's. But it is absolutely not needed for the strawberry problem. It can perform complicated arithmatic without tools - multiplying multiple 20 digit numbers, division and so on (to an extent).

simianwords 8mo ago

you have just linked the fact that they have code executions but not proved that it is needed for strawberry problem. there are multiple ways to disprove this 1. GPT o1 was released and it never supported the tools and it easily solved the strawberry problem - it was named strawberry internally 2. you can run GPT 5.2-thinking in the API right now and deny access to any tools, it will still work 3. you can run deepseek locally without tools and run it, it will still work Overall this idea that LLM's cant reason and need tools to do that is misleading and false and easily disproven.

danpalmer 8mo ago
Oh right you're very focused on specifically the strawberry problem. I just gave that as a throwaway example. It's a solution but not necessarily the solution for something that simple. My point was much more general, that code execution is a key part of these models ability to perform maths, analysis, and provide precise answers. It's not the only way, but a key way that's very efficient compared to more inference for CoT.
- simianwords 8mo ago
  I agree that tool usage dramatically improves the utility of LLM's. But it is absolutely not needed for the strawberry problem. It can perform complicated arithmatic without tools - multiplying multiple 20 digit numbers, division and so on (to an extent).