3 ms·
Custom tool calling formats are iffy in my experience. The models are all reinforcement learned to follow specific ones, so it’s always a battle and feels to me
by data-ottawa 8mo ago
Custom tool calling formats are iffy in my experience. The models are all reinforcement learned to follow specific ones, so it’s always a battle and feels to me like using the tool wrong.
Have you had good results with the other frontier models?
- thegeomaster 8mo agoNot the parent commenter, but in my testing, all recent Claudes (4.5 onward) and the Gemini 3 series have been pretty much flawless in custom tool call formats.
- data-ottawa 8mo agoThanks. I’ve tested local models from Qwen, GLM, and Devstral families.
- pcwelder 8mo agoAll anthropic models. Gemini 2.5 pro and above. Gemini 3 flash is very good too. GPT models can follow tool format correctly but don't keep on going. Grok-4+ are decent but with issues in longer chats. Kimi 2.5 has issues with it reverting to its RL tool format.