24 ms·
I'm not sure we're at "great" yet. Gemini 2.5 pro fails maybe 50% of the time for me at even generating a syntactically successful tool call.
by svachalek 1y ago
I'm not sure we're at "great" yet. Gemini 2.5 pro fails maybe 50% of the time for me at even generating a syntactically successful tool call.
- simonw 1y agoAre you using Gemini's baked in API tool calling mechanisms or are you prompting it and telling it to produce specific XML/JSON?
- mediaman 1y agoWhat do you recommend for this? I've actually had good luck having them create XML, even though you're "supposed" to use the native tool calling in a JSON schema. There seems to be far fewer issues with getting JSON syntax correct.
- simonw 1y agoI'm using their native tool calling: https://github.com/simonw/llm-gemini/commit/a7f1096cfbb733018eb41c29028a8cc6160be298 https://github.com/simonw/llm-gemini/commit/a7f1096cfbb73301... - it's been working really well for me so far.