3 ms·
Awesome, thanks! I do keep running on the issues, but the levers as you say make it harder to implement. As of right now, I could only replace the JSON tool ca
by tough 1y ago
Awesome, thanks! I do keep running on the issues, but the levers as you say make it harder to implement.
As of right now, I could only replace the JSON tool calling on LLM's on something I fully control like vLLM, and the big labs probably are happy to over-charge a 20-30% tokens for each tool call, so they wouldn't really be interested on replacing json any time soon)
also it feels like battling against a giant which is already an standard, maybe there's a place for it on really specialized workflows where those savings make the difference (not only money, but you also gain a 20-30% extra token window, if you don't waste it on quotes and braces and what not
Thanks for replying!