3 ms·
Grok 4 has about 99% accuracy in picking the right tools and making tool calls with proper arguments almost every single time. Where did this number come from?
by ajd555 1y ago
Grok 4 has about 99% accuracy in picking the right tools and making tool calls with proper arguments almost every single time.
Where did this number come from? What is "the right tool"? I find this extremely subjective. As most engineers know, there is no right tool, but mostly a compromise where you pick the least worst tool and choose what risks you're willing to manage or not.
- Byamarro 1y agoThat's langchain terminology. LLMs usually are exposed to a set of tools. It's usually pretty obvious which are obvious, since there's only one tool that's even remotely associated with the task at hand.
- ajd555 1y agoThanks for the info. This makes the article slightly less intolerable!
- mdaniel 1y agoI believe in this context it means "tool" as in the MCP definition, e.g. "of the catalog of MCP integrations, it doesn't try to use the playright one to browse the web, it'd use the AWS docs one directly" This is just my speculation, though, as I've never used Grok anything
- ajd555 1y agoYeah, based on a previous comment, that makes sense. I am a little reassured that is what the author meant.