4 ms·
It's not just the retrieval, tool calls entail another call to the LLM (ToolMessage) and possibly the result will then require other tool calls. Massive latenc
by laichzeit0 2y ago
It's not just the retrieval, tool calls entail another call to the LLM (ToolMessage) and possibly the result will then require other tool calls. Massive latency.