3 ms·
How would "a million different tool calls at the same time" work? For instance, MCP is HTTP based, even at low latency in incredibly parallel environments that
by ZYbCRq22HbJ2y7 1y ago
How would "a million different tool calls at the same time" work? For instance, MCP is HTTP based, even at low latency in incredibly parallel environments that would take forever.
- kiitos 1y agoHTTP is an implementation detail, and doesn't represent any kind of unavoidable bottleneck vs. any other transport protocol one might use to do these kinds of request/response interactions.
- Jarwain 1y agoMCPs aren't the only way to embed tool calls into an LLM
- coldtea 1y agoDoesn't change the argument.
- tptacek 1y agoIt obviously does.
- Art9681 1y agoIt does not. Context is context no matter how you process it. You can configure tools without MCP or with it. No matter. You still have to provide that as context to an LLM.
- tptacek 1y agoIf you're using native tool calls and not MCP, the latency of calls is a nonfactor; that was the concern raised by the root comment.
- Art9681 1y agoIt wouldn't. There is a difference between theory and practicality. Just because we could, doesnt mean we should, especially when costs per token are considered. Capability and scale are often at odds.
- nkohari 1y agoThere's a difference between discovery (asking an MCP server what capabilities it has) and use (actually using a tool on the MCP server). I think the comment you're replying to is talking about discovery rather than use; that is, offering a million tools to the model, not calling a million tools simultaneously.