4 ms·
Those issues are considered artifacts of the current crop of LLMs in academic circles; there is already research allowing LLMs to use millions of different tool
by storus 1y ago
Those issues are considered artifacts of the current crop of LLMs in academic circles; there is already research allowing LLMs to use millions of different tools at the same time, and stable long contexts, likely reducing the amount of agents to one for most use cases outside interfacing different providers.
Anyone basing their future agentic systems on current LLMs would likely face LangChain fate - built for GPT-3, made obsolete by GPT-3.5.
- simonw 1y agoCan you link to the research on millions of different terms and stable long contexts? I haven't come across that yet.
- storus 1y agoYou can look at AnyTool, 2024 (16,000 tools) and start looking at newer research from there. https://arxiv.org/abs/2402.04253 https://arxiv.org/abs/2402.04253 For long contexts start with activation beacons and RoPE scaling.
- simonw 1y agoI would classify AnyTool as a context engineering trick. It's using GPT-4 function calls (what we would call tool calls today) to find the best tools for the current job based on a 3-level hierarchy search. Drew calls that one "Tool Loadout" https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html#tool-loadout https://www.dbreunig.com/2025/06/26/how-to-fix-your-context....
- timr 1y agoSo great. We have not one, but two different ways of saying "use text search to find tools". This field, I swear...it's the PPAP [1] of engineering. [1] https://www.youtube.com/watch?v=NfuiB52K7X8 https://www.youtube.com/watch?v=NfuiB52K7X8 I have a toool...I have a seeeeearch...unh! Now I have a Tool Loadout!" *dances around in leopard print pyjamas*
- nyrikki 1y agoThanks for the link. It finally explained why I was getting hit up by recruiters for a job that was for a data broker looking to do what seemed like silly uses. Cloud API recommender systems must seem like a gift to that industry. Not my area anyways but I couldn't see a profit model for a human search for an API when what they wanted is well covered by most core libraries in Python etc...
- Art9681 1y agoRoPE scaling is not an ideal solution since all LLMs in general start degrading at around 8k. You also have the problem of cost by yolo'ing long context per task turn even if the LLM were capable of crunching 1M tokens. If you self host then you have the problem of prompt processing time. So it doesnt matter in the end if the problem is solved and we can invoke n number of tools per task turn. It will be a quick way to become poor as long as providers are charging per token. The only viable solution is to use a smart router so only the relevant tools and their descriptions are appended to the context per task turn.
- Foreignborn 1y agoyes, but those aren’t released and even then you’ll always need glue code. you just need to knowingly resource what glue code is needed, and build it in a way it can scale with whatever new limits that upgraded models give you. i can’t imagine a world where people aren’t building products that try to overcome the limitations of SOTA models
- storus 1y agoMy point is that newer models will have those baked in, so instead of supporting ~30 tools before falling apart they will reliably support 10,000 tools defined in their context. That alone would dramatically change the need for more than one agent in most cases as the architectural split into multiple agents is often driven by the inability to reliably run many tools within a single agent. Now you can hack around it today by turning tools on/off depending on the agent's state but at some point in the future you might afford not to bother and just dump all your tools to a long stable context, maybe cache it for performance, and that will be it.
- ZYbCRq22HbJ2y7 1y agoThere will likely be custom, large, and expensive models at an enterprise level in the near future (some large entities and governments already have them (niprgpt)). With that in mind, what would be the business sense in siloing a single "Agent" instead of using something like a service discovery service that all benefit from?
- storus 1y agoMy guess is the main issue is latency and accuracy; a single agent without all the routing/evaluation sub-agents around it that introduce cumulative errors, lead to infinite loops and slow it down would likely be much faster, accurate and could be cached at the token level on a GPU, reducing token preprocessing time further. Now different companies would run different "monorepo" agents and those would need something like MCP to talk to each other at the business boundary, but internally all this won't be necessary. Also the current LLMs have still too many issues because they are autoregressive and heavily biased towards the first few generated tokens. They also still don't have full bidirectional awareness of certain relationships due to how they are masked during the training. Discrete diffusion looks interesting but I am not sure how does that one deal with tools as I've never seen a model from that class using any tools.
- ZYbCRq22HbJ2y7 1y agoHow would "a million different tool calls at the same time" work? For instance, MCP is HTTP based, even at low latency in incredibly parallel environments that would take forever.
- kiitos 1y agoHTTP is an implementation detail, and doesn't represent any kind of unavoidable bottleneck vs. any other transport protocol one might use to do these kinds of request/response interactions.
- Jarwain 1y agoMCPs aren't the only way to embed tool calls into an LLM
- coldtea 1y agoDoesn't change the argument.
- tptacek 1y agoIt obviously does.
- Art9681 1y agoIt does not. Context is context no matter how you process it. You can configure tools without MCP or with it. No matter. You still have to provide that as context to an LLM.
- tptacek 1y agoIf you're using native tool calls and not MCP, the latency of calls is a nonfactor; that was the concern raised by the root comment.
- Art9681 1y agoIt wouldn't. There is a difference between theory and practicality. Just because we could, doesnt mean we should, especially when costs per token are considered. Capability and scale are often at odds.
- dinvlad 1y ago> already research allowing LLMs to use millions of different tools Hmm first time hearing about this, could you share any examples please?
- simonw 1y agoSee this comment https://news.ycombinator.com/item?id=44428548 https://news.ycombinator.com/item?id=44428548