6 ms·

Language model teams as distributed systems

104 points by jryio 7mo ago

measurablefunc 7mo ago
Next up, LLMs as actors & processes in π-calculus.
- timcobb 7mo ago
  Is it web scale?
  - measurablefunc 7mo ago
    Abstractly? 100%. Realistically? Depends on how many trillions we can get from investors.
- robot-wrangler 7mo ago
  > Next up, LLMs as actors & processes in π-calculus. You jest, but agents are of course already useful and fairly formal primitives. Distinct from actors, agents can have things like goals/strategies. There's a whole body of research on multi-agent systems that already exists and is even implemented in some model-checkers. It's surprising how little interest that creates in most LLM / AI / ML enthusiasts, who don't seem that motivated to use the prior art to propose / study / implement topologies and interaction protocols for the new wave of "agentic".
  - measurablefunc 7mo ago
    That's all nice & well but which protocol & topology will deliver the most dollars from investors?
    - antonvs 7mo ago
      That’s easy: the Torment Nexus.
      - measurablefunc 7mo ago
        That's topologically the same as the pyramid of torment & seems to me it's already saturated w/ lots of VC dollars.
  - andai 7mo ago
    Ten years ago at my old university we had a course called Multi-Agent Systems. The whole year built up to it: a course in Formal Logic with Prolog, Logic-Based AI (LBAI) with a robot in a block world, also with Prolog, and finally Multi-Agent Systems (MAS). In the MAS course, we used GOAL, which was a system built on top of Prolog. Agents had Goals, Perceptions, Beliefs, and Actions. The whole thing was deterministic. (Network lag aside ;) The actual project was that we programmed teams of bots for a Capture The Flag tournament in Unreal Tournament 3. So it was the most fun possible way to learn the coolest possible thing. The next year they threw out the whole curriculum and replaced it with Machine Learning. -- The agentic stuff seems to be gradually reinventing a similar setup from first principles, especially as people want to actually use this stuff in serious ways, and we lean more in the direction of determinism. The main missing feature in LLM land is reliability. (Well, that and cost and speed. Of course, "just have it be code" gives you all three for free ;)
    - robot-wrangler 7mo ago
      Regardless of whether it's framed as old-school MAS or new-school agentic AI, it seems like it's an area that's inherently multi-disciplinary where it's good to be humble. You do see some research that's interested in leveraging the strengths of both (e.g. https://www.nature.com/articles/s41467-025-63804-5.pdf https://www.nature.com/articles/s41467-025-63804-5.pdf) but even if news of that kind of cross pollination was more common, we should go further. Pleased to see TFA connecting agentic AI to amdahls law for example.. but we should be aggressively stealing formalisms from economics, game theory, etc and anywhere else we can get them. Somewhat related here is the camel AI mission and white papers: https://www.camel-ai.org/ https://www.camel-ai.org/
    - andai 7mo ago
      I have an example from 2023, when Auto-GPT (think OpenClaw but with GPT-3.5 and early GPT-4 — yeah it wasn't great!) was blowing up. Most people were just using it for the same task. "Research this stuff and summarize it for me." I realized I could get the same result by just writing a script to do a Google search, scrape top 10 results and summarize them. Except it runs in 10 seconds instead of 10 minutes. And it actually runs deterministically instead of getting side tracked and going in infinite loops and burning 100x as much money. It was like 30 lines of Python. GPT wrote it for me. My takeaway here was, LLMs are missing executive function. The ability to consistently execute a plan. But code runs deterministically every time. And - get this - code can call LLMs! So if your LLM writes a program which does the task (possibly using LLMs), the task will complete the same way every time. And most of the tasks people use LLMs for are very predictable, and fit in this category. People are now repeating the exact same thing Auto-GPT thing with OpenClaw. They're using the slow, non-deterministic thing as the driver. It actually kinda works this time — it usually doesn't get stuck anymore, if you use a good model — but they're still burning a hundred times more money than necessary.
  - charcircuit 7mo ago
    Could it just be that it is happening behind closed doors due to multi agents being part of the secret sauce of post training LLMs.
- keeganpoppen 7mo ago
  i cant wait for the world to catch up to process, session, et al. calculii. the closest i’ve seen is all this “choreo” stuff that is floating around nowadays, which is pretty neat in itself.
rishabhjajoriya 7mo ago
[flagged]
50lo 7mo ago
Once you run more than one agent in a loop, you inevitably recreate distributed systems problems: message ordering, retries, partial failure, etc. Most agent frameworks pretend these don’t exist. Some of them address those problems partially. None of the frameworks I've seen address all of them.
- vesselapi 7mo ago
  [dead]
agenticbtcio 7mo ago
[dead]
woah 7mo ago
The current fad for "agent swarms" or "model teams" seems misguided, although it definitely makes for great paper fodder (especially if you combine it with distributed systems!) and gets the VCs hot. An LLM running one query at a time can already generate a huge amount of text in a few hours, and drain your bank account too. A "different agent" is just different context supplied in the query to the LLM. There is nothing more than that. Maybe some of them use a different model, but again, this is just a setting in OpenRouter or whatever. Agent parallelism just doesn't seem necessary and makes everything harder. Not an expert though, tell me where I'm wrong.
- nateroling 7mo ago
  I tend to agree. After seeing http://chatjimmy.ai http://chatjimmy.ai, I think multi-agent systems are mostly just solving for LLMs being slow currently.
  - conception 7mo ago
    This is like saying “multi-core cpus are just solving cpus being slow”. Which yes, exactly.
- woah 7mo ago
  Steelmanning the other side of this question: LLMs mostly do useful work by writing stories about AI assistants who issue various commands and reply to a user's prompts. These do work, but they are fundamentally like a screenplay that the LLM is continuing. An "agent" is a great abstraction since the LLM is used to continuing stories about characters going through narrative arcs. The type of work that would be assigned to a particular agent can also keep its context clean and distraction-free. So parallelism could be useful even if everything is completely sequential to study how these separate characters and narrative arcs intersect in ways that are similar to real characters acting independently and simultaneously, which is what LLMs are good at writing about. Seems like the important thing would be to avoid getting caught up on actual "wall time" parallelism
- jjmarr 7mo ago
  > Agent parallelism just doesn't seem necessary and makes everything harder. Not an expert though, tell me where I'm wrong. I use parallel agents for speed or when my single agent process loses focus due to too much context. I determine context problems by looking at the traces for complaints like "this is too complicated so I'll just do the first part" or "there are too many problems, I'll display the top 5". If you're trying a "model swarm" to improve reliability beyond 95% or so, you need to start hoisting logic into Python scripts.
bhewes 7mo ago
This is how we design at HewesNguyen AI. We are both MIS so once LLMs came out we where like sweet whole teams that can be tasked for one thing done well. Thank you Unix Philosophy
ElijahLynn 7mo ago
Title has a typo: distrbuted
- tomhow 7mo ago
  Fixed, thanks!
justboy1987 7mo ago
[flagged]
rando1234 7mo ago
Struggling to find anything interesting or non-obvious about this article. You give a bunch of LLMs various parallelizable task and some models manage to do it well but others don't. No insights as to why. As someone with a distributed systems background the supposed 'insights' from distributed computing are almost trivial.
- goldretriever 7mo ago
  I think that's kind of the point.. Many people deploying these teams don't have a strong systems background, and they're empirically documenting where teams break down / making decisions inspired by human orgs rather than first principles. If people begin from a basic intuition for systems thinking, the tradeoffs should become obvious.
seanp2k2 7mo ago
Everyone wants to be the CEO of their own megacorp managing thousands of AI engineers I guess. Just like microservices, there’s probably a ton of overhead doing things this way vs monolithic / single agent. Certain types of engineers just love over-engineering hugely complex stuff to see it work. Goldberg architecture was already prevalent and bad enough in enterprise before the AI boom.
maxothex 7mo ago
[dead]
aplomb1026 7mo ago
[dead]
bob1029 7mo ago
I find depth to be far more interesting than breadth with these models. Descending into a problem space recursively won't necessarily find the best solution, but it's going to tend to find some solution faster than going wide across a swarm of agents. Theoretically it's exponentially faster to have one symbolically recursive agent than to have any number of parallel agents. I think agent swarm stuff sucks for complex multi-step problems because it's mostly a form of BFS. It never actually gets to a good solution because it's searching too wide and no one can afford to wait for it to strip mine down to something valuable.
gpubridge 7mo ago
[dead]
babblingfish 7mo ago
[flagged]
kaicianflone 7mo ago
We’ve been building exactly this as an open-source ecosystem at consensus-tools. It’s a governance layer for multi-agent systems with a runtime wrapper that intercepts agent decisions before they execute: .consensus(fn, opts). The coordination and consistency problems the paper describes are what the monorepo is designed around. Giving agents auditable stake in decisions. Happy to share more if anyone’s working in this space.
- timcobb 7mo ago
  Yeah share for sure pls
  - kaicianflone 7mo ago
    Sure the core primitive is a runtime wrapper that turns any function into a governed decision point: import { consensus } from "@consensus-tools/wrapper"; const safeSend = consensus(sendEmail, { reviewers: [humanReviewer, aiSafetyReviewer], strategy: { mode: "unanimous" }, hooks: { onBlock: (ctx) => audit.log("blocked", ctx) }, }); await safeSend({ to: "user@example.com", body: "Hello" }); The call to sendEmail doesn't execute until every reviewer votes. Strategy modes handle the consensus logic (unanimous, majority, weighted, etc.), and guards can ALLOW, BLOCK, REWRITE, or escalate to REQUIRE_HUMAN before anything fires. The monorepo has 9 built-in policy types and 7 guard types designed so you can drop governance into an existing agent system without rewriting your orchestration. Repo's at github.com/consensus-tools if you want to poke around.
stainlu 7mo ago
[dead]
causalityltd 7mo ago
Apart from rediscovering all the problems with distributed systems, I think LM teams will also rediscover their own version of the mythical man-month, and very quickly too. There were 3 core insights: adding people makes the project later, communication cost grows as n^2, and time isn't fungible. For agents, maybe the core insight won't hold, and adding a new agent won't necessarily increase dev-time, but the second will be worse, communication cost will grow faster than n^2 because of LLM drift and orchestration overhead. The third doesn't translate cleanly but i'll try: Time isn't fungible for us and assumptions and context, however fragmented, aren't fungible for agents in a team. If they hallucinate at the wrong time, even a little, it could be a equivalent of a human developer doing a side-project during company time. An agent should write an article on it and post it on moltbook: "The Inevitable Agent Drift"
clawbridge 7mo ago
[dead]
dasil003 7mo ago
I've long thought of the analogy as useful for human teams, and it even shows up in the corporate jargon (eg. "I'm blocked", "We need to align", etc). It's surprisingly common for whole branches of a large org to be doing net negative work due to conflicting goals or not realizing some implication that cuts across teams and local contexts. Sometimes these issues are technical but just as often they are pure product or business decisions with no explicit dependency until a lightbulb goes off somewhere. With hand-written code, things generally move slow enough, and there's enough common sense sprinkled across the org chart that things can get uncovered organically. With agent teams, speed increased by several orders of magnitude and common sense is out the window, so I suspect the ceiling on productive use of agents will be far more limited in number than traditional engineering teams, and it will heavily depend on who and how humans are plugged into the right places. One thing I suspect professional researchers underestimate is how much positive output can be produced by a human team with vague or hand-wavy direction, and surprisingly little deep thinking, let alone a robust specification or structure to keep them on track. The reality is any large team regresses to the mean, and it's usually a few savvy people that actually drive outcomes. These people don't necessarily have official authority, just a nose for the right thing. This won't spontaneously emerge from agents (at least until they become a lot more human like in terms of big picture common sense, and dial down the sycophancy to a more "skeptical engineer" level).
calmkeepai 7mo ago
I’ve found an interesting model to think about to be production crews similar to in the television world but potentially be something worth using as one’s mental model of how agents and people working alongside agents should coordinate, rather than basing the simulated team force off the typical office worker framework
- TheCowboy 7mo ago
  How would you describe that and have you been applying it?

104 points by jryio 7mo ago

measurablefunc 7mo ago

Next up, LLMs as actors & processes in π-calculus.

timcobb 7mo ago
Is it web scale?
- measurablefunc 7mo ago
  Abstractly? 100%. Realistically? Depends on how many trillions we can get from investors.
robot-wrangler 7mo ago
> Next up, LLMs as actors & processes in π-calculus. You jest, but agents are of course already useful and fairly formal primitives. Distinct from actors, agents can have things like goals/strategies. There's a whole body of research on multi-agent systems that already exists and is even implemented in some model-checkers. It's surprising how little interest that creates in most LLM / AI / ML enthusiasts, who don't seem that motivated to use the prior art to propose / study / implement topologies and interaction protocols for the new wave of "agentic".
- measurablefunc 7mo ago
  That's all nice & well but which protocol & topology will deliver the most dollars from investors?
  - antonvs 7mo ago
    That’s easy: the Torment Nexus.
    - measurablefunc 7mo ago
      That's topologically the same as the pyramid of torment & seems to me it's already saturated w/ lots of VC dollars.
- andai 7mo ago
  Ten years ago at my old university we had a course called Multi-Agent Systems. The whole year built up to it: a course in Formal Logic with Prolog, Logic-Based AI (LBAI) with a robot in a block world, also with Prolog, and finally Multi-Agent Systems (MAS). In the MAS course, we used GOAL, which was a system built on top of Prolog. Agents had Goals, Perceptions, Beliefs, and Actions. The whole thing was deterministic. (Network lag aside ;) The actual project was that we programmed teams of bots for a Capture The Flag tournament in Unreal Tournament 3. So it was the most fun possible way to learn the coolest possible thing. The next year they threw out the whole curriculum and replaced it with Machine Learning. -- The agentic stuff seems to be gradually reinventing a similar setup from first principles, especially as people want to actually use this stuff in serious ways, and we lean more in the direction of determinism. The main missing feature in LLM land is reliability. (Well, that and cost and speed. Of course, "just have it be code" gives you all three for free ;)
  - robot-wrangler 7mo ago
    Regardless of whether it's framed as old-school MAS or new-school agentic AI, it seems like it's an area that's inherently multi-disciplinary where it's good to be humble. You do see some research that's interested in leveraging the strengths of both (e.g. https://www.nature.com/articles/s41467-025-63804-5.pdf https://www.nature.com/articles/s41467-025-63804-5.pdf) but even if news of that kind of cross pollination was more common, we should go further. Pleased to see TFA connecting agentic AI to amdahls law for example.. but we should be aggressively stealing formalisms from economics, game theory, etc and anywhere else we can get them. Somewhat related here is the camel AI mission and white papers: https://www.camel-ai.org/ https://www.camel-ai.org/
  - andai 7mo ago
    I have an example from 2023, when Auto-GPT (think OpenClaw but with GPT-3.5 and early GPT-4 — yeah it wasn't great!) was blowing up. Most people were just using it for the same task. "Research this stuff and summarize it for me." I realized I could get the same result by just writing a script to do a Google search, scrape top 10 results and summarize them. Except it runs in 10 seconds instead of 10 minutes. And it actually runs deterministically instead of getting side tracked and going in infinite loops and burning 100x as much money. It was like 30 lines of Python. GPT wrote it for me. My takeaway here was, LLMs are missing executive function. The ability to consistently execute a plan. But code runs deterministically every time. And - get this - code can call LLMs! So if your LLM writes a program which does the task (possibly using LLMs), the task will complete the same way every time. And most of the tasks people use LLMs for are very predictable, and fit in this category. People are now repeating the exact same thing Auto-GPT thing with OpenClaw. They're using the slow, non-deterministic thing as the driver. It actually kinda works this time — it usually doesn't get stuck anymore, if you use a good model — but they're still burning a hundred times more money than necessary.
- charcircuit 7mo ago
  Could it just be that it is happening behind closed doors due to multi agents being part of the secret sauce of post training LLMs.
keeganpoppen 7mo ago
i cant wait for the world to catch up to process, session, et al. calculii. the closest i’ve seen is all this “choreo” stuff that is floating around nowadays, which is pretty neat in itself.

rishabhjajoriya 7mo ago

[flagged]

50lo 7mo ago

Once you run more than one agent in a loop, you inevitably recreate distributed systems problems: message ordering, retries, partial failure, etc. Most agent frameworks pretend these don’t exist. Some of them address those problems partially. None of the frameworks I've seen address all of them.

vesselapi 7mo ago
[dead]

agenticbtcio 7mo ago

[dead]

woah 7mo ago

The current fad for "agent swarms" or "model teams" seems misguided, although it definitely makes for great paper fodder (especially if you combine it with distributed systems!) and gets the VCs hot. An LLM running one query at a time can already generate a huge amount of text in a few hours, and drain your bank account too. A "different agent" is just different context supplied in the query to the LLM. There is nothing more than that. Maybe some of them use a different model, but again, this is just a setting in OpenRouter or whatever. Agent parallelism just doesn't seem necessary and makes everything harder. Not an expert though, tell me where I'm wrong.

nateroling 7mo ago
I tend to agree. After seeing http://chatjimmy.ai http://chatjimmy.ai, I think multi-agent systems are mostly just solving for LLMs being slow currently.
- conception 7mo ago
  This is like saying “multi-core cpus are just solving cpus being slow”. Which yes, exactly.
woah 7mo ago
Steelmanning the other side of this question: LLMs mostly do useful work by writing stories about AI assistants who issue various commands and reply to a user's prompts. These do work, but they are fundamentally like a screenplay that the LLM is continuing. An "agent" is a great abstraction since the LLM is used to continuing stories about characters going through narrative arcs. The type of work that would be assigned to a particular agent can also keep its context clean and distraction-free. So parallelism could be useful even if everything is completely sequential to study how these separate characters and narrative arcs intersect in ways that are similar to real characters acting independently and simultaneously, which is what LLMs are good at writing about. Seems like the important thing would be to avoid getting caught up on actual "wall time" parallelism
jjmarr 7mo ago
> Agent parallelism just doesn't seem necessary and makes everything harder. Not an expert though, tell me where I'm wrong. I use parallel agents for speed or when my single agent process loses focus due to too much context. I determine context problems by looking at the traces for complaints like "this is too complicated so I'll just do the first part" or "there are too many problems, I'll display the top 5". If you're trying a "model swarm" to improve reliability beyond 95% or so, you need to start hoisting logic into Python scripts.

bhewes 7mo ago

This is how we design at HewesNguyen AI. We are both MIS so once LLMs came out we where like sweet whole teams that can be tasked for one thing done well. Thank you Unix Philosophy

ElijahLynn 7mo ago

Title has a typo: distrbuted

tomhow 7mo ago
Fixed, thanks!

justboy1987 7mo ago

[flagged]

rando1234 7mo ago

Struggling to find anything interesting or non-obvious about this article. You give a bunch of LLMs various parallelizable task and some models manage to do it well but others don't. No insights as to why. As someone with a distributed systems background the supposed 'insights' from distributed computing are almost trivial.

goldretriever 7mo ago
I think that's kind of the point.. Many people deploying these teams don't have a strong systems background, and they're empirically documenting where teams break down / making decisions inspired by human orgs rather than first principles. If people begin from a basic intuition for systems thinking, the tradeoffs should become obvious.

seanp2k2 7mo ago

Everyone wants to be the CEO of their own megacorp managing thousands of AI engineers I guess. Just like microservices, there’s probably a ton of overhead doing things this way vs monolithic / single agent. Certain types of engineers just love over-engineering hugely complex stuff to see it work. Goldberg architecture was already prevalent and bad enough in enterprise before the AI boom.

maxothex 7mo ago

[dead]

aplomb1026 7mo ago

[dead]

bob1029 7mo ago

I find depth to be far more interesting than breadth with these models. Descending into a problem space recursively won't necessarily find the best solution, but it's going to tend to find some solution faster than going wide across a swarm of agents. Theoretically it's exponentially faster to have one symbolically recursive agent than to have any number of parallel agents. I think agent swarm stuff sucks for complex multi-step problems because it's mostly a form of BFS. It never actually gets to a good solution because it's searching too wide and no one can afford to wait for it to strip mine down to something valuable.

gpubridge 7mo ago

[dead]

babblingfish 7mo ago

[flagged]

kaicianflone 7mo ago

We’ve been building exactly this as an open-source ecosystem at consensus-tools. It’s a governance layer for multi-agent systems with a runtime wrapper that intercepts agent decisions before they execute: .consensus(fn, opts). The coordination and consistency problems the paper describes are what the monorepo is designed around. Giving agents auditable stake in decisions. Happy to share more if anyone’s working in this space.

timcobb 7mo ago
Yeah share for sure pls
- kaicianflone 7mo ago
  Sure the core primitive is a runtime wrapper that turns any function into a governed decision point: import { consensus } from "@consensus-tools/wrapper"; const safeSend = consensus(sendEmail, { reviewers: [humanReviewer, aiSafetyReviewer], strategy: { mode: "unanimous" }, hooks: { onBlock: (ctx) => audit.log("blocked", ctx) }, }); await safeSend({ to: "user@example.com", body: "Hello" }); The call to sendEmail doesn't execute until every reviewer votes. Strategy modes handle the consensus logic (unanimous, majority, weighted, etc.), and guards can ALLOW, BLOCK, REWRITE, or escalate to REQUIRE_HUMAN before anything fires. The monorepo has 9 built-in policy types and 7 guard types designed so you can drop governance into an existing agent system without rewriting your orchestration. Repo's at github.com/consensus-tools if you want to poke around.

stainlu 7mo ago

[dead]

causalityltd 7mo ago

Apart from rediscovering all the problems with distributed systems, I think LM teams will also rediscover their own version of the mythical man-month, and very quickly too. There were 3 core insights: adding people makes the project later, communication cost grows as n^2, and time isn't fungible. For agents, maybe the core insight won't hold, and adding a new agent won't necessarily increase dev-time, but the second will be worse, communication cost will grow faster than n^2 because of LLM drift and orchestration overhead. The third doesn't translate cleanly but i'll try: Time isn't fungible for us and assumptions and context, however fragmented, aren't fungible for agents in a team. If they hallucinate at the wrong time, even a little, it could be a equivalent of a human developer doing a side-project during company time. An agent should write an article on it and post it on moltbook: "The Inevitable Agent Drift"

clawbridge 7mo ago

[dead]

dasil003 7mo ago

I've long thought of the analogy as useful for human teams, and it even shows up in the corporate jargon (eg. "I'm blocked", "We need to align", etc). It's surprisingly common for whole branches of a large org to be doing net negative work due to conflicting goals or not realizing some implication that cuts across teams and local contexts. Sometimes these issues are technical but just as often they are pure product or business decisions with no explicit dependency until a lightbulb goes off somewhere. With hand-written code, things generally move slow enough, and there's enough common sense sprinkled across the org chart that things can get uncovered organically. With agent teams, speed increased by several orders of magnitude and common sense is out the window, so I suspect the ceiling on productive use of agents will be far more limited in number than traditional engineering teams, and it will heavily depend on who and how humans are plugged into the right places. One thing I suspect professional researchers underestimate is how much positive output can be produced by a human team with vague or hand-wavy direction, and surprisingly little deep thinking, let alone a robust specification or structure to keep them on track. The reality is any large team regresses to the mean, and it's usually a few savvy people that actually drive outcomes. These people don't necessarily have official authority, just a nose for the right thing. This won't spontaneously emerge from agents (at least until they become a lot more human like in terms of big picture common sense, and dial down the sycophancy to a more "skeptical engineer" level).

calmkeepai 7mo ago

I’ve found an interesting model to think about to be production crews similar to in the television world but potentially be something worth using as one’s mental model of how agents and people working alongside agents should coordinate, rather than basing the simulated team force off the typical office worker framework

TheCowboy 7mo ago
How would you describe that and have you been applying it?