4 ms·
I don't know where "more agents" is coming from.
by tptacek 1y ago
I don't know where "more agents" is coming from.
- lotyrin 1y agoSeems they can't imagine the constraints being implemented as code a human wrote so they're just imagining you're adding another LLM to try to enforce them?
- saurik 1y ago(EDIT: THIS WAS WRONG.) [[FWIW, I definitely can imagine that (and even described multiple ways of doing that in a lightweight manner: pattern whitelisting and fine-grained permissions); but, that isn't what everyone has been calling an "agent" (aka, an LLM that is able to autonomously use tools, usually, as of recent, via MCP)? My best guess is that the use of "agent code" didn't mean the same version of "agent" that I've been seeing people use recently ;P.]] EDIT TO CORRECT: Actually, no, you're right: I can't imagine that! The pattern whitelisting doesn't work between two LLMs (vs. between an LLM and SQL, where I put it; I got confused in the process of reinterpreting "agent") as you can still smuggle information (unless the queries are entirely fully baked, which seems to me like it would be nonsensical). You really need a human in the loop, full stop. (If tptacek disagrees, he should respond to the question asked by the people--jstummbillig and stuart73547373--who wanted more information on how his idea would work, concretely, so we can check whether it still would be subject to the same problem.) NOT PART OF EDIT: Regardless, even if tptacek meant adding trustable human code between those two LLM+MCP agents, the more important part of my comment is that the issue tracking part is a red herring anyway: the LLM context/agent/thing that has access to the Supabase database is already too dangerous to exist as is, because it is already subject to occasionally seeing user data (and accidentally interpreting it as instructions).
- lotyrin 1y agoI actually agree with you, to be clear. I do not trust these things to make any unsupervised action, ever, even absent user-controlled input to throw wrenches into their "thinking". They simply hallucinate too much. Like... we used to be an industry that saw value in ECC memory because a one-in-a-million bit flip was too much risk, that understood you couldn't represent arbitrary precision numbers as floating point, and now we're handing over the keys to black boxes that literally cannot be trusted?
- tptacek 1y agoIt's fine if you want to talk about other bugs that can exist; I'm not litigating that. I'm talking about foreclosing on this bug.
- baobun 1y agoI guess this part > there should be one LLM context that is reading tickets, and another LLM context that can drive MCP SQL calls, and then agent code in between those contexts to enforce invariants. I get the impression that saurik views the LLM contexts as multiple agents and you view the glue code (or the whole system) as one agent. I think both of youses points are valid so far even if you have semantic mismatch on "what's the boundary of an agent". (Personally I hope to not have to form a strong opinion on this one and think we can get the same ideas across with less ambiguous terminology)
- saurik 1y agoYou said you wanted to take the one agent, split it into two agents, and add a third agent in between. It could be that we are equivocating on the currently-dubious definition of "agent" that has been being thrown around in the AI/LLM/MCP community ;P.
- tptacek 1y agoNo, I didn't. An LLM context is just an array of strings. Every serious agent manages multiple contexts already.
- baobun 1y agoIf I have two agents and make them communicate, at what point should we start to consider them to have become a single agent?
- tptacek 1y agoThey don’t communicate directly. They’re mediated by agent code.
- baobun 1y agoNow I'm more confused. So does that mediating agent code constitute a separate agent Z, making it three agents X,Y,Z? Explicitly or not (is this the meaningful distinction?) information flowing between them constitutes communication for this purpose. It's a hypothetical example where I already have two agents and then make one affect the other.