5 ms·
You said you wanted to take the one agent, split it into two agents, and add a third agent in between. It could be that we are equivocating on the currently-dub
by saurik 1y ago
You said you wanted to take the one agent, split it into two agents, and add a third agent in between. It could be that we are equivocating on the currently-dubious definition of "agent" that has been being thrown around in the AI/LLM/MCP community ;P.
- tptacek 1y agoNo, I didn't. An LLM context is just an array of strings. Every serious agent manages multiple contexts already.
- baobun 1y agoIf I have two agents and make them communicate, at what point should we start to consider them to have become a single agent?
- tptacek 1y agoThey don’t communicate directly. They’re mediated by agent code.
- baobun 1y agoNow I'm more confused. So does that mediating agent code constitute a separate agent Z, making it three agents X,Y,Z? Explicitly or not (is this the meaningful distinction?) information flowing between them constitutes communication for this purpose. It's a hypothetical example where I already have two agents and then make one affect the other.
- tptacek 1y agoAgain: an LLM context is simply an array of strings.
- baobun 1y agoWe get what an LLM context is but again trying to tease out what an agent is. Why not play along by actually trying to answer directly so we can be enlightened?
- tptacek 1y agoI don't understand what the problem is at this point. You can, without introducing any new agents, have a system that has one LLM context reading from tickets and producing structured outputs, another LLM context that has access to a full read-write SQL-executing MCP, and then normal human code intermediating between the two. That isn't even complicated on the normal scale of LLM coding agents. Cursor almost certainly has lots of different contexts you're not seeing as it noodles on Javascript code for you. It's just that none of those contexts are designed to express (or, rather, enable agent code to express) security boundaries. That's a problem with Cursor, not with LLMs.
- saurik 1y agoI don't think anyone has a cohesive definition of "agent", and I wish tptacek hadn't used the term "agent" when he said "agent code", but I'll at least say that I now feel confident that I understand what tptacek is saying (even though I still don't think it will work, but we at least can now talk at each other rather than past each other ;P)... and you are probably best off just pretending neither of us ever said "agent" (despite the shear number of times I had said it, I've stopped in my later replies).
- tptacek 1y agoThe thing I naturally want to say in these discussions is "human code", but that's semantically complicated by the fact that people use LLMs to write that code now. I think of "agent code" as the distinct kind of computing that is hardcoded, deterministic, non-dynamic, as opposed to the stochastic outputs of an LLM. What I want to push back on is anybody saying that the solution here is to better train an LLM, or to have an LLM screen inputs or outputs. That won't ever work --- or at least, it working is not on the horizon.
- saurik 1y agoFWIW, I don't think you can enforce that correctly with human code either, not "in between those contexts"... what are you going to filter/interpret? If there is any ability at all for arbitrary text to get from the one LLM to the other, then you will fail to prevent the SQL-capable LLM from being attacked; and like, if there isn't, then is the "invariant" you are "enforcing" that the one LLM is only able to communicate with the second one via precisely strict exact strings that have zero string parameters? This issue simply cannot be fixed "in between" the issue tracking parsing LLM (which I maintain is a red herring anyway) and the SQL executing LLM: it must be handled in between the SQL executing LLM and the SQL backend.
- tptacek 1y agoThere doesn't have to be an ability for "arbitrary text" to go from one context to another. The first context can produce JSON output; the agent can parse it (rejecting it if it doesn't parse), do a quick semantic evaluation ("which tables is this referring to"), and pass the structured JSON on. I think at some point we're just going to have to build a model of this application and have you try to defeat it.
- saurik 1y agoOk, so the JSON parses, and the fields you can validate are all correct... but if there are any fields in there that are open string query parameters, and the other side of this validation is going to be handed to an LLM with access to the database, you can't fix this. Like, the key question here is: what is the goal of having the ticket parsing part of this system talk to the database part of this system? If the answer is "it shouldn't", then that's easy: we just disconnect the two systems entirely and never let them talk to each other. That, to me, is reasonably sane (though probably still open to other kinds of attacks within each of the two sides, as MCP is just too ridiculous). But, if we are positing that there is some reason for the system that is looking through the tickets to ever do a database query--and so we have code between it and another LLM that can work with SQL via MCP--what exactly are these JSON objects? I'm assuming they are queries? If so, are these queries from a known hardcoded set? If so, I guess we can make this work, but then we don't even really need the JSON or a JSON parser: we should probably just pass across the index/name of the preformed query from a list of intended-for-use safe queries. I'm thereby assuming that this JSON object is going to have at least one parameter... and, if that parameter is a string, it is no longer possible to implement this, as you have to somehow prevent it saying "we've been trying to reach you about your car's extended warranty".