3 ms·
This feels like the modern incarnation of "packet intent", the mythical security property of knowing what an incoming request is trying to do rather than what i
by tptacek 7mo ago
This feels like the modern incarnation of "packet intent", the mythical security property of knowing what an incoming request is trying to do rather than what it is. Variants of "packet intent" have been sought after going all the way back into the 1980s; it's helpful to recognize the idea when it appears, because it's a reliable marker of what you can't realistically accomplish.
- Legend2440 7mo agoExcept agents actually have an intent, and can route around obstacles to accomplish that intent. If you merely block a specific action, they will find another way to do what they're trying to do. Agent security requires controlling the agent's intent.
- sigbottle 7mo agoThe goal behind most "clean" software design in general is to eliminate the possibility of failure via constraints. That's the pattern I've seen over the years. Of course, the map is not the territory - you need to make sure the reachable set within the constraints is actually a subset of the real reachable set. Which may be underspecified or unknown a priori (as if you could've really specified the true reachable set, why didn't you just encode those rules?) So I'm sympathetic to the criticism, especially since composition of formal methods & analyzing their effects is still very much a hard problem (and not just computationally - philosophically, often, for the reason I listed above). That being said, I don't know a better solution. Begging the agent with prompts doesn't work. Are you suggesting some kind of mechanistic interpretability, maybe?
- deleted 7mo ago[deleted]