2 ms·
ooh this is a dense and useful paper. i like that they took the time to apply it to a bunch of case studies and its all in 30 pages. i think basically all of t
by swyx 1y ago
ooh this is a dense and useful paper. i like that they took the time to apply it to a bunch of case studies and its all in 30 pages.
i think basically all of them involve reducing the "agency" of the agents though - which is a fine tradeoff - but i think one should be aware that the Big Model folks dont try to engineer any of these and just collect data to keep reducing injection risk. the tradeoff of capability maxxing vs efficiency/security often tends to be won by the capabilitymaxxers in terms of product adoption/marketing.
eg the SWE Agent case study recommends Dual LLM with strict data formatting - would like to see this benchmarked in terms of how much of a perfomance an agent like this would be, perhaps doable by forking openai codex and implementing the dual llm.
- simonw 1y agoYeah, this paper is refreshingly conservative and practical: it takes the position that robust protection against prompt injection requires very painful trade-offs: These patterns impose intentional constraints on agents, explicitly limiting their ability to perform arbitrary tasks. That's a bucket of cold water in a lot of things people are trying to build. I imagine a lot of people will ignore this advice!
- NoMoreNicksLeft 1y agoLLMs are too useful to allow the commoner access to them. The question remains, how best to fleece those commoners with perceived utility while providing them none?
- theHolyTrynity 1y agoyes agree most hype around agents is around stuff that ignore these patterns
- hooverd 1y agoWhat if we could define what a computer could do via some symbolic notation? Perhaps program it in some kind of language?