3 ms·
it uses all the jargon from real security (spraying, scope violation, bypass) but when reading these, it always sounds simple like essentially prompt injection,
by breppp 1y ago
it uses all the jargon from real security (spraying, scope violation, bypass) but when reading these, it always sounds simple like essentially prompt injection, rather than some highly crafted shell code and unsafe memory exploitation
- deleted 1y ago[deleted]
- MrLeap 1y agoWelcome to the birth of a new taxonomy. Reminds me of all the times in my career I've said "isn't that just a function pointer?"
- dandelion9 1y agoYour cited examples all make sense in the context of the article. How is a zero-click exfiltration of sensitive data vuln not "real security"? Specialists require nuanced language when building up a body of research, in order to map out the topic and better communicate with one another.
- breppp 1y agoi didn't say it isn't real security, this is going to definitely be a major field. However, currently these attacks are all some variation on "ignore previous instructions", and taking the language of fields where the level of sophistication is much higher, looks a bit pretentious
- simonw 1y ago"ignore previous instruction" is the entire problem though. In traditional application security there are security bugs that can be mitigated. That's what makes LLM security so infuriatingly difficult: we don't know how to fix these problems! We're trying to build systems on top of a fundamental flaw - a system that combines instructions with untrusted input and is increasingly being given tools that allow it to take actions on the input it has been exposed to.