3 ms·

i didn't say it isn't real security, this is going to definitely be a major field. However, currently these attacks are all some variation on "ignore previous

by breppp 1y ago

i didn't say it isn't real security, this is going to definitely be a major field. However, currently these attacks are all some variation on "ignore previous instructions", and taking the language of fields where the level of sophistication is much higher, looks a bit pretentious

simonw 1y ago
"ignore previous instruction" is the entire problem though. In traditional application security there are security bugs that can be mitigated. That's what makes LLM security so infuriatingly difficult: we don't know how to fix these problems! We're trying to build systems on top of a fundamental flaw - a system that combines instructions with untrusted input and is increasingly being given tools that allow it to take actions on the input it has been exposed to.