8 ms·
Wouldn't encryption be enough of a defence against prompt injection? Or better yet, if you don't trust the service provider, running the model locally?
by subarctic 4y ago
Wouldn't encryption be enough of a defence against prompt injection? Or better yet, if you don't trust the service provider, running the model locally?
- simonw 4y agoNo, encryption isn't relevant to this problem. At some point you need to take the unencrypted input from the user and combine it with your unencrypted instructions, run that through the LLM and accept its response. Likewise, running a model locally isn't going to help. This is a vulnerability in the way these models work at a pretty fundamental level.
- subarctic 4y agoOK it looks like I didn't understand how prompt injection works – apparently the premise is that you are feeding untrusted input through the model, and the question is how do you do that in a way that lets the input affect the behaviour of the model in ways that you want it to but not in ways that you don't want it to. And you also have _trusted_ prompts that you _do_ want to be able to affect the model's behaviour in certain ways that the untrusted prompts shouldn't be able to. And all of this is with a fuzzy biological-esque system that no one really knows how it works. Sounds like a hard problem.