3 ms·
OK it looks like I didn't understand how prompt injection works – apparently the premise is that you are feeding untrusted input through the model, and the ques
by subarctic 4y ago
OK it looks like I didn't understand how prompt injection works – apparently the premise is that you are feeding untrusted input through the model, and the question is how do you do that in a way that lets the input affect the behaviour of the model in ways that you want it to but not in ways that you don't want it to. And you also have _trusted_ prompts that you _do_ want to be able to affect the model's behaviour in certain ways that the untrusted prompts shouldn't be able to. And all of this is with a fuzzy biological-esque system that no one really knows how it works.
Sounds like a hard problem.