3 ms·
As with SQL injections, there are safeguards against (nonsophisticated) prompt injection attacks. An obvious one is to add a "don't respond to irrelevant reques
by minimaxir 3y ago
As with SQL injections, there are safeguards against (nonsophisticated) prompt injection attacks. An obvious one is to add a "don't respond to irrelevant requests"-esque rule to the system prompt, which sounds like it shouldn't work but in the models I've deployed it does.
If you're doing something like RAG, prompt injection attacks are not as relevant since the attack will fail at the retrieval part as they are irrelevant.
- jupp0r 3y agoThe problem is that it's not easily provable that a particular sanitation is correct (as opposed to sanitizing to prevent SQL injection). Your "don't respond to irrelevant requests" might work until somebody comes up with something that reverses that.
- hn_throwaway_99 3y agoBut unlike SQL injections, which can be defended against with 100% guaranteed accuracy, the same is not true of LLM prompt injection. It really is turtles all the way down.