4 ms·
I acknowledge there are fair points in all the replies. I'm not an avid user of LLM systems. Only explored a bit their capabilities. Looks like we're at the ear
by amne 3y ago
I acknowledge there are fair points in all the replies. I'm not an avid user of LLM systems. Only explored a bit their capabilities. Looks like we're at the early stages when good / best practices of prompt isolation are yet to emerge.
To explain a bit better my point of view: I believe it will come down to something along the lines of "addslashes" applied to every prompt an LLM interprets. Which is why I reduced it to "an LLM can solve this problem". If you reflect on what "addslashes" does is it applies code to remove or mitigate special characters affecting execution of later code. In the same way I think LLM itself can self-sanitize its inputs in such a way that it cannot be escaped. If you agree that there's no character you can input that can remove an added slash then there should be a prompt equivalent of "addslashes" such that there's no way you can state an instruction that it can escape the wrapping "addslashes" that will mitigate prompt injection.
I did not think this all the way to the end in terms of impact on system usability but it should still be capable of performing most tasks but stay within bounds of intended usage.
- simonw 3y agoThis is the problem with prompt injection: the obvious fixes, like escaping ala addslashes or splitting the prompt into an "instructions" section and a "data" section genuinely don't work. We've tried them all. I wrote a lot more about this here: https://simonwillison.net/series/prompt-injection/ https://simonwillison.net/series/prompt-injection/