3 ms·
LLMs can't distinguish between input that is instructions from a trusted source and input that is text that they are meant to be operating on. The way you "pro
by simonw 3y ago
LLMs can't distinguish between input that is instructions from a trusted source and input that is text that they are meant to be operating on.
The way you "program" a LLM is you feed it prompts like this one:
Summarize this email: <text of email>
Anyone who understands SQL injection should instantly spot why that's a problem.
I've written a lot about prompt injection over the past year. I suggest https://simonwillison.net/2023/May/2/prompt-injection-explained/ https://simonwillison.net/2023/May/2/prompt-injection-explai... - I have a whole series here https://simonwillison.net/series/prompt-injection/ https://simonwillison.net/series/prompt-injection/