3 ms·
This is exactly why I think it's so important that we separate jailbreaking from prompt injection. Jailbreaking is mainly about stopping the model saying somet
by simonw 3y ago
This is exactly why I think it's so important that we separate jailbreaking from prompt injection.
Jailbreaking is mainly about stopping the model saying something that would look embarrassing in a screenshot.
Prompt injection is about making sure your "personal digital assistant" doesn't forward copies of your password reset emails to any stranger who emails it and asks for them.
Jailbreaking is mostly a PR problem. Prompt injection is a security problem. Security problems are worth solving!
- theptip 3y agoIsn’t jailbreaking a strict superset of prompt injection? I would assume the agent instructions would include “don’t share the user’s docs” and so you need to jailbreak to actually succeed with prompt injection these days? Maybe just an overlapping set?
- simonw 3y agoI see them as overlapping. Protections against jailbreaking are often but not always relevant to prompt injection.
- cjonas 3y agoIf that scenario exists, is not a problem with the LLM, but with the fundamental application architecture... That's the equivalent of an API that allows the client to pass a user ID without auth check