3 ms·
isn't the solution to train a model to detect instructions in text and reject the request before passing it to the llm?
by itake 3y ago
isn't the solution to train a model to detect instructions in text and reject the request before passing it to the llm?
- phillipcarter 3y agoAnd how do you protect against jailbreaking that model? More elaboration here: https://simonwillison.net/2023/May/2/prompt-injection-explained/#prompt-injection.013 https://simonwillison.net/2023/May/2/prompt-injection-explai...
- simonw 3y agoPlenty of people have tried that approach, none have been able to prove that it's robust against all future attack variants. Imagine how much trouble we would be in if our only protection against SQL injection was some statistical model that might fail to protect us in the future.