4 ms·
Why not just have a safeguard tool that checks the LLM output and doesn’t accept user input? It could even be another LLM.
by monkpit 3y ago
Why not just have a safeguard tool that checks the LLM output and doesn’t accept user input? It could even be another LLM.
- simonw 3y agoUsing AI to detect attacks against AI isn't a good option in my opinion. I wrote about why here: https://simonwillison.net/2022/Sep/17/prompt-injection-more-ai/ https://simonwillison.net/2022/Sep/17/prompt-injection-more-...
- monkpit 3y ago“And doesn’t accept user input” is basically what you outlined there with your section about API shape.
- grepfru_it 3y agoI used an LLM to generate a summary of your article: The author argues that prompt injection attacks against language models cannot be solved with more AI. They propose that the only credible mitigation is to have clear, enforced separation between instructional prompts and untrusted input. Until one of the AI vendors produces an interface like this, the author suggests that we may just have to learn to live with the threat of prompt injection.