2 ms·
It feels like an LLM classifying the prompts without cumulative context as well as the prompt output from the LLM would be pretty effective. Like in the human m
by fnordpiglet 3y ago
It feels like an LLM classifying the prompts without cumulative context as well as the prompt output from the LLM would be pretty effective. Like in the human mind, with its varying levels of judgement and thought, it may be a case of multiple LLMs watching the overall process.
- simonw 3y agoI wrote about why I don't think that's a good approach here: https://simonwillison.net/2023/May/2/prompt-injection-explained/ https://simonwillison.net/2023/May/2/prompt-injection-explai...
- fnordpiglet 3y agoI think like any security system you work through layers of depth and breadth. 99% security is actually not a failing grade - think about security for a bank vault. It’s layer after layer of protections, each one with a probability of being thwarted. But it’s also possible that you’re detected, which can be enough. None the less people still manage to rob bank vaults through extraordinary measures. In this case you employ classical measures to detect malicious activity, provide sufficient logging and monitoring to detect activity post attack, and layers of measures (input output detection, dual AI as you propose, prompt protections, etc). You still end up with a system that can fail. But the joint probability of all systems failing is very low.