3 ms·

Has anyone experimented with having a second LLM that is laser focused on stopping prompt injection? It would probably be small, cheap and fast relative to the

Has anyone experimented with having a second LLM that is laser focused on stopping prompt injection? It would probably be small, cheap and fast relative to the main LLM.

simonw 4y ago
That's a really common suggestion for this problem - using AI to try to detect prompt injection attacks. I don't trust it at all. It seems clear to me that someone will eventually figure out a prompt injection attack that subverts the "outer layer" first - there was an example of that in my very first piece about prompt injection here: https://simonwillison.net/2022/Sep/12/prompt-injection/#more-ai https://simonwillison.net/2022/Sep/12/prompt-injection/#more... I wrote more about this here: https://simonwillison.net/2022/Sep/17/prompt-injection-more-ai/ https://simonwillison.net/2022/Sep/17/prompt-injection-more-...