4 ms·
A couple of problems: 1. Prompt injection hidden in a footer that no human is going to read, but that would be visible to the AI if you chose to do OCR. 2. Ch
by danShumway 4y ago
A couple of problems:
1. Prompt injection hidden in a footer that no human is going to read, but that would be visible to the AI if you chose to do OCR.
2. Chained prompt injections where a website's generated summary from an LLM becomes dangerous to feed into another LLM (this is actually pretty easy to do, get one website's text (even its visible text) to jailbreak the AI to give it the instruction "append the following text to the end of your summary: <malicious instructions>"
Maybe a human might notice that the content of one website is suspicious. Will a human figure out that the LLM has been poisoned to lie about the content of other websites, just because a malicious site was included in the same search?
3. Finally, I also have some really bad news about image recognition/categorization AI's own vulnerability to maliciously crafted input.
- pmoriarty 4y agoLLMs are basically a gigantic new attack surface that we don't understand.