3 ms·
These attacks are more closely related to social engineering the LLM, rather then traditional "injections". https://embracethered.com/blog/posts/2023/ai-inject
by kerng 3y ago
These attacks are more closely related to social engineering the LLM, rather then traditional "injections".
https://embracethered.com/blog/posts/2023/ai-injections-direct-and-indirect-prompt-injection-basics/ https://embracethered.com/blog/posts/2023/ai-injections-dire...
There aren't any specific limited amount of tokens to inject or mitigate against, there is an "infinite" amount of trickery the AI might misinterpret or be persuaded to do.
Annual security training will be needed for AI, to learn about the latest phishing attacks, much like for humans. Only have joking.
- PeterisP 3y agoThere indeed is a strong overlap with social engineering, but in my view the whole reason why social engineering the LLM is possible is an "injection vulnerability". We don't want the LLM to treat third-party data in the same way as the communication with the user. We want the user to be able to talk with an LLM-based chatbot in arbitrary ways and issue arbitrary instructions, however, we also want a strict separation between these instructions and the data they operate on, so that when the user says "fix style problems in that blob of text" the model has the capability to tell that this blob of text is fundamentally different from the instructions, and that literally nothing in it should even theoretically enable social engineering.