2 ms·
No, that particular attack vector goes away. The attack does not, and is kind of fundamental to how these things currently work.
by MattPalmer1086 2y ago
No, that particular attack vector goes away. The attack does not, and is kind of fundamental to how these things currently work.
- tsimionescu 2y agoThe attack vector is the only relevant thing here. The attack "feeding malicious prompts to an LLM makes it produce malicious output" is a fundamental feature of LLMs, not an attack. It's just as relevant as C's ability to produce malicious effects if you compile and run malicious source code.
- MattPalmer1086 2y agoWell, that is my point. There is an inbuilt vulnerability in these systems as they do not (and apparently cannot) separate data and commands. This is just one vector for this, there will be many, many more.
- red75prime 2y agoLLMs are doing what you train them to do. See for example " The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions " by Eric Wallace et al.
- MattPalmer1086 2y agoInteresting. Doesn't solve the problem entirely but seems to be a viable strategy to mitigate it somewhat.