3 ms·
Case in point for my reply to Simon on why I draw this distinction :) The natural instinct for many people about prompt injection is to assume that contextual
by danShumway 3y ago
Case in point for my reply to Simon on why I draw this distinction :)
The natural instinct for many people about prompt injection is to assume that contextual separation between data and commands is enough to solve the problem. The short answer is:
A) we don't know how to enforce that separation with current LLMs.
B) Even if we did, there is another category of attack (or Simon might argue if I understand his position, a sub-category of prompt injection) that can not be defended against by separating untrusted input and system/user prompts.