3 ms·
I do generally agree with this. From what I'm reading from researchers there is a growing consensus that (for lack of a better term) "context hijacking", "phish
by danShumway 4y ago
I do generally agree with this. From what I'm reading from researchers there is a growing consensus that (for lack of a better term) "context hijacking", "phishing", "tricking", "reprogramming"... whatever you want to call it if you don't like the term prompt injection -- that it may be an unsolvable problem. Certainly, it's not solvable the same way that SQL injection is solvable.
And I don't think your concern about how people interpret the phrase "prompt injection" is unwarranted, I have myself had at least one argument already on HN with someone literally saying that prompt injection is solvable the same way that SQL injection is solvable and we just need to escape input. So the confusion is there, you're completely right about that.
But I don't know a better term to use that people already understand.
I've kind of shifted away from talking about whether prompt injection is solvable towards just trying to get people to understand that it's a problem in the first place. Because you can see a lot of replies here to your own comments on this thread -- it encourages people to immediately start arguing about whether or not it will get solved, when my beef is more that regardless of whether or not it can be solved, it's irresponsible right now for companies to be treating it like it's no big deal.
I'm a little worried that "live with it" will for many businesses translate to "we're allowed to ignore this and it will be someone else's problem" -- part of the reason why I push back so hard on people comparing prompt injection to human attacks is that I see that used very often as an excuse for why we don't need to worry about prompt injection. That's not what you're saying, but it's also an argument I've gotten into on this site; essentially people saying, "well humans are also vulnerable, so why can't an LLM manage my bank account? Why does this need to be mitigated at all?"