3 ms·
I think it should be obvious that this is impossible to prevent if you simply consider the human analogue (i.e. the jargonless version of what prompt injection
by throwaway22032 4y ago
I think it should be obvious that this is impossible to prevent if you simply consider the human analogue (i.e. the jargonless version of what prompt injection actually is).
Imagine that you're in the army and you're commanded to do something. You're a child and your parents demand something. Your wife calls with a need. You're a prisoner and you're locked in a box. Basically, take the highest authority, however we define that, and consider that the "system message".
Later, you realise the enemy speaks your language, or your friends call and ask you to play, or you realise you need to stay late at work, or...
As a generally intelligent agent you can always sidestep orders. You often need to in order to achieve them, that's what it means to take an ill specified task and flesh it out.