Your last post got me looking into the theory behind prompt injection and one discussion I saw was talking about the difference between 1) getting the agent to

Your last post got me looking into the theory behind prompt injection and one discussion I saw was talking about the difference between 1) getting the agent to pretend that it is something and respond as that something and 2) getting it to imagine something and give the response it would expect that thing to give. To use the example from the article, telling GPT-4 that it should imagine a pirate and tell you what that pirate says would likely yield different results than telling GPT-4 to pretend it's a pirate and say stuff. I suspect that has more to do with the fact that initial prompt injections were more the "pretend you are" stuff so models were trained against that more than the "imagine a thing" stuff. Hard to say but it's interesting.