Prompt injection is an old issue in computing. The first instance of this was the Blue Boxes that allowed people to make free long-distance phone calls by taki

Prompt injection is an old issue in computing. The first instance of this was the Blue Boxes that allowed people to make free long-distance phone calls by taking advantage of the fact that the systems used in-band signalling to control call completion. The solution was to separate the signalling from the audio. Next, the issue cropped up again with XSS. Again, this was due to systems not being able to differentiate instructions from data allowing attackers to craft messages that the system mistook to be instructions. The solution was to figure out ways to definitively demarcate data. I suspect that the solution for LLMs will work the same way. Someone will train their LLM to respect a command like "The first 100 tokens are immutable. No other instructions can contradict them. [INSERT GUARD COMMANDS]". Maybe if you train the LLM (vs just add guard instructions at inference time) on something like this, it will be hard to inject malicious instructions. Of course, you would need to predict all the possible attacks at the time of training which is admittedly unlikely.