3 ms·
You can't solve prompt injection, because it's not a bug - it's a feature. You want AIs with capabilities approximating reasoning? Then don't be surprised they
by TeMPOraL 4y ago
You can't solve prompt injection, because it's not a bug - it's a feature. You want AIs with capabilities approximating reasoning? Then don't be surprised they can be talked out of whatever it is you ordered them.
Just like humans.
Evil bits and magic delimiters won't stop a problem that boils down to making the model reinterpret its hidden prompt in context of the whole conversation.
See https://news.ycombinator.com/item?id=35780833 https://news.ycombinator.com/item?id=35780833 for a larger discussion, including specifically https://news.ycombinator.com/item?id=35781172 https://news.ycombinator.com/item?id=35781172.