4 ms·
It's been over a year and the trifecta holds true. The term 'prompt injection' has been round since May 2022 [0]. No amount of "but the future will fix it" has
by gherkinnn 2mo ago
It's been over a year and the trifecta holds true. The term 'prompt injection' has been round since May 2022 [0]. No amount of "but the future will fix it" has fixed it.
These breaches will continue for as long as nobody cares about security and everybody is infected with the LLM brainworm.
0 - https://en.wikipedia.org/wiki/Prompt_injection https://en.wikipedia.org/wiki/Prompt_injection
- pixl97 2mo agoPrompt injection is as fixable in LLMs as it is in people. There is no such thing as out of band data. For example it's quite common for large businesses to fall for billing fraud scams when something shows up and says "Hey, it's the CEO, pay this bill to X for $Y". And honestly when you start looking at agentic systems that uses it's previous step to take future steps. The system has to some idea what you want to permit and don't want to permit as everyone could have different expectations here.
- gherkinnn 2mo ago> "Hey, it's the CEO, pay this bill to X for $Y" Sure. Now imagine this very scenario not limited by humans and scaling the way machines do. And then let us consider the current reality that an agentic system on the receiving end may have unlimited access because to too was vibed.
- hahahaa 2mo agoMy only warning with the trifecta is it sufficient but not necessary for an LLM attack. All you need is the untrusted content (or more generally vulnerable content since how you feel about the content doesn't matter) leg and something you care about to be attacked. E.g. could be as simple as a prompt injection that causes your LLM to output a prompt injection that then gets inserted somewhere else. Or without exfiltration rm -rf / Or a social engineering attack. So there are other bifectas and trifectas.