3 ms·

> The victim uploads a file to Rovo that contains a hidden prompt injection Yeah this attack is possible on all modern agentic systems. * Access to your priva

by hahahaa 2mo ago

> The victim uploads a file to Rovo that contains a hidden prompt injection Yeah this attack is possible on all modern agentic systems. * Access to your private data * Exposure to untrusted content * The ability to externally communicate in a way that could be used to steal your data (https://simonw.substack.com/p/the-lethal-trifecta-for-ai-agents https://simonw.substack.com/p/the-lethal-trifecta-for-ai-age...) And blocking it wholesale reduces usefulness of the agent so it is a tradeoff.

gherkinnn 2mo ago
It's been over a year and the trifecta holds true. The term 'prompt injection' has been round since May 2022 [0]. No amount of "but the future will fix it" has fixed it. These breaches will continue for as long as nobody cares about security and everybody is infected with the LLM brainworm. 0 - https://en.wikipedia.org/wiki/Prompt_injection https://en.wikipedia.org/wiki/Prompt_injection
- pixl97 2mo ago
  Prompt injection is as fixable in LLMs as it is in people. There is no such thing as out of band data. For example it's quite common for large businesses to fall for billing fraud scams when something shows up and says "Hey, it's the CEO, pay this bill to X for $Y". And honestly when you start looking at agentic systems that uses it's previous step to take future steps. The system has to some idea what you want to permit and don't want to permit as everyone could have different expectations here.
  - gherkinnn 2mo ago
    > "Hey, it's the CEO, pay this bill to X for $Y" Sure. Now imagine this very scenario not limited by humans and scaling the way machines do. And then let us consider the current reality that an agentic system on the receiving end may have unlimited access because to too was vibed.
- hahahaa 2mo ago
  My only warning with the trifecta is it sufficient but not necessary for an LLM attack. All you need is the untrusted content (or more generally vulnerable content since how you feel about the content doesn't matter) leg and something you care about to be attacked. E.g. could be as simple as a prompt injection that causes your LLM to output a prompt injection that then gets inserted somewhere else. Or without exfiltration rm -rf / Or a social engineering attack. So there are other bifectas and trifectas.

by hahahaa 2mo ago

gherkinnn 2mo ago

It's been over a year and the trifecta holds true. The term 'prompt injection' has been round since May 2022 [0]. No amount of "but the future will fix it" has fixed it. These breaches will continue for as long as nobody cares about security and everybody is infected with the LLM brainworm. 0 - https://en.wikipedia.org/wiki/Prompt_injection https://en.wikipedia.org/wiki/Prompt_injection

pixl97 2mo ago
Prompt injection is as fixable in LLMs as it is in people. There is no such thing as out of band data. For example it's quite common for large businesses to fall for billing fraud scams when something shows up and says "Hey, it's the CEO, pay this bill to X for $Y". And honestly when you start looking at agentic systems that uses it's previous step to take future steps. The system has to some idea what you want to permit and don't want to permit as everyone could have different expectations here.
- gherkinnn 2mo ago
  > "Hey, it's the CEO, pay this bill to X for $Y" Sure. Now imagine this very scenario not limited by humans and scaling the way machines do. And then let us consider the current reality that an agentic system on the receiving end may have unlimited access because to too was vibed.
hahahaa 2mo ago
My only warning with the trifecta is it sufficient but not necessary for an LLM attack. All you need is the untrusted content (or more generally vulnerable content since how you feel about the content doesn't matter) leg and something you care about to be attacked. E.g. could be as simple as a prompt injection that causes your LLM to output a prompt injection that then gets inserted somewhere else. Or without exfiltration rm -rf / Or a social engineering attack. So there are other bifectas and trifectas.