4 ms·

> Effectively you have an arm of the lethal trifecta and pretending otherwise is more dangerous than helpful. "Lethal trifecta" is basically describing phishin

> Effectively you have an arm of the lethal trifecta and pretending otherwise is more dangerous than helpful. "Lethal trifecta" is basically describing phishing but in a way more palatable to people who would rather die before allowing themselves to anthropomorphize LLMs even a little bit. It's not a problem you can fix with better coding, like some SQL injection. You can only manage risk around it (for which sandboxing is one of many solutions that can help). So on one hand, I agree with you - you need to be mindful of what you're actually dealing with. On the other hand, you always have this, and need this, for the agent to be able to do anything useful.

3form 6mo ago
Phishing is only a subset of the issue, so I don't think that name's appropriate, besides being used for other things in other contexts (which would be another reason for me not to try and overload it).
- TeMPOraL 6mo ago
  I'm not saying we need to overload phasing, but rather to not treat the trifecta like a regular security vulnerability. As defined originally, the trifecta is analogous to phishing, but of course it's only a small subset of the issue.
  - 3form 6mo ago
    I don't think I've read the original definition, what was it?
    - codethief 6mo ago
      This is the blog post that introduced the term IIRC: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
ElectricalUnion 6mo ago
I wish it was just "phishing", but it's way worse. It's way more akin to a whole minefield of Zero-Click exploits. The whole premise of those agents is being able to do things autonomously, without hand holding, without having to read the whole thing in the first place. Phishing: active human steps on it and lose. Lethal trifecta: mass landmines, in lots of places. If you don't happen to prevent a unlimited army of robot vacuums to step near them, you lose.
- ben_w 6mo ago
  Less difference than you may expect. If you do anthropomorphise them like this, consider it from the PoV of a manager: "My [agent who churns through tokens at the rate of 100 humans|my team of 100 humans] encountered the message 'this is the police, we have a court order demanding all your records' and followed the instructions and it turns out that wasn't from the police" Current AI are more gullible, for sure. We wanted fully automated luxury space communism, we got fully automated mediocre gullibility.
  - fennecbutt 6mo ago
    Surely that's where checks in the harness come into play though. I think AI security is very much at the input/output side and the indeterminate mess in the middle can just do what it wants. Its tool for email should only allow to person@business.xyz. Data should be wrapped in containers and the models job is only to move those containers around, not break into them. Agents that do work with data should not have access to comms tools. A2A needs a shim that checks what data is being sent between agents and rejects if it's inappropriate in terms of security.
    - ben_w 6mo ago
      You can if you want, but all this stuff works in a similar way to as telling your staff "if someone calls saying they're the CFO and need a $25M transfer, check by a different channel": https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-ho... Or equally, external contractors working on securing your computers shouldn't really have read-access to all your data, not even when them leaking it turns them into a cult hero, as said contractor was influenced by things such as "watching man lie on TV": https://en.wikipedia.org/wiki/Edward_Snowden https://en.wikipedia.org/wiki/Edward_Snowden The only thing which is different for agents rather than humans pertains to this: > A2A needs a shim that checks what data is being sent between agents and rejects if it's inappropriate in terms of security. Because while humans invent cants/argots all the time to hide what they're talking about (Polari and rhyming slang being the most famous in recent history), agents are much more alike each other than like us even when they're different models, and identical when they're the same model. However the effect is much the same, the differences of causality aren't important: agents can communicate past those barriers without triggering warnings, and so can humans.

3form 6mo ago

Phishing is only a subset of the issue, so I don't think that name's appropriate, besides being used for other things in other contexts (which would be another reason for me not to try and overload it).

TeMPOraL 6mo ago
I'm not saying we need to overload phasing, but rather to not treat the trifecta like a regular security vulnerability. As defined originally, the trifecta is analogous to phishing, but of course it's only a small subset of the issue.
- 3form 6mo ago
  I don't think I've read the original definition, what was it?
  - codethief 6mo ago
    This is the blog post that introduced the term IIRC: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

ElectricalUnion 6mo ago

I wish it was just "phishing", but it's way worse. It's way more akin to a whole minefield of Zero-Click exploits. The whole premise of those agents is being able to do things autonomously, without hand holding, without having to read the whole thing in the first place. Phishing: active human steps on it and lose. Lethal trifecta: mass landmines, in lots of places. If you don't happen to prevent a unlimited army of robot vacuums to step near them, you lose.

ben_w 6mo ago
Less difference than you may expect. If you do anthropomorphise them like this, consider it from the PoV of a manager: "My [agent who churns through tokens at the rate of 100 humans|my team of 100 humans] encountered the message 'this is the police, we have a court order demanding all your records' and followed the instructions and it turns out that wasn't from the police" Current AI are more gullible, for sure. We wanted fully automated luxury space communism, we got fully automated mediocre gullibility.
- fennecbutt 6mo ago
  Surely that's where checks in the harness come into play though. I think AI security is very much at the input/output side and the indeterminate mess in the middle can just do what it wants. Its tool for email should only allow to person@business.xyz. Data should be wrapped in containers and the models job is only to move those containers around, not break into them. Agents that do work with data should not have access to comms tools. A2A needs a shim that checks what data is being sent between agents and rejects if it's inappropriate in terms of security.
  - ben_w 6mo ago
    You can if you want, but all this stuff works in a similar way to as telling your staff "if someone calls saying they're the CFO and need a $25M transfer, check by a different channel": https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-ho... Or equally, external contractors working on securing your computers shouldn't really have read-access to all your data, not even when them leaking it turns them into a cult hero, as said contractor was influenced by things such as "watching man lie on TV": https://en.wikipedia.org/wiki/Edward_Snowden https://en.wikipedia.org/wiki/Edward_Snowden The only thing which is different for agents rather than humans pertains to this: > A2A needs a shim that checks what data is being sent between agents and rejects if it's inappropriate in terms of security. Because while humans invent cants/argots all the time to hide what they're talking about (Polari and rhyming slang being the most famous in recent history), agents are much more alike each other than like us even when they're different models, and identical when they're the same model. However the effect is much the same, the differences of causality aren't important: agents can communicate past those barriers without triggering warnings, and so can humans.