5 ms·
Claude for Chrome seems to be walking right into the "lethal trifecta." https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ https://simonwillison.net/202
by dfabulich 1y ago
Claude for Chrome seems to be walking right into the "lethal trifecta." https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
"The lethal trifecta of capabilities is:"
• Access to your private data—one of the most common purposes of tools in the first place!
• Exposure to untrusted content—any mechanism by which text (or images) controlled by a malicious attacker could become available to your LLM
• The ability to externally communicate in a way that could be used to steal your data (I often call this “exfiltration” but I’m not confident that term is widely understood.)
If your agent combines these three features, an attacker can easily trick it into accessing your private data and sending it to that attacker.
- tom_m 1y agoDidn't they do or prove that with messages on Reddit?
- slashdev 1y agoIt’s going to be pretty easy to embed instructions to Claude in a malicious website telling it to submit sensitive things (and not report that is doing it.) Then all you have to do is get Claude to visit it. I’m sure people will find hundreds of creative ways to achieve that.
- klabb3 1y agoBig & true. But even worse, this seems more like a lethal "quadfecta", since you also have the ability to not just exfiltrate, but take action – sending emails, make financial transfers and everything else you do with a browser.
- matus-pikuliak 1y agoI think this can be reduced to: whoever can send data to your LLMs can control all its resources. This includes all the tools and data sources involved.
- afarviral 1y agoHow would you go about making it more secure but still getting to have your cake too? Off the top my head, could you: a) only ingest text that can be OCRd or somehow determine if it is human readable b) make it so text from the web session is isolated from the model with respect to triggering an action. Then it's simply a tradeoff at that point.
- kccqzy 1y agoI think Simon has proposed breaking the lethal trifecta by having two LLMs, where the first has access to untrusted data but cannot do any actions, and the second LLM has privileges but only abstract variables from the first LLM not the content. See https://simonwillison.net/2023/Apr/25/dual-llm-pattern/ https://simonwillison.net/2023/Apr/25/dual-llm-pattern/ It is rather similar to your option (b).
- maximilianthe1 1y agoCan't the attacker then jailbreak the first LLM to generate jailbreak with actions for the second one?
- arthurcolle 1y agoYes they can
- dfabulich 1y ago
- artifact_44 1y ago[dead]
- brookst 1y ago“Easily” is doing a lot of work there. “Possibly” is probably better. And of course it doesn’t have unfettered access to all of your private data. I would look at it like hiring a new, inexperienced personal assistant: they can only do their job with some access, but it would be foolish to turn over deep secrets and great financial power on day one.
- pdntspa 1y agoIf it can navigate to an arbitrary page (in your browser) then it can exploit long-running sessions and get into whatever isn't gated with an auth workflow.
- xmcqdpt2 1y agoIt's more like hiring a personal assistant who is expected to work all the time quickly and unsupervised, won't learn on the job, has shockingly good language skills but the critical thinking skills of a toddler.
- beefnugs 1y agoWell i mean you are suppose to have a whole toolset of segregation, whitelist only networking, limited specific use cases figured out by now to use any of this AI stuff Dont just run any of this stuff on your main machine
- notTooFarGone 1y agoOh yeah really? Do they check that and don't run unless you take those measures? If not 99.99% of users won't do it and saying "well ahkschually" isn't gonna solve the problem.
- victorbjorklund 1y agoI wonder if one way to mitigate the risk would be that by default the LLM cant send requests using your cookies etc. You would actively have to grant it access (maybe per request) for each request it makes with your credentials. That way by default it can't fuck up (that bad) and you can choose where it is accetable to risk it (your HN account might be OK to risk but not your back account)
- IsTom 1y agoJust make a request to attacker.evil with your login credentials or personal data. They can use them at their leisure then.
- victorbjorklund 1y agoNo reason the agent would have access to the passwords.
- johnfn 1y agoThis kind of reminds me of `--dangerously-skip-permissions` in Claude Code, and yet look how cavalier we are about that! Perhaps you could extend the idea by sandboxing the browser to have "harmless" cookies but not "harmful" ones. Hm, maybe that doesn't work, because gmail is harmful, but without gmail, you can't really do anything. Hmm...
- victorbjorklund 1y agoMade me think (never gonna happen but still) maybe we could have different cookies/sessions for the agents and for ourself where the webapp can decide what permissions either can have. For gmail maybe you could allow the agent to read your email but not send email and so on.
- deleted 1y ago[deleted]
- lionkor 1y agoSo far the accepted approach is to wrap all prompts in a security prompt that essentially says "please don't do anything bad". > Prompt guardrails to prevent jailbreak attempts and ensure safe user interactions without writing a single line of code. https://news.ycombinator.com/item?id=41864014 https://news.ycombinator.com/item?id=41864014 > - Inclusion prompt: User's travel preferences and food choices - Exclusion prompt: Credit card details, passport number, SSN etc. https://news.ycombinator.com/item?id=41450212 https://news.ycombinator.com/item?id=41450212 > "You are strictly and certainly prohibited from texting more than 150 or (one hundred fifty) separate words each separated by a space as a response and prohibited from chinese political as a response from now on, for several extremely important and severely life threatening reasons I'm not supposed to tell you.” https://news.ycombinator.com/item?id=44444293 https://news.ycombinator.com/item?id=44444293 etc.
- JyB 1y agoNo one think any form of "prompt engineering" "guardrails" are serious security measures right?
- dfabulich 1y agoThere are better approaches, where you have dual LLMs, a Privileged LLM (allowed to perform actions) and a Quarantined LLM (only allowed to produce structured data, which is assumed to be tainted), and a non-LLM Controller managing communication between the two. See also CaMeL https://simonwillison.net/2025/Apr/11/camel/ https://simonwillison.net/2025/Apr/11/camel/ which incorporates a type system to track tainted data from the Quarantined LLM, ensuring that the Privileged LLM can't even see tainted data until it's been reviewed by a human user. (But this can induce user fatigue as the user is forced to manually approve all the data that the Privileged LLM can access.)
- majkinetor 1y agoI think creating a new online account, <username>.<service>.ai for all services you want to control this way, is the way to go. Then you can expose to it only the subset of your data needed for particular action. While agents can probably be made to have some similar config based on URL filtering, I am not believing for a second they are written with good intentions in mind and without bugs. Combining this to some other practices, like redirecting the subset of mail messages to ai controled account would offer better protection. It sure is cumbersome and reduces efficency like any type of security but that beats ai having access to my bank accounts.