3 ms·
> Would love feedback - what MCP security issues have you seen? For me the number one problem with MCP security is the lethal trifecta - the fact that it's so
by simonw 1y ago
> Would love feedback - what MCP security issues have you seen?
For me the number one problem with MCP security is the lethal trifecta - the fact that it's so easy to combine MCPs from different vendors (or even from the same vectors) that provide exposure to potentially untrusted/malicious instructions in a way that can then trigger exfiltration of private data.
https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
https://simonwillison.net/2025/Aug/9/bay-area-ai/ https://simonwillison.net/2025/Aug/9/bay-area-ai/
I don't know how we can solve this with more technology - it seems to me to be baked into the very concept of how MCP works.
- tptacek 1y agoI'm going to pick a fight on this one; I think you know I'm a fan, so take this in the spirit I intend†. My contention is that "lethal trifecta" is the AI equivalent of self-XSS. It's not apparent yet, because all this stuff is just months old, but a year from now we'll be floored by the fact that people just aimed Cursor or Claude Code at a prod database. To my lights, the core security issue with tool/function calling in agents isn't MCP; it's context hygiene. Because people aren't writing their own agents, they're convinced that the single-visible-context-window idiom of tools like Cursor are just how these systems work. But a context is just a list of strings. You can have as any of them in an agent as you want. Once you've got untrusted data hitting one context window, and sensitive tool calls isolated in another context window, the problem of securing the system isn't much different than it is with a traditional web application; some deterministic code that a human reviewed and pentested mediates between those contexts, transforming untrusted inputs into trustable commands for the sensitive context. That's not a trivial task, but it's the same task as we do now when, for instance, we need to generate a PDF invoice in an invoicing application. Pentesters find vulnerabilities in those apps! But it's not a news story when it happens, so much. † More a note for other people who might thing I'm being irritable. :)
- simonw 1y agoI think the core of the whole problem is that if you have an LLM with access to tools and exposure to untrusted input, you should consider the author of that untrusted input to be have total control over the execution of those tools. MCP is just a widely agreed upon abstraction over hooking an LLM up to some tools. A significant potion of things people want to do with LLMs and with tools in general involve tasks where a malicious attacker taking control of those tools is a bad situation. Is that what you mean by context hygiene? That end users need to assume that anything bad in the context can trigger unwanted actions, just like you shouldn't blindly copy and paste terminal commands from a web page into your shell (cough, curl https://.../install.sh | sh) or random chunks of JavaScript into the Firefox devtools console on Facebook.com ?
- tptacek 1y agoOn the first two paragraphs: we agree. (I just think that's both more obvious and less fundamental to the model than current writing on this suggests). On the latter two paragraphs: my point is that there's nothing fundamental to the concept of an agent that requires you to mix untrusted content with sensitive tool calls. You can confine untrusted content to its own context window, and confine sensitive tool calls to "sandboxed" context windows; you can feed raw context from both to a third context window to summarize or synthesize; etc.
- simonw 1y agoRight - that's more or less the idea behind https://simonwillison.net/2023/Apr/25/dual-llm-pattern/ https://simonwillison.net/2023/Apr/25/dual-llm-pattern/ and the DeepMind CaMeL paper: https://simonwillison.net/2025/Apr/11/camel/ https://simonwillison.net/2025/Apr/11/camel/ The challenge is that you have to implement really good taint tracking (as seen in old school Perl) - you need to make sure that the output of a model that was exposed to untrusted data never gets fed into some other model that has access potentially harmful tool calls. I think that is possible to build, but I haven't seen any convincing implementation of the pattern yet. Hopefully soon!
- tptacek 1y ago