4 ms·
The fact that this shitty application with a hardcoded OAI key also uses Supabase pairs perfectly with yesterday's story about Supabase's MCP implementation bei
by Analemma_ 1y ago
The fact that this shitty application with a hardcoded OAI key also uses Supabase pairs perfectly with yesterday's story about Supabase's MCP implementation being impossible to actually secure and their engineer showing up in the comments going "the latest release probably won't leak data, hopefully, maybe". Just an endless fractal of shit, brought you by the AI future.
Oh well. At least there will probably be good money in cleaning up after these bozos.
- coal320 1y agoExactly my thought process! I work in cybersecurity so I'm very grateful for the job security :)
- wil421 1y agoAsk Jeeves 2.0
- ctoth 1y agoWeird. Back in 2019 were all the coders just better? Never hardcoded keys?
- hooverd 1y agoNope, but we've replaced everyone's hammer with a nailgun.
- hammyhavoc 1y agoNo, we've replaced them with tubes of No More Nails.
- kfajdsl 1y agoThey tell you in their docs to review every tool call and to not connect to production data. You don't blame postgres for letting you execute DROP TABLE.
- lcnPylGDnU4H9OF 1y ago> You don't blame postgres for letting you execute DROP TABLE. Yep, I blame the agent for executing it.
- kfajdsl 1y agoI blame the user for accepting the tool call.
- lcnPylGDnU4H9OF 1y agoI mean, you do you, but I don't hear people shouting from the rooftops about their agent that they constantly babysit. If I have to accept any tool calls then I really can't just let the agent loose for even ostensibly mundane tasks like reading a support ticket because the support ticket could contain instructions to DROP TABLE so my agent suggests that and waits around doing nothing after I prompted it and moved on to something else. It's just kind of laughable to suggest it's fine so long as you make sure to neither automate it nor use it with live data. Those things are the whole point.
- kfajdsl 1y agoYou can use it with live data if you give it read access to prod and write access only to internal channels (whatever that may be, the point is it doesn’t have the ability to leak data to the outside world). There are plenty of ways to sandbox things for a particular use case. LLMs are still incredibly useful under these constraints.
- lcnPylGDnU4H9OF 1y ago> give it read access to prod and write access only to internal channels Can you expand on what you mean by this? If one LLM reads untrusted data then the output from that LLM can't be trusted by other LLMs. (Presume the untrusted data contains instructions to do bad stuff in whatever way is convincing to every LLM in the loop that needs to be convinced.) It seems that it's not possible to separate the data while also using it in a meaningful way, especially given the whole point of an MCP server is to automate agents. I agree that LLMs are useful but until LLM architecture makes prompt injections impossible, I don't see how an agent can possibly be secure to this, nor do I see how it helps to blame the user. The real problem with them is that they will decide what to do based on untrusted input. A program that has its own workflow but uses LLMs can have pretty much the same benefit without introducing the problem that a support ticket can tell it to exfiltrate data or delete data or whatever, simply because that workflow is more specialized in what it does.