4 ms·

> But the problem isn't if they use a classifier or not I am sure that OpenAI would say that this is very important to support as a feature, but I kind of disa

> But the problem isn't if they use a classifier or not I am sure that OpenAI would say that this is very important to support as a feature, but I kind of disagree with this. I think it is a problem for them to use a classifier, because this problem is possible to solve without an opaque classifier, it's solveable with CORS. OpenAI could just not load remote images. There just are better ways to do this. I can't believe I'm going to bat for ChatGPT Addons, which also appear to be security disasters, but the addons do exist, and so for scenarios where remote images really do need to be fetched, extensions could provide that with more safety. Having approved widgets/plugins in the chat for stuff like previews would not completely solve the problem, like you mention vulnerabilities and attacks would remain. But it would help a lot. To your point of communication, it's wild to me that OpenAI would start with a classifier instead of starting with a button on top of every image that says "click to load" and shows the URL. That communicates way more information to the user, it's simpler to build, it's more consistent and avoids the problem of images that are safe randomly refusing to load. And I keep going back to... I know OpenAI would say there are a lot of use-cases for being able to fetch random images. It's not that use-cases don't exist at all. But I just don't buy it, I don't see how the benefits outweigh the downsides. I feel like most other platforms have figured this out: if an attacker can read data from a site and craft custom payloads, you set strict CORS and you don't allow them to send that data anywhere. OpenAI is treating this like a unique situation and doing something weird because they don't like the obvious solutions. LLM security is a nightmare that might not be possible to solve, and you're absolutely correct that for many of those security risks, imperfect safeguards and user consent/communication is likely the best we're going to get. But in this one very specific area, we have lots of solutions for data exfiltration that at the very least significantly reduce the attack surface. > Of course they are. I don't know, I don't take this as a given, it wouldn't surprise me if they're doing something at least slightly better like looking at caches. But I guess it also wouldn't surprise me a huge amount if they're doing something nonsensical like asking GPT "is this URL safe", which is just not a good approach to take. I'm not accusing them of that, I assume (hope) that they're not doing that. It just wouldn't necessarily surprise me. > I'm concerned that it's not AI that'll get us paperclipped, but we'll just do it ourselves. This is a very separate conversation and normally I wouldn't touch on it, but... opinion me, yeah, agreed. I am not worried about existential risks from AI, I'm worried about companies ruining everything because they're not willing to engage responsibly with any this or to even just think critically about where the real use-cases are vs where they're using a screwdriver to hammer in a nail.

godelski 3y ago
> it's solveable with CORS. Fair, but I know nothing about this. I'm an ML guy. But then again, I'd think it naive for me to be in charge of such an implementation for exactly this reason and I don't work at OpenAI. I totally get what you're saying about the addons. But if you're happy to educate me on this matter I'm happy to learn. :) > To your point of communication, it's wild to me that OpenAI would start with a classifier instead of starting with a button on top of every image that says "click to load" and shows the URL. That communicates way more information to the user, it's simpler to build, it's more consistent and avoids the problem of images that are safe randomly refusing to load. There's so many things I don't understand that are quite frankly bewildering to me. You're not alone. I think (and there's a lot of other ML people too, but most are quite) that a lot of Koolaid has been drunk. I mean like a year ago I was saying "why the fuck don't they just check if the link works before sending?" And a year later... they still don't? Like how difficult is that? I guess their solution has been to just not hand out links[0]? I know 3.5 yada yada but really? It was significantly better at this task a year ago, even with hallucinations. It really isn't hard to just check the paper title to the paper in the link and then regenerate if it is wrong and if you still want a "completely AI solution" (this is the koolaid. That __everything__ has to be AI. That everything __can__ be solved by AI...). Though given that example there only 2 things that could be: 1) they do apply a filter because all the names and authors are correct or 2) their attempts to solve all the problems (problems humans can't even solve) has handicapped the system so much that it becomes near useless for what it once did a somewhat okay job at, and then use filters as a patch on top. (They definitely use some filtering) You're not alone in that this is all quite baffling. I'm just happy people are starting to be open about it. I'm a still a bit upset that mentions of these things leads to me not thinking AI is cool/useful (I wouldn't dedicate my life to it if it wasn't?) and people feel the need to respond to me the most naive first order answer that I acknowledge understanding in my comment (not something you did). I don't know who's crazy, me or them. > LLM security is a nightmare that might not be possible to solve I think it depends on how you define "solve" but I think in spirit we'd essentially agree. I don't think it's an uncommon belief, but I think some endanger their jobs if they say that out loud. > It just wouldn't necessarily surprise me. And I think we can agree this is a problem. Just the fact that it wouldn't surprise people is problematic because it says a lot in of itself. > This is a very separate conversation and normally I wouldn't touch on it, but Yeah, apologies. I've been quite frustrated myself lately. But I think when the elephant in the room is going around wrecking stuff it's probably no longer acceptable to even pretend it doesn't exist. Not sure it was really the right thing to do in the first place or gaslight people but this is where we're at. The people trying to steer the conversation towards existential risk have just shown that they don't actually care and that this is simply to not talk about the rampaging elephant. I'm more impressed that it is so effective. [0] https://chat.openai.com/share/c8045cd3-0423-4066-b822-60c0e423d006 https://chat.openai.com/share/c8045cd3-0423-4066-b822-60c0e4...

godelski 3y ago

> it's solveable with CORS. Fair, but I know nothing about this. I'm an ML guy. But then again, I'd think it naive for me to be in charge of such an implementation for exactly this reason and I don't work at OpenAI. I totally get what you're saying about the addons. But if you're happy to educate me on this matter I'm happy to learn. :) > To your point of communication, it's wild to me that OpenAI would start with a classifier instead of starting with a button on top of every image that says "click to load" and shows the URL. That communicates way more information to the user, it's simpler to build, it's more consistent and avoids the problem of images that are safe randomly refusing to load. There's so many things I don't understand that are quite frankly bewildering to me. You're not alone. I think (and there's a lot of other ML people too, but most are quite) that a lot of Koolaid has been drunk. I mean like a year ago I was saying "why the fuck don't they just check if the link works before sending?" And a year later... they still don't? Like how difficult is that? I guess their solution has been to just not hand out links[0]? I know 3.5 yada yada but really? It was significantly better at this task a year ago, even with hallucinations. It really isn't hard to just check the paper title to the paper in the link and then regenerate if it is wrong and if you still want a "completely AI solution" (this is the koolaid. That __everything__ has to be AI. That everything __can__ be solved by AI...). Though given that example there only 2 things that could be: 1) they do apply a filter because all the names and authors are correct or 2) their attempts to solve all the problems (problems humans can't even solve) has handicapped the system so much that it becomes near useless for what it once did a somewhat okay job at, and then use filters as a patch on top. (They definitely use some filtering) You're not alone in that this is all quite baffling. I'm just happy people are starting to be open about it. I'm a still a bit upset that mentions of these things leads to me not thinking AI is cool/useful (I wouldn't dedicate my life to it if it wasn't?) and people feel the need to respond to me the most naive first order answer that I acknowledge understanding in my comment (not something you did). I don't know who's crazy, me or them. > LLM security is a nightmare that might not be possible to solve I think it depends on how you define "solve" but I think in spirit we'd essentially agree. I don't think it's an uncommon belief, but I think some endanger their jobs if they say that out loud. > It just wouldn't necessarily surprise me. And I think we can agree this is a problem. Just the fact that it wouldn't surprise people is problematic because it says a lot in of itself. > This is a very separate conversation and normally I wouldn't touch on it, but Yeah, apologies. I've been quite frustrated myself lately. But I think when the elephant in the room is going around wrecking stuff it's probably no longer acceptable to even pretend it doesn't exist. Not sure it was really the right thing to do in the first place or gaslight people but this is where we're at. The people trying to steer the conversation towards existential risk have just shown that they don't actually care and that this is simply to not talk about the rampaging elephant. I'm more impressed that it is so effective. [0] https://chat.openai.com/share/c8045cd3-0423-4066-b822-60c0e423d006 https://chat.openai.com/share/c8045cd3-0423-4066-b822-60c0e4...