4 ms·
> Okay, I have no evidence of this, but they'd better not be using another AI classifier to try and tell whether that link is safe. Of course they are. Probabl
by godelski 3y ago
> Okay, I have no evidence of this, but they'd better not be using another AI classifier to try and tell whether that link is safe.
Of course they are. Probably a combination. I mean you can't validate every url or domain, which is constantly changing.
But the problem isn't if they use a classifier or not, the problem is fundamentally about communication. Security requires acknowledgement of limitations and understanding what failure points exist (and not all networks are absolutely opaque). Frankly nothing is invulnerable, so this clarity is the critical aspect (as you state).
What I just don't understand (especially being a ML researcher) is why we over sell things. The things we build and tools we make are impressive in of themselves. I mean if I hand you a nice piece of chocolate you'll probably be happy and probably even want more. But if I hand you a nice piece of chocolate and tell you that by eating you'll wake in the morning cured of cancer and the richest man in the world you're going to rightfully be disappointed and/or upset at me no matter how good that chocolate was. Can't we just make really fucking good chocolate without the snake oil? I'm afraid that not calling out the snake oil is going to make the whole chocolate industry crumble, even factories which never added the snake oil. I'm unconvinced people are concerned with existential risks to ML/AI if they're not concerned about this. I'm concerned that it's not AI that'll get us paperclipped, but we'll just do it ourselves.
- eviks 3y agoYou over sell things because it then sells for way more than the proper "impressive tooling" would ever allow
- godelski 3y agoAnd you make a quick buck but then everyone stops buying chocolate and then nobody gets chocolate anymore. It seems like a bad strategy for big companies and it seems like a bad strategy for the community (including big tech) to allow others to do it. We don't need ML to go the way of crypto, especially considering it has higher utility. But I guess tech literally thrives on hype so that's why everyone is complacent.
- delusional 3y agoSam Bankman Fried had a good take on this when he was in conversation with Matt Levine before FTX collapsed. You and I might agree that OpenAI is misrepresenting their product, that their real value is significantly lower than they present it, but in which way are we right? We might believe that their technology is worth 100 million dollars, but if they can convince Microsoft that it's worth 10 billion and get Microsoft to give them 1 billion in exchange for 10%, then they now have 1 billion dollars. The value of the technology doesn't change, it might be 100 million or it may be 0, but you don't get any money for assessing that value correctly. In a very real sense the economic system we have created means that the value of a thing is whatever people are willing to pay. If you can hype it up, it actually increases the REAL value of the thing.
- godelski 3y ago> In a very real sense the economic system we have created means that the value of a thing is whatever people are willing to pay. If you can hype it up, it actually increases the REAL value of the thing. Look, I understand econ 101. That's not the part I don't understand. Hell, my cat understands that. It's the other part I don't understand. We're humans and have the ability to be more intelligent than my cat.
- danShumway 3y ago> But the problem isn't if they use a classifier or not I am sure that OpenAI would say that this is very important to support as a feature, but I kind of disagree with this. I think it is a problem for them to use a classifier, because this problem is possible to solve without an opaque classifier, it's solveable with CORS. OpenAI could just not load remote images. There just are better ways to do this. I can't believe I'm going to bat for ChatGPT Addons, which also appear to be security disasters, but the addons do exist, and so for scenarios where remote images really do need to be fetched, extensions could provide that with more safety. Having approved widgets/plugins in the chat for stuff like previews would not completely solve the problem, like you mention vulnerabilities and attacks would remain. But it would help a lot. To your point of communication, it's wild to me that OpenAI would start with a classifier instead of starting with a button on top of every image that says "click to load" and shows the URL. That communicates way more information to the user, it's simpler to build, it's more consistent and avoids the problem of images that are safe randomly refusing to load. And I keep going back to... I know OpenAI would say there are a lot of use-cases for being able to fetch random images. It's not that use-cases don't exist at all. But I just don't buy it, I don't see how the benefits outweigh the downsides. I feel like most other platforms have figured this out: if an attacker can read data from a site and craft custom payloads, you set strict CORS and you don't allow them to send that data anywhere. OpenAI is treating this like a unique situation and doing something weird because they don't like the obvious solutions. LLM security is a nightmare that might not be possible to solve, and you're absolutely correct that for many of those security risks, imperfect safeguards and user consent/communication is likely the best we're going to get. But in this one very specific area, we have lots of solutions for data exfiltration that at the very least significantly reduce the attack surface. > Of course they are. I don't know, I don't take this as a given, it wouldn't surprise me if they're doing something at least slightly better like looking at caches. But I guess it also wouldn't surprise me a huge amount if they're doing something nonsensical like asking GPT "is this URL safe", which is just not a good approach to take. I'm not accusing them of that, I assume (hope) that they're not doing that. It just wouldn't necessarily surprise me. > I'm concerned that it's not AI that'll get us paperclipped, but we'll just do it ourselves. This is a very separate conversation and normally I wouldn't touch on it, but... opinion me, yeah, agreed. I am not worried about existential risks from AI, I'm worried about companies ruining everything because they're not willing to engage responsibly with any this or to even just think critically about where the real use-cases are vs where they're using a screwdriver to hammer in a nail.