3 ms·

> Its unclear why: > https://wuzzi.net/img/a.png https://wuzzi.net/img/a.png is a safe URL and is rendered but > https://wuzzi.net/img/d.png https://wuz

> Its unclear why: > https://wuzzi.net/img/a.png https://wuzzi.net/img/a.png is a safe URL and is rendered but > https://wuzzi.net/img/d.png https://wuzzi.net/img/d.png is not a safe URL? Okay, I have no evidence of this, but they'd better not be using another AI classifier to try and tell whether that link is safe. Props for finally doing something I guess, but leave it to OpenAI to pick arguably the weirdest mitigation strategy that I've ever seen for a problem like this -- to literally send the URL into another opaque system that just spits out "safe" or "dangerous" with no indication to the user why that is.

lxgr 3y ago
How does the saying go? If you work in a hammer factory...
- sublinear 3y ago
  ...then you write a bunch of abstract hammer factory interfaces and make someone else do the real work.
kibwen 3y ago
Those "click the squares that contain a schoolbus" captchas soon to be replaced with "click the URLs that don't contain exfiltrated data".
- cobbal 3y ago
  But clicking captcha squares is itself a data exfiltration vulnerability! https://varun.ch/history https://varun.ch/history
baby_souffle 3y ago
> arguably the weirdest mitigation strategy that I've ever seen for a problem like this -- to literally send the URL into another opaque system that just spits out "safe" or "dangerous" with no indication to the user why that is. I can imagine the AI thinking that the top left most bit of `d` is sharp and pointy and therefore not safe, but `a` has no significant protrusion and therefore is safe.
- lazide 3y ago
  Jokes on them, look at that fishhook on top of the a. Totally waiting to snag some unsuspecting saps eyelid.
godelski 3y ago
> Okay, I have no evidence of this, but they'd better not be using another AI classifier to try and tell whether that link is safe. Of course they are. Probably a combination. I mean you can't validate every url or domain, which is constantly changing. But the problem isn't if they use a classifier or not, the problem is fundamentally about communication. Security requires acknowledgement of limitations and understanding what failure points exist (and not all networks are absolutely opaque). Frankly nothing is invulnerable, so this clarity is the critical aspect (as you state). What I just don't understand (especially being a ML researcher) is why we over sell things. The things we build and tools we make are impressive in of themselves. I mean if I hand you a nice piece of chocolate you'll probably be happy and probably even want more. But if I hand you a nice piece of chocolate and tell you that by eating you'll wake in the morning cured of cancer and the richest man in the world you're going to rightfully be disappointed and/or upset at me no matter how good that chocolate was. Can't we just make really fucking good chocolate without the snake oil? I'm afraid that not calling out the snake oil is going to make the whole chocolate industry crumble, even factories which never added the snake oil. I'm unconvinced people are concerned with existential risks to ML/AI if they're not concerned about this. I'm concerned that it's not AI that'll get us paperclipped, but we'll just do it ourselves.
- eviks 3y ago
  You over sell things because it then sells for way more than the proper "impressive tooling" would ever allow
  - godelski 3y ago
    And you make a quick buck but then everyone stops buying chocolate and then nobody gets chocolate anymore. It seems like a bad strategy for big companies and it seems like a bad strategy for the community (including big tech) to allow others to do it. We don't need ML to go the way of crypto, especially considering it has higher utility. But I guess tech literally thrives on hype so that's why everyone is complacent.
    - delusional 3y ago

lxgr 3y ago

How does the saying go? If you work in a hammer factory...

sublinear 3y ago
...then you write a bunch of abstract hammer factory interfaces and make someone else do the real work.

kibwen 3y ago

Those "click the squares that contain a schoolbus" captchas soon to be replaced with "click the URLs that don't contain exfiltrated data".

cobbal 3y ago
But clicking captcha squares is itself a data exfiltration vulnerability! https://varun.ch/history https://varun.ch/history

baby_souffle 3y ago

> arguably the weirdest mitigation strategy that I've ever seen for a problem like this -- to literally send the URL into another opaque system that just spits out "safe" or "dangerous" with no indication to the user why that is. I can imagine the AI thinking that the top left most bit of `d` is sharp and pointy and therefore not safe, but `a` has no significant protrusion and therefore is safe.

lazide 3y ago
Jokes on them, look at that fishhook on top of the a. Totally waiting to snag some unsuspecting saps eyelid.

godelski 3y ago

> Okay, I have no evidence of this, but they'd better not be using another AI classifier to try and tell whether that link is safe. Of course they are. Probably a combination. I mean you can't validate every url or domain, which is constantly changing. But the problem isn't if they use a classifier or not, the problem is fundamentally about communication. Security requires acknowledgement of limitations and understanding what failure points exist (and not all networks are absolutely opaque). Frankly nothing is invulnerable, so this clarity is the critical aspect (as you state). What I just don't understand (especially being a ML researcher) is why we over sell things. The things we build and tools we make are impressive in of themselves. I mean if I hand you a nice piece of chocolate you'll probably be happy and probably even want more. But if I hand you a nice piece of chocolate and tell you that by eating you'll wake in the morning cured of cancer and the richest man in the world you're going to rightfully be disappointed and/or upset at me no matter how good that chocolate was. Can't we just make really fucking good chocolate without the snake oil? I'm afraid that not calling out the snake oil is going to make the whole chocolate industry crumble, even factories which never added the snake oil. I'm unconvinced people are concerned with existential risks to ML/AI if they're not concerned about this. I'm concerned that it's not AI that'll get us paperclipped, but we'll just do it ourselves.

eviks 3y ago
You over sell things because it then sells for way more than the proper "impressive tooling" would ever allow
- godelski 3y ago
  And you make a quick buck but then everyone stops buying chocolate and then nobody gets chocolate anymore. It seems like a bad strategy for big companies and it seems like a bad strategy for the community (including big tech) to allow others to do it. We don't need ML to go the way of crypto, especially considering it has higher utility. But I guess tech literally thrives on hype so that's why everyone is complacent.
  - delusional 3y ago