4 ms·

The point is that the human brain is just as vulnerable to the adversarial attacks that people are claiming will be the downfall of ML. It's like the people tha

by saulrh 9y ago

The point is that the human brain is just as vulnerable to the adversarial attacks that people are claiming will be the downfall of ML. It's like the people that scream about people shining lasers into the cameras of autonomous vehicles - yeah, sure, if you did the same thing to a human driver you'd get exactly the same crash, why are you suddenly worried about this? And it's more than just "making it look like a dog". If you look at it for any length of time it's obvious that it's a picture of a cat with some noise on it. But if you only have 100ms to respond to it you are guaranteed (>95% misidentification) to say that it's a cat. Now what happens if you take this and make your car look like a tree? Someone could crash and that would be terrible! Cancel driving, tear up the roads, ban wheels, cars were a mistake! /s

matte_black 9y ago
It’s not the same. Shining lasers into eyes is not scalable. However, placing a little sign at the side of a road that causes self driving vehicles to swerve out and crash violently, killing the occupants, can be done in mass.
- saulrh 9y ago
  If you make the sign red and octagonal you could even kill human drivers too! Or maybe if you slapped an illusory-motion pattern over something that looked like a dog, then people might swerve into the opposite lane to avoid it, double kill! Shock, horror! Human vision is slightly more robust solely because it's had more time to go back and forth with adversaries. Nothing prevents ML from reaching the same levels of safety. Nothing prevents you from deploying attacks against humans that're identical to to the attacks against artificial systems.
- thisacctforreal 9y ago
  Further, laser beaming eyes isn't an intellectual pursuit; entering an arms race with some of the world's top engineers certainly is. I imagine most self-driving car hackers will react to any successes with "holy shit it worked" followed by remorse.
ouid 9y ago
The argument that there exists SOME mask which will transform a cat into a dog for humans and computers is not relevant to the discussion. The anatomical features of the face in this picture have changed. The reason adversarial attacks against ML are scary is very simply that you can accomplish them robustly in ways that humans won't detect simply by looking at the images. Obviously I can manipulate a picture of a cat to make it look like a dog.
wzdd 9y ago
> The point is that the human brain is just as vulnerable to the adversarial attacks that people are claiming will be the downfall of ML That may be the point, but it's not proven by this study. Cats and dogs have many structural similarities, as do the other adversarial examples (panda / gibbon, cabbage / broccoli). We know that this isn't just an artefact of the human visual system because we know they are very similar in non-visual ways too, i.e. they are genetically and behaviourally similar (compared with random other items in the world, such as bananas and toasters). This study's choice of images seems to acknowledge that when the human visual system makes mistakes, it does so in a far more robust way than ML-generated models do. Even the fact that this effect is robust across many individuals, versus adversarial ML images being model-specific, demonstrates this. Mistaking a cat for a dog, given a 50ms window, is much less likely to be disadvantageous to us than mistaking a cat for a computer, or a banana for a toaster. In other words, this study is a long way away from demonstrating that applying a bit of static to an image could make us mistake a car for a tree, whereas in an ML scenario such a mistake seems quite plausible.
mannykannot 9y ago
The issue is not the risk of adversarial images being used to create mayhem, which was originally a straw man of the 'vehicle automation is infeasible' faction, but is becoming something of a straw man when it is used as the primary target of the 'this is not a problem' side. The issue is firstly the fragility of current image recognition in ways that are not easily bounded, and secondly the fact that humans use aspects of generalized intelligence in recognizing such situations and dealing with them, something that is not yet feasible to the same level in vehicle automation.

by saulrh 9y ago

matte_black 9y ago

It’s not the same. Shining lasers into eyes is not scalable. However, placing a little sign at the side of a road that causes self driving vehicles to swerve out and crash violently, killing the occupants, can be done in mass.

saulrh 9y ago
If you make the sign red and octagonal you could even kill human drivers too! Or maybe if you slapped an illusory-motion pattern over something that looked like a dog, then people might swerve into the opposite lane to avoid it, double kill! Shock, horror! Human vision is slightly more robust solely because it's had more time to go back and forth with adversaries. Nothing prevents ML from reaching the same levels of safety. Nothing prevents you from deploying attacks against humans that're identical to to the attacks against artificial systems.
thisacctforreal 9y ago
Further, laser beaming eyes isn't an intellectual pursuit; entering an arms race with some of the world's top engineers certainly is. I imagine most self-driving car hackers will react to any successes with "holy shit it worked" followed by remorse.

ouid 9y ago

The argument that there exists SOME mask which will transform a cat into a dog for humans and computers is not relevant to the discussion. The anatomical features of the face in this picture have changed. The reason adversarial attacks against ML are scary is very simply that you can accomplish them robustly in ways that humans won't detect simply by looking at the images. Obviously I can manipulate a picture of a cat to make it look like a dog.

wzdd 9y ago

> The point is that the human brain is just as vulnerable to the adversarial attacks that people are claiming will be the downfall of ML That may be the point, but it's not proven by this study. Cats and dogs have many structural similarities, as do the other adversarial examples (panda / gibbon, cabbage / broccoli). We know that this isn't just an artefact of the human visual system because we know they are very similar in non-visual ways too, i.e. they are genetically and behaviourally similar (compared with random other items in the world, such as bananas and toasters). This study's choice of images seems to acknowledge that when the human visual system makes mistakes, it does so in a far more robust way than ML-generated models do. Even the fact that this effect is robust across many individuals, versus adversarial ML images being model-specific, demonstrates this. Mistaking a cat for a dog, given a 50ms window, is much less likely to be disadvantageous to us than mistaking a cat for a computer, or a banana for a toaster. In other words, this study is a long way away from demonstrating that applying a bit of static to an image could make us mistake a car for a tree, whereas in an ML scenario such a mistake seems quite plausible.

mannykannot 9y ago

The issue is not the risk of adversarial images being used to create mayhem, which was originally a straw man of the 'vehicle automation is infeasible' faction, but is becoming something of a straw man when it is used as the primary target of the 'this is not a problem' side. The issue is firstly the fragility of current image recognition in ways that are not easily bounded, and secondly the fact that humans use aspects of generalized intelligence in recognizing such situations and dealing with them, something that is not yet feasible to the same level in vehicle automation.