6 ms·
Whenever this sort of issue comes up, there are a number of responses pointing out that humans can be fooled too, and I am guessing that the generally unstated
by mannykannot 9y ago
Whenever this sort of issue comes up, there are a number of responses pointing out that humans can be fooled too, and I am guessing that the generally unstated implication is that therefore it is not a big deal. Two important differences that this point of view overlooks are that humans are usually able to tell when they are not fully understanding their visual input, and they are able to respond appropriately, which includes acting cautiously and taking actions that will help resolve the uncertainty. Artificial visual systems that have, at best, only a rudimentary understanding of what they are looking at, are in no position to act this way, and can assign high confidence values to what we would regard as ludicrous interpretations of the scene.
These things will be fixed in time. There is nothing to be gained by pretending that they are not problems.
- visarga 9y ago> Artificial visual systems that have, at best, only a rudimentary understanding of what they are looking at, are in no position to act this way, and can assign high confidence values to what we would regard as ludicrous interpretations of the scene. It's not a good idea to generalize like that. There are many AI systems that use neural nets for perception and then use different models, such as conditional random fields (CRF) to see if entities make sense in context. Not to mention that these adversarial examples don't work well from all distances and angles. As a car changes position slightly, the adversarial example disappears. It's very hard to create an adversarial example that works from all angles on all networks. There is also a special training regimen that fortifies neural nets against adversarial examples. The problem of adversarial images is closely related to GANs and has been brought into attention by the same researcher, Ian Goodfellow. So it's not a neglected problem - some of the top minds are on it.
- dogma1138 9y agoThe adversarial examples don't need to work from all distances and angles. Many street signs especially in rural areas might have a pretty small visibility window and they will only be visible from a pretty fixed angle until you pass them. So while you might not be able to develop an adversarial example that works for all signs from all distances and angles it wouldn't be hard to develop one for a specific sign. The angle for that sign is known, the distance in which most cars could resolve the sign is also known, the average speed on the road can also be taken into account which gives you a rough idea of the time span the autonomous driver would have to identify and resolve the sign. Heck if you know the algorithm it wouldn't be that hard to build an app that basically allows you to take a picture of the sign with your phone from the road, select a car or subset of cars you want to jam and it would generate a likely pattern to apply to the sign to jam those cars. I should also add that in this case some of the examples that do seem to work look to be of the shape and size of common bumper stickers, this is pretty worrisome since these are not that uncommon on street signs, especially in rural areas where they are more or less at reachable heights. I also wonder how well do these systems deal with graffiti.
- ethbro 9y agoWait until AI puts most truckers out of a job and you suddenly have a substantial number of people in rural America feeling resentful towards fully automated freight trucks... adversarial examples are one thing, but I think the US military's experience in Iraq has taught us that a determined, intelligent, and constantly innovating human adversary is a continuing challenge. Especially given that the cost of crashing an unmanned truck will be 100% capital and 0% human, I expect we'll start seeing some pretty serious criminal penalties passed once the tech gets there.
- mannykannot 9y ago> So it's not a neglected problem - some of the top minds are on it. That's why I believe it will be fixed. It is the "it's not a problem" crowd that I take issue with, and I would be surprised if many of the top researchers held that position, unless they are confident that they are close to a solution. I think you are under-generalizing here. The risk here is not of a sign being mistaken for a gorilla specifically, or even about signs specifically, but the fragility of image recognition in general. Similarly, because the issue is not (just) about adversarial images, the fragility of adversarial images is not much of a mitigating factor.
- ajmurmann 9y agoI think another big issue is that humans can tell when a sign is manipulated or damaged to the point where it will be problematic to identify for other humans and it now can be corrected before something goes wrong. If it's only misleading to computers, chances are humans won't fix it till something went wrong.
- rsync 9y ago"These things will be fixed in time." ... and the solutions will be written in blood.
- civilitty 9y agoAs opposed to bunnies and in kittens like they are now?
- skybrian 9y agoYes, as happened with trains, automobiles, and airplanes. But I suspect that driverless cars will start out pretty safe and improve from there.
- rsync 9y ago"Yes, as happened with trains, automobiles, and airplanes." I think there is an interesting distinction that is worth discussing ... At the advent of train/plane travel, you could opt out of those novel risks by eschewing those modes of travel. However, you could be driving along in your driver-piloted car, minding your own business and find yourself involved in an accident caused by the software in a driverless car you are sharing the road with.
- resu_nimda 9y agoI don't think that is an important distinction because that is still very true of driver-piloted vehicles as well. Also people still die in train and plane accidents. You still have to eschew any form of travel to be safe from its risks.
- skybrian 9y agoConsider that trains cannot stop quickly or turn and safety consists almost entirely of making sure people get out of the way. And yet, lots of people (even kids) would walk along tracks, etc. And yet there are still urban areas with active train tracks running along surface streets, for example near Jack London Square in Oakland. Before cars, people routinely walked in the middle of streets. "Jaywalking" wasn't a concept. It was entirely up to early cars to stop in time, and they didn't have good brakes. Fortunately they weren't very fast. Also, early pilots, besides the obvious risks, were sometimes daredevils who would do things like buzz houses. I think the main point here is that society's attitude towards risk has changed; we take small risks much more seriously than they used to when earlier forms of transportation were introduced. Accidents involving driverless cars make the national news and manufacturers are being very cautious. With earlier attitudes towards risk, I expect they'd be in widespread use by now.
- mrtksn 9y agoAt the point when the AI operated cars can handle all kind of situations better than humans, they will be intelligent enough to be classified as human beings.
- DannyBee 9y agoThey aren't real problems because nobody smart is trying to directly hook up a road sign classifier to a steering wheel. They are trying to build complex systems where this is just one signal. If they were trying to directly use this info, this would be the least of the serious issues. For example, there are plenty of stop signs well hidden by trees. Such a car would probably last about a mile in a non freeway setting. A trivial model of "hey, does this make any sense appearing at this place on a map" defeats this, etc. Of course this stuff is it is infancy and can do dumb things. But this is just isn't that big a deal. Humans also act much worse than you imply. Look at the number of people who accelerate into crashes instead of braking, etc. Again, doesn't mean AI should get a pass, and humans are infinitely more complex, but we shouldn't be held up as amazing at this either. (That's actually the part that worries me. That we aren't good at it)
- mannykannot 9y agoThe fragility of AI decision-making in general absolutely is an issue. Feel free to post a submission when it has been solved to a level that matches or exceeds human capabilities.
- DannyBee 9y agoFeel free to actually provide a real counterpoint to my argument instead of "well, you're wrong, so there".
- dboreham 9y agoLet's start with Spam : filtering it out with 100% guarantee of no false positives has still not been achieved.
- MR4D 9y agoBut also true of humans. In fact, I would guess I that advanced spam filtering engines might be _better_ than the average human. Remember, the Nigerian scammers are still around because someone is dumb enough to still give them money.
- microcolonel 9y agoWell, given the example in the article, any human with functioning vision would be able to identify that stop sign. A driverless car with inadequate detection would run through the intersection without stopping. I'd say that's a pretty big deal. I think the problem with the driver-less car is that it doesn't have goal-oriented general perception like human beings do. Humans filter the world through goals. The world is interpreted as a set of affordances and threats to your goals, your brain is predisposed with distinguishing the two. As it stands, this is not how driver-less cars are being designed, they are being designed to specifically recognize the presence and content of street features like signs and other moving bodies. A human driver might see a tall parked car with a big gap in front, and recognize a pedestrian affordance which he should steer clear of in case somebody pops out. The way I see people talk about driver-less cars, this "common sense" type of judgement would have to be deliberately integrated.
- IanCal 9y agoIs the important thing that humans and machines classify them differently? Or that one or both might be fooled? Because the latter already exists as a problem but doesn't seem to be a major issue. You can already, with physical access to a sign, change what it looks like to fool any observer.
- pishpash 9y agoI think you are overstating what "understanding" is. It is well known that birds see more of the EM spectrum than humans. We can make -- and indeed there are -- designs that are only visible to birds but are invisible to humans in the sense that our three-cone visual system conflates certain inputs from the physical world. That stop sign can be designed in a way to look like a stop sign to a bird but make us think it is something else entirely. Are we not understanding the stop sign? No. It's just a fact of parsimonious signal representation that sometimes you get null spaces with which you can manipulate to maximally separate the performance of two very different receivers.
- aetherson 9y agoNo, that's nothing like analogous. Adversarial examples aren't giving machines information that is invisible to humans.
- Houshalter 9y agoThe fact is that these adversarial examples are incredibly rare. 1 in a trillion trillion or more chance that they would occur in natural data. We dont know that human brains aren't vulnerable to something similar. I bet if we could back propagate through the visual cortex, we would find similar things. They can also be mitigated a bit by training on them.
- kthejoker2 9y agoOptical illusions certainly suggest that our visual processing involves trade-offs.