3 ms·

They aren't real problems because nobody smart is trying to directly hook up a road sign classifier to a steering wheel. They are trying to build complex system

They aren't real problems because nobody smart is trying to directly hook up a road sign classifier to a steering wheel. They are trying to build complex systems where this is just one signal. If they were trying to directly use this info, this would be the least of the serious issues. For example, there are plenty of stop signs well hidden by trees. Such a car would probably last about a mile in a non freeway setting. A trivial model of "hey, does this make any sense appearing at this place on a map" defeats this, etc. Of course this stuff is it is infancy and can do dumb things. But this is just isn't that big a deal. Humans also act much worse than you imply. Look at the number of people who accelerate into crashes instead of braking, etc. Again, doesn't mean AI should get a pass, and humans are infinitely more complex, but we shouldn't be held up as amazing at this either. (That's actually the part that worries me. That we aren't good at it)

mannykannot 9y ago
The fragility of AI decision-making in general absolutely is an issue. Feel free to post a submission when it has been solved to a level that matches or exceeds human capabilities.
- DannyBee 9y ago
  Feel free to actually provide a real counterpoint to my argument instead of "well, you're wrong, so there".
  - dboreham 9y ago
    Let's start with Spam : filtering it out with 100% guarantee of no false positives has still not been achieved.
    - MR4D 9y ago
      But also true of humans. In fact, I would guess I that advanced spam filtering engines might be _better_ than the average human. Remember, the Nigerian scammers are still around because someone is dumb enough to still give them money.
- rm999 9y ago
  I build AI/ML systems for a living and it's not as big of a deal as you think. Production machine learning systems built by experienced teams will almost always have business logic protections built-in to stop the machine from doing something totally stupid. For example, an auto-bidder may have capped velocities or auto-shutoff mechanisms (those examples of bidders spending billions of dollars in minutes are exceedingly sloppy). This is not unlike the human body's deeply-evolved reflexes - before the neocortex can even process that a pan is boiling hot, the nervous system bypasses the brain and and tells the hands to drop it. In the case of driverless cars, DannyBee is absolutely correct: truly driverless cars will not make decisions from a single data point. I have some past experience with unmanned aerial vehicles and those things had insane amounts of redundancy in almost every signal and decision making process; driverless cars are more complex and will come decades later and will probably take this even further.
  - oneshot908 9y ago
    As another such practitioner, why oh why would we ignore hard-learned unambiguous signals and wisdom because we jumped on the machine learning bandwagon? We embraced machine learning to detect and uncover the things we don't see, not to replace the things we already know. There's no conflict here whatsoever.
  - mannykannot 9y ago
    > Production machine learning systems built by experienced teams will almost always have business logic protections built-in to stop the machine from doing something totally stupid. In the case of driving, however, the problem of deciding what behavior to veto is itself a hard problem - perhaps the hardest of all. When the system monitoring the AI is itself AI, the argument becomes circular - which is not to say it cannot be solved, but I think it makes it harder to dismiss the concerns.
samstave 9y ago
Hw difficult would a GPS spoofer be, i.e. to transmit the wrong GPS coords via very cheap devices to at least confuse the heck out of any automated car? Oh look: http://www.rtl-sdr.com/spoofing-gps-locations-with-low-cost-tx-sdrs/ http://www.rtl-sdr.com/spoofing-gps-locations-with-low-cost-...
- DannyBee 9y ago
  Again, this assumes this is the only input, and things can't detect when some inputs seem completely insane with high probability and raise an error Seriously. Y'all realize how much in your daily life would pretty much explode and kill people if that wasn't true, right?
  - samstave 9y ago
    Yeah, but that wasn't my point.. my point is that these attacks will be tried as soon as opportunity>cost-to-attempt
  - pjscott 9y ago
    It's worth mentioning that car GPS navigation explicitly calculates how plausible each GPS reading is, based on a combination of other position signals and basic physics modeling, and discounts anything that sounds too crazy. If it didn't, simple things like tall buildings would be huge problems.
    - kevin_thibedeau 9y ago
      Tall buildings are huge problems for GPS. Multipath reflections will inflate the uncertainty and can put you some distance from your true location.
      - pjscott 9y ago
        Yes, they're a serious problem. What I'm saying is that they'd be a much bigger problem for GPS-based car navigation if there weren't logic in place to combine the GPS info with accelerometers and gyroscopes.
gambler 9y ago
> They aren't real problems because nobody smart is trying to directly hook up a road sign classifier to a steering wheel. They are trying to build complex systems where this is just one signal. What are the guarantees that those higher-level "complex systems" aren't going to have some weird behaviors as well? The problem isn't just that ANNs misclassify adversarial examples. The problem is that it's counter-intuitive to an average observer and that no one clearly knows why those examples generalize so well. It's a remarkable property that a lot of AI "enthusiasts" try to downplay. It's one thing to wire up unreliable but simple and fully understood components into a more reliable system. It's an entirely different level of challenge if the "unreliable components" are complex and poorly understood. > Humans also act much worse than you imply. Look at the number of people who accelerate into crashes instead of braking, etc. Except we know very well how and how often people make mistakes. For most of those mistakes we have a pretty good idea why they happen (at the high level, I'm not talking about neuroscience). Our roads, our cars and our laws are designed to handle these failures. Also, we have a pretty good model of how other people behave on the road, so we cal react accordingly. All of this goes out of the window with self-driving cars. Hell, most people naively assume that if a single self-driving gets into 50% less accidents than an average person than replacing all drivers with self-driving cars will reduce the global accident rate by at least 50%. This assumption doesn't take into account the fact that many accidents are caused by complex interactions between several vehicles and the environment. So introducing many self-driving cars can lead to some nasty emergent behaviors and some mass accidents that just aren't possible with human drivers.
kevin_thibedeau 9y ago
Tesla did it for their billboard detection. Didn't turn out so well and their "fix" is a horrid hack.

mannykannot 9y ago

The fragility of AI decision-making in general absolutely is an issue. Feel free to post a submission when it has been solved to a level that matches or exceeds human capabilities.

DannyBee 9y ago
Feel free to actually provide a real counterpoint to my argument instead of "well, you're wrong, so there".
- dboreham 9y ago
  Let's start with Spam : filtering it out with 100% guarantee of no false positives has still not been achieved.
  - MR4D 9y ago
    But also true of humans. In fact, I would guess I that advanced spam filtering engines might be _better_ than the average human. Remember, the Nigerian scammers are still around because someone is dumb enough to still give them money.
rm999 9y ago
I build AI/ML systems for a living and it's not as big of a deal as you think. Production machine learning systems built by experienced teams will almost always have business logic protections built-in to stop the machine from doing something totally stupid. For example, an auto-bidder may have capped velocities or auto-shutoff mechanisms (those examples of bidders spending billions of dollars in minutes are exceedingly sloppy). This is not unlike the human body's deeply-evolved reflexes - before the neocortex can even process that a pan is boiling hot, the nervous system bypasses the brain and and tells the hands to drop it. In the case of driverless cars, DannyBee is absolutely correct: truly driverless cars will not make decisions from a single data point. I have some past experience with unmanned aerial vehicles and those things had insane amounts of redundancy in almost every signal and decision making process; driverless cars are more complex and will come decades later and will probably take this even further.
- oneshot908 9y ago
  As another such practitioner, why oh why would we ignore hard-learned unambiguous signals and wisdom because we jumped on the machine learning bandwagon? We embraced machine learning to detect and uncover the things we don't see, not to replace the things we already know. There's no conflict here whatsoever.
- mannykannot 9y ago
  > Production machine learning systems built by experienced teams will almost always have business logic protections built-in to stop the machine from doing something totally stupid. In the case of driving, however, the problem of deciding what behavior to veto is itself a hard problem - perhaps the hardest of all. When the system monitoring the AI is itself AI, the argument becomes circular - which is not to say it cannot be solved, but I think it makes it harder to dismiss the concerns.

samstave 9y ago

Hw difficult would a GPS spoofer be, i.e. to transmit the wrong GPS coords via very cheap devices to at least confuse the heck out of any automated car? Oh look: http://www.rtl-sdr.com/spoofing-gps-locations-with-low-cost-tx-sdrs/ http://www.rtl-sdr.com/spoofing-gps-locations-with-low-cost-...

DannyBee 9y ago
Again, this assumes this is the only input, and things can't detect when some inputs seem completely insane with high probability and raise an error Seriously. Y'all realize how much in your daily life would pretty much explode and kill people if that wasn't true, right?
- samstave 9y ago
  Yeah, but that wasn't my point.. my point is that these attacks will be tried as soon as opportunity>cost-to-attempt
- pjscott 9y ago
  It's worth mentioning that car GPS navigation explicitly calculates how plausible each GPS reading is, based on a combination of other position signals and basic physics modeling, and discounts anything that sounds too crazy. If it didn't, simple things like tall buildings would be huge problems.
  - kevin_thibedeau 9y ago
    Tall buildings are huge problems for GPS. Multipath reflections will inflate the uncertainty and can put you some distance from your true location.
    - pjscott 9y ago
      Yes, they're a serious problem. What I'm saying is that they'd be a much bigger problem for GPS-based car navigation if there weren't logic in place to combine the GPS info with accelerometers and gyroscopes.

gambler 9y ago

> They aren't real problems because nobody smart is trying to directly hook up a road sign classifier to a steering wheel. They are trying to build complex systems where this is just one signal. What are the guarantees that those higher-level "complex systems" aren't going to have some weird behaviors as well? The problem isn't just that ANNs misclassify adversarial examples. The problem is that it's counter-intuitive to an average observer and that no one clearly knows why those examples generalize so well. It's a remarkable property that a lot of AI "enthusiasts" try to downplay. It's one thing to wire up unreliable but simple and fully understood components into a more reliable system. It's an entirely different level of challenge if the "unreliable components" are complex and poorly understood. > Humans also act much worse than you imply. Look at the number of people who accelerate into crashes instead of braking, etc. Except we know very well how and how often people make mistakes. For most of those mistakes we have a pretty good idea why they happen (at the high level, I'm not talking about neuroscience). Our roads, our cars and our laws are designed to handle these failures. Also, we have a pretty good model of how other people behave on the road, so we cal react accordingly. All of this goes out of the window with self-driving cars. Hell, most people naively assume that if a single self-driving gets into 50% less accidents than an average person than replacing all drivers with self-driving cars will reduce the global accident rate by at least 50%. This assumption doesn't take into account the fact that many accidents are caused by complex interactions between several vehicles and the environment. So introducing many self-driving cars can lead to some nasty emergent behaviors and some mass accidents that just aren't possible with human drivers.

kevin_thibedeau 9y ago

Tesla did it for their billboard detection. Didn't turn out so well and their "fix" is a horrid hack.