11 ms·

Adversarial Attacks? Like removing a speed limit sign? Painting over lane markers? Dropping bricks off of overpasses? Throwing down a spike strip? Sugar in

Adversarial Attacks? Like removing a speed limit sign? Painting over lane markers? Dropping bricks off of overpasses? Throwing down a spike strip? Sugar in the gas tank? Unless you're talking traditional computer security, which it doesn't seem like you are, these types of threats have not prohibited human drivers despite the fact that humans are very susceptible to "adversarial attacks" while driving too. Whether it's putting carefully crafted stickers over a stop sign to confuse a CNN or yanking it out of the ground to get a human driver killed.. you're talking about interfering with the operator of a moving vehicle... so what's the critical difference here?

sorokod 8y ago
Adversarial attacks with low friction for the attacker e.g malicious software updates.
- fattire 8y ago
  Do you mean OTA or just any attack because I think non-autonomous vehicles would have the same concerns... or really any equipment with embedded computers.
  - sorokod 8y ago
    I mean any attack where the attacker is not required to move away from the keyboard and can corrupt multiple vehicles in one go.
    - fattire 8y ago
      Could the keyboard be in a radio-equipped car? https://jalopnik.com/its-scarily-easy-to-hack-a-traffic-light-1785313010 https://jalopnik.com/its-scarily-easy-to-hack-a-traffic-ligh...
dTal 8y ago
I think the difference is that software tends to be much more fragile - and more predictable - than humans. Paint fake lane markers, and an autopilot might drive full speed into a wall because it trusts them; the next 5 cars with autopilots will all do the same thing. An attacker can verify that an autopilot will do this ahead of time. A human on the other hand will be more likely to notice that things are amiss - they can pick up on contextual clues, like the fresh paint, and the fact they've driven that road hundreds of times before and instantly notice the change, and the pile of burning self-driving cars.
dmitriid 8y ago
> Adversarial Attacks? Like Like literally the article we’re commenting on. Image recognition systems in general are much more susceptible to errors in cases where humans wouldn’t even think twice. And yes, “hey, a stop sign was here just yesterday” is also a situation for which humans are uniquely equipped, and computers aren’t.
bnegreve 8y ago
I'm not sure why hackers prefer to hack large scale computer systems rather than individual humans, but they do. So we have to protect neural networks against adversarial examples for the same reason we have to protect databases against sql injections. > so what's the critical difference here? If neural networks are deployed at scale in self driving cars, a single bug could trigger millions of accidents. Printing an adversarial example on billboards would lead to crashes all around the country. Are we going to assume no one is going to try? (btw: real world adversarial examples are easy to craft [1]). [1] https://arxiv.org/pdf/1707.07397.pdf https://arxiv.org/pdf/1707.07397.pdf

sorokod 8y ago

Adversarial attacks with low friction for the attacker e.g malicious software updates.

fattire 8y ago
Do you mean OTA or just any attack because I think non-autonomous vehicles would have the same concerns... or really any equipment with embedded computers.
- sorokod 8y ago
  I mean any attack where the attacker is not required to move away from the keyboard and can corrupt multiple vehicles in one go.
  - fattire 8y ago
    Could the keyboard be in a radio-equipped car? https://jalopnik.com/its-scarily-easy-to-hack-a-traffic-light-1785313010 https://jalopnik.com/its-scarily-easy-to-hack-a-traffic-ligh...

dTal 8y ago

I think the difference is that software tends to be much more fragile - and more predictable - than humans. Paint fake lane markers, and an autopilot might drive full speed into a wall because it trusts them; the next 5 cars with autopilots will all do the same thing. An attacker can verify that an autopilot will do this ahead of time. A human on the other hand will be more likely to notice that things are amiss - they can pick up on contextual clues, like the fresh paint, and the fact they've driven that road hundreds of times before and instantly notice the change, and the pile of burning self-driving cars.

dmitriid 8y ago

> Adversarial Attacks? Like Like literally the article we’re commenting on. Image recognition systems in general are much more susceptible to errors in cases where humans wouldn’t even think twice. And yes, “hey, a stop sign was here just yesterday” is also a situation for which humans are uniquely equipped, and computers aren’t.

bnegreve 8y ago

I'm not sure why hackers prefer to hack large scale computer systems rather than individual humans, but they do. So we have to protect neural networks against adversarial examples for the same reason we have to protect databases against sql injections. > so what's the critical difference here? If neural networks are deployed at scale in self driving cars, a single bug could trigger millions of accidents. Printing an adversarial example on billboards would lead to crashes all around the country. Are we going to assume no one is going to try? (btw: real world adversarial examples are easy to craft [1]). [1] https://arxiv.org/pdf/1707.07397.pdf https://arxiv.org/pdf/1707.07397.pdf