3 ms·

I think the traffic sign example was a nice setting for explaining the research. But of course really good examples always risk interfering with communication a

I think the traffic sign example was a nice setting for explaining the research. But of course really good examples always risk interfering with communication as much as they assist communication. I think that might've happened here. Concretely, I have a hard time believing that adversarial traffic signs will be the Achilles heel of self-driving cars. Vandalizing signs is already illegal, and if doing so causes or was intended to cause a crash, law enforcement will throw the book at the culprit. And this already happens to human drivers. See e.g., http://globegazette.com/news/local/prank-turns-to-peril-vandalism-called-likely-cause-of-serious/article_ae3c15e9-f3c0-5b00-a21d-2804a4d08806.html http://globegazette.com/news/local/prank-turns-to-peril-vand... or http://articles.latimes.com/1997-06-11/news/mn-2303_1_one-stop-sign http://articles.latimes.com/1997-06-11/news/mn-2303_1_one-st... Besides which, you can also just rip the sign out of the ground or replace the stop sign with an actual speed limit sign. From a SDC perspective, this is kind of the equivalent of using super clever software vulnerabilities when you have physical access to an un-encrypted HDD. Impressive technical tour de force, but misses the forest for the trees if your goal is security. Traffic sign vandalism is a social problem with a social solution. The first person who tries something like this in production with malicious intent toward SDCs will be charged with some variation of attempted or actual manslaughter/murder and opponents of SDCs will switch to strategies other than domestic terrorism to advance their agenda. This research is still very interesting and useful, of course, but primarily for other reasons. I just don't see traffic sign vandalism intended to confuse DNNs as a serious -- or unique -- threat to self-driving cars.

cscurmudgeon 9y ago
I think there is a point here that is being overlooked. The vandalism, a couple of small stickers, would be less detectable compared with a whole missing sign. I can see such attack methods evolving to produce undetectetable (to humans without software) adversarial images.
- throwawayjava 9y ago
  I'm not saying we shouldn't practice defense in depth where it's possible. It's just that I don't think preventing this style of attack is actually the most relevant/impactful application of this style of research. Again, I'm not saying this research is irrelevant. I'm saying it's way more relevant than this particular example implies. > would be less detectable compared with a whole missing sign. This kind of assumes that the attack wouldn't work in the first place, no? If cars are crashing people are going to investigate to figure out why, and even the most thick-skulled beat cop can figure out that stop signs with rando weird ass stickers on them are probably worth running up the chain...
  - cscurmudgeon 9y ago
    > This kind of assumes that the attack wouldn't work in the first place, no? If cars are crashing people are going to investigate to figure out why, and even the most thick-skulled beat cop can figure out that stop signs with rando weird ass stickers on them are probably worth running up the chain... 1. That attack may not target only self-driving cars. 2. Imagine someone driving around with some random picture on their car at the back. Except that the picture is an attack.
pessimizer 9y ago
> Traffic sign vandalism is a social problem with a social solution. The first person who tries something like this in production with malicious intent toward SDCs will be charged with some variation of attempted or actual manslaughter/murder and opponents of SDCs will switch to strategies other than domestic terrorism to advance their agenda. It's tunnelvision to assume that the only attackers would be individual radical luddites. There do exist terrorist organizations and state actors who intend to run risks to do damage. And unless you're planning to monitor and respond quickly to all potential threats to all stop signs at all times (and all other possible attacks), what's to stop a group of 40 terrorists from covering their faces, stealing motorcycles, and putting 10,000 stickers on 10,000 signs in 10,000 intersections in 20 different cities in 1 night?
- throwawayjava 9y ago
  What's to stop 40 terrorists from covering their faces, stealing motorcycles, and clipping brake cables (whatever you get the point...) on 10,000 cars parked along 10,000 streets in 20 different cities in 1 night? ...Or just blowing up a shitload of random houses or whatever. Basically, the exact same thing as what would stop your scenario. Is it perfect? Even good? Again, no. But striving for good security in a public setting is how you get TSA. If there are 40 committed and sophisticated terrorists willing to act, there are a lot of easier targets... Again, I think the "clever localhost software vuln" analogy is apt. It's not that this isn't one possible attack vector. And no attack vector should be ignored. But... really?

cscurmudgeon 9y ago

I think there is a point here that is being overlooked. The vandalism, a couple of small stickers, would be less detectable compared with a whole missing sign. I can see such attack methods evolving to produce undetectetable (to humans without software) adversarial images.

throwawayjava 9y ago
I'm not saying we shouldn't practice defense in depth where it's possible. It's just that I don't think preventing this style of attack is actually the most relevant/impactful application of this style of research. Again, I'm not saying this research is irrelevant. I'm saying it's way more relevant than this particular example implies. > would be less detectable compared with a whole missing sign. This kind of assumes that the attack wouldn't work in the first place, no? If cars are crashing people are going to investigate to figure out why, and even the most thick-skulled beat cop can figure out that stop signs with rando weird ass stickers on them are probably worth running up the chain...
- cscurmudgeon 9y ago
  > This kind of assumes that the attack wouldn't work in the first place, no? If cars are crashing people are going to investigate to figure out why, and even the most thick-skulled beat cop can figure out that stop signs with rando weird ass stickers on them are probably worth running up the chain... 1. That attack may not target only self-driving cars. 2. Imagine someone driving around with some random picture on their car at the back. Except that the picture is an attack.

pessimizer 9y ago

> Traffic sign vandalism is a social problem with a social solution. The first person who tries something like this in production with malicious intent toward SDCs will be charged with some variation of attempted or actual manslaughter/murder and opponents of SDCs will switch to strategies other than domestic terrorism to advance their agenda. It's tunnelvision to assume that the only attackers would be individual radical luddites. There do exist terrorist organizations and state actors who intend to run risks to do damage. And unless you're planning to monitor and respond quickly to all potential threats to all stop signs at all times (and all other possible attacks), what's to stop a group of 40 terrorists from covering their faces, stealing motorcycles, and putting 10,000 stickers on 10,000 signs in 10,000 intersections in 20 different cities in 1 night?

throwawayjava 9y ago
What's to stop 40 terrorists from covering their faces, stealing motorcycles, and clipping brake cables (whatever you get the point...) on 10,000 cars parked along 10,000 streets in 20 different cities in 1 night? ...Or just blowing up a shitload of random houses or whatever. Basically, the exact same thing as what would stop your scenario. Is it perfect? Even good? Again, no. But striving for good security in a public setting is how you get TSA. If there are 40 committed and sophisticated terrorists willing to act, there are a lot of easier targets... Again, I think the "clever localhost software vuln" analogy is apt. It's not that this isn't one possible attack vector. And no attack vector should be ignored. But... really?