3 ms·

Can't wait to adversarially modify pedestrian crossing signs to stop signs.

by cozzyd 8y ago

cshimmin 8y ago
I appreciate that your comment was tongue-in-cheek, but note that what you propose is actually a case of a standard targeted adversarial attack (changing a label prediction from "ped x-ing" to "stop"). This is fundamentally different from what's being proposed in the paper, which is to repurpose an existing NN (and its output labels) to perform another task altogether. I'm still trying to figure how it could possibly be useful... the authors suggest it could be used to "steal resources", but it seems a bit contrived.
- cozzyd 8y ago
  Well, if you could transform your problem into the "stop" vs. "ped x-ing" problem, you could let the future smart self-driving cars solve it for you. Maybe they have more computational resources than you can muster (but that sounds very contrived, even if you could somehow harness the computational power of a freeway full of future cars). Or you could imagine another very contrived case where some self-driving car company controls some important algorithm you need to solve your problem via a patent and it would be illegal to solve your problem otherwise (also very contrived...).
- nerdponx 8y ago
  It would be useful to anyone looking to cause havoc, terror, and/or harm.
- king07828 8y ago
  > how it could possibly be useful 1 create a GPU friendly cryptocurrency where the payload includes a two dimensional data set (or image) that is processed with a standardized neural network model 2 let programmers pay a mining fee to expedite execution 3 instead of GPUs doing nothing but redundant calculations and getting replaced by ASICs, mining GPUs can dual purpose mine and execute arbitrary code Will it be efficient, probably not. It does kill two birds with one stone
- sgibat 8y ago
  It could be the beachhead of a larger attack. You could use the model's authority to steal its data or gain access to other resources.
jacksmith21006 8y ago
I would think the self driving car database knows where the stop sign is and then creates an exception when inconsistent with what is sees in the real world which is then sent to a team to look into. Maybe the stop was knocked down, or changed, etc. Also would expect changes are streamed from the gov group that makes changes to the the people that maintain the data base at Waymo. The car should take the safest option when an inconsistency is found and then the data base updated. How Waymo is doing level 4 is more like those slot cars we had when a kid and the map is the slot. Level 5 will be like when the cars no longer had slots and much more difficult to navigate. This is about life and death and be cool to do more interesting technical approaches but really taking smaller steps in the future is more prudent. The entire PR from Tesla and level 5 is not based on reality. We do not have the algorithms to do level 5 and therefore do not have the hardware as we do not know what is required. Musk is much smarter than I and no way he does not know this. But he must know at some point it will be discovered?

Can't wait to adversarially modify pedestrian crossing signs to stop signs.

by cozzyd 8y ago

Can't wait to adversarially modify pedestrian crossing signs to stop signs.

I appreciate that your comment was tongue-in-cheek, but note that what you propose is actually a case of a standard targeted adversarial attack (changing a label prediction from "ped x-ing" to "stop"). This is fundamentally different from what's being proposed in the paper, which is to repurpose an existing NN (and its output labels) to perform another task altogether. I'm still trying to figure how it could possibly be useful... the authors suggest it could be used to "steal resources", but it seems a bit contrived.

cozzyd 8y ago
Well, if you could transform your problem into the "stop" vs. "ped x-ing" problem, you could let the future smart self-driving cars solve it for you. Maybe they have more computational resources than you can muster (but that sounds very contrived, even if you could somehow harness the computational power of a freeway full of future cars). Or you could imagine another very contrived case where some self-driving car company controls some important algorithm you need to solve your problem via a patent and it would be illegal to solve your problem otherwise (also very contrived...).
nerdponx 8y ago
It would be useful to anyone looking to cause havoc, terror, and/or harm.
king07828 8y ago
> how it could possibly be useful 1 create a GPU friendly cryptocurrency where the payload includes a two dimensional data set (or image) that is processed with a standardized neural network model 2 let programmers pay a mining fee to expedite execution 3 instead of GPUs doing nothing but redundant calculations and getting replaced by ASICs, mining GPUs can dual purpose mine and execute arbitrary code Will it be efficient, probably not. It does kill two birds with one stone
sgibat 8y ago
It could be the beachhead of a larger attack. You could use the model's authority to steal its data or gain access to other resources.

jacksmith21006 8y ago

I would think the self driving car database knows where the stop sign is and then creates an exception when inconsistent with what is sees in the real world which is then sent to a team to look into. Maybe the stop was knocked down, or changed, etc. Also would expect changes are streamed from the gov group that makes changes to the the people that maintain the data base at Waymo. The car should take the safest option when an inconsistency is found and then the data base updated. How Waymo is doing level 4 is more like those slot cars we had when a kid and the map is the slot. Level 5 will be like when the cars no longer had slots and much more difficult to navigate. This is about life and death and be cool to do more interesting technical approaches but really taking smaller steps in the future is more prudent. The entire PR from Tesla and level 5 is not based on reality. We do not have the algorithms to do level 5 and therefore do not have the hardware as we do not know what is required. Musk is much smarter than I and no way he does not know this. But he must know at some point it will be discovered?