3 ms·

I've been concerned about the "move fast and actually break things" aspect of autonomous devices for the past year and a half because I view malcode pushed from

I've been concerned about the "move fast and actually break things" aspect of autonomous devices for the past year and a half because I view malcode pushed from the update server as a mass damage event / equivalent to weapons of mass destruction if intentional. I'm not sure how it is now, but according to someone talking to me privately at one company early programmers were SSHing into fucking moving cars with passengers and running commands on them and had it not been for the layers of redundant code would have crashed the car. We need to regulate autonomous devices and we need to do it before there are billions of them. We need a public effort to get policy makers open source outlines of regulations; otherwise the car companies are going to be writing them and they aren't going to write in protections against black swan events and we're going to get the autonomous equivalent of a BP oil spill or worse one day.

simion314 9y ago
I think if there is a large enough penalty for crashes then the car companies would take quality more serious, if the costs are too low I can see them pushing updates with just a few hours of testing, There is also the issue with adversarial inputs to the AI, you could have someone put stickers everywhere that would crash the AI .
- MBCook 9y ago
  The penalty only works after someone has been caught at it once (it seems clear to me these companies aren’t proactive about his stuff). That ‘once’ could be One (or many) very bad accidents. ‘Company X fines $750m for incident where they caused 250 car crashes and 478 deaths’ is not a headline I want to see.
  - komali2 9y ago
    >[In Texas] in 2016, 987 people lost their lives to DUI incidents https://www.rightstep.com/resources/texas-addiction-information/texas-drunk-driving-statistics/ https://www.rightstep.com/resources/texas-addiction-informat... I'll take that headline.
wu_tang_chris 9y ago
> early programmers were SSHing into fucking moving cars better than telnet!

simion314 9y ago

I think if there is a large enough penalty for crashes then the car companies would take quality more serious, if the costs are too low I can see them pushing updates with just a few hours of testing, There is also the issue with adversarial inputs to the AI, you could have someone put stickers everywhere that would crash the AI .

MBCook 9y ago
The penalty only works after someone has been caught at it once (it seems clear to me these companies aren’t proactive about his stuff). That ‘once’ could be One (or many) very bad accidents. ‘Company X fines $750m for incident where they caused 250 car crashes and 478 deaths’ is not a headline I want to see.
- komali2 9y ago
  >[In Texas] in 2016, 987 people lost their lives to DUI incidents https://www.rightstep.com/resources/texas-addiction-information/texas-drunk-driving-statistics/ https://www.rightstep.com/resources/texas-addiction-informat... I'll take that headline.

wu_tang_chris 9y ago

> early programmers were SSHing into fucking moving cars better than telnet!