12 ms·

I'm worried about the following: Hapless Joe buys a nice, well reviewed, good looking and performant self-driving car. He enjoys it very much for a year, goes f

by krisoft 10y ago

I'm worried about the following: Hapless Joe buys a nice, well reviewed, good looking and performant self-driving car. He enjoys it very much for a year, goes for all the required maintenance checks. Then suddenly as he is taking a nap on his usual daily commute the car swerves onto the sidewalk and runs over my wife. Turns out there is a very rare race condition in the path finding algorithm, which makes it do that with astronomically low probability. What I don't want: I don't want Hapless Joe's life to be ruined, he did nothing wrong. I also don't want my wife to become just a statistics. What I want: A crack team of industry veterans get to work. Both engineers from the self driving car company, and independent experts. Since accidents of this kind are so rare they have ample amount of resources to overturn every rock, develop new investigation techniques if must, and nail down all the nodes in the causality chain. Then they disseminate their findings and make all cars on the road, as well as the industry best practices, so much safer. Is this how you mean the owner's liability to work out? Because then I agree with you. If you want to punish Joe. Take away his savings, possibly his home. I don't see a point in that honestly.

paulmd 10y ago
Exactly. Consumers have no insight into the design process, they have no ability to affect the operation of the autonomous mode, it's insane to assign them liability. Once the car comes out of autonomous mode we can assign them some liability, depending on the situation.
ghaff 10y ago
In your hypothetical, I have a hard time giving an example from current consumer products where it isn't the manufacturer's fault. Approved device, used as directed, properly maintained causes a fatality. With what consumer device today wouldn't it be the fault of the device maker/seller. At the same time, I'm hard put to think of products/services used by consumers--outside of medicine perhaps--where it's the norm that sometimes "shit happens" because statistics and it's not considered reasonable to sue the manufacturer (though we often still do). Interesting times. Especially given that the facts (and the degree of automation) won't typically be nearly as clear cut especially over the next few decades.
Animats 10y ago
The NTSB is doing that right now.[1] They're looking at Tesla's first fatal crash with the level of detail they apply to air crash investigations. This takes months. Incidentally, in aviation and aerospace, everybody reads crash reports. Pilots study them in training. Engineers read them at work. Flying magazine publishes monthly summaries in their "Aftermath" column. The safety culture is very different from software. In commercial aviation, just about all the single points of failure have already been dealt with. It's a design criterion of aviation that there must be a way to recover from single failures. When you're one single failure away from real trouble, it's time to land. When you read a crash report today, there are two or three things that went wrong, not just one. That's what a safety culture looks like. Self-driving cars may have to get there. [1] http://www.ntsb.gov/investigations/AccidentReports/Pages/HWY16FH018-preliminary.aspx http://www.ntsb.gov/investigations/AccidentReports/Pages/HWY...

by krisoft 10y ago

paulmd 10y ago

Exactly. Consumers have no insight into the design process, they have no ability to affect the operation of the autonomous mode, it's insane to assign them liability. Once the car comes out of autonomous mode we can assign them some liability, depending on the situation.

ghaff 10y ago

In your hypothetical, I have a hard time giving an example from current consumer products where it isn't the manufacturer's fault. Approved device, used as directed, properly maintained causes a fatality. With what consumer device today wouldn't it be the fault of the device maker/seller. At the same time, I'm hard put to think of products/services used by consumers--outside of medicine perhaps--where it's the norm that sometimes "shit happens" because statistics and it's not considered reasonable to sue the manufacturer (though we often still do). Interesting times. Especially given that the facts (and the degree of automation) won't typically be nearly as clear cut especially over the next few decades.

Animats 10y ago

The NTSB is doing that right now.[1] They're looking at Tesla's first fatal crash with the level of detail they apply to air crash investigations. This takes months. Incidentally, in aviation and aerospace, everybody reads crash reports. Pilots study them in training. Engineers read them at work. Flying magazine publishes monthly summaries in their "Aftermath" column. The safety culture is very different from software. In commercial aviation, just about all the single points of failure have already been dealt with. It's a design criterion of aviation that there must be a way to recover from single failures. When you're one single failure away from real trouble, it's time to land. When you read a crash report today, there are two or three things that went wrong, not just one. That's what a safety culture looks like. Self-driving cars may have to get there. [1] http://www.ntsb.gov/investigations/AccidentReports/Pages/HWY16FH018-preliminary.aspx http://www.ntsb.gov/investigations/AccidentReports/Pages/HWY...