3 ms·

Civil liability is only half the issue. What about criminal liability? If your autonomous vehicle runs someone down because of a bug, who goes to jail?

kijinbear 16y ago
Same as today, I guess -- whoever is responsible for the bug, but only if criminal negligence can be proven.
ajju 16y ago
What happens when this is due to a hardware bug, like with the Toyotas recently? I don't think criminal liability is involved except in special circumstances. Why should software be any different. IANAL however.
- alextgordon 16y ago
  I think the difference between this and the Toyota incident is that the driver still maintains absolute control over the autonomous google car. So it could be argued that anything the AI does is still ultimately the driver's responsibility.
jfager 16y ago
We deal with that issue today. Who is liable when brakes fail? It's usually determined on a case-by-case basis. Why would this system be any different?
- ajays 16y ago
  Failure is one aspect. But what if there's a bug? Suppose there's an extreme edge case where a bicyclists fluorescent jacket appears as a road to the car; so the car chases down the cyclist and runs him over. This is not a hardware failure, but an active decision made by the car to cause injury to a person. So in such a situation, who would be responsible? I guess it is a question for technolawyers to decide.
  - jfager 16y ago
    How is that not a system failure? Because the car has some kind of intent? Well, we have an analogue for that today, too: who gets punished when a dog attacks someone? I'm not saying there are no details to work out. I'm just saying that it's not some unfathomable chasm we have to cross. We deal with similar questions all the time right now.
- Someone 16y ago
  I guess this could be different just because 'the public' does not accept it to be the same. That may be irrational, but it also is quite natural. We accept traffic deaths more than plane crashes, invest disproportionately in research on some diseases, etc.
tbrownaw 16y ago
My understanding is that criminal liability almost always requires either intent negligence. If the automated system is less error-prone than a human, the driver can't reasonably be negligent for doing nothing ("it would be incredibly foolish and arrogant for me to think I could handle that situation better than a computer designed by 10 Phds, especially when the insurance company agrees that the computer drives better"). For the manufacturer the reasonable position would be that as long as they followed certain verification/testing standards, and fixed any discovered bugs as fast as reasonably possible (probably not issuing a recall or telling people to turn it off, assuming the system was still safer than a human driver), they should be fine. The industry might need to get together and come up with standards and a certification body for the first part to work, and I have no idea how well the no-recall thing would work without something explicitly permitting it.
- alextgordon 16y ago
  This is surely the rational way to handle it. But I'm not convinced a judge and jury would see it the same way. The prosecution would likely argue that no computer program can be a match for a human's ability handle new and surprising situations: the "driver" must still pay full attention to the road, whether a computer is driving or not. It would not be enough for a computer to be statistically better at driving, it would also have to not make mistakes that a reasonable person wouldn't make. Also, we don't have information on how error-prone this system actually is. All we know is that a car it was driving had one accident. We don't know how many times the human driver had to take control because it would have otherwise crashed. [obligatory IANAL]
  - tbrownaw 16y ago
    > argue that no computer program can be a match for a human's ability handle new and surprising situations I suspect that this ability is due to our having more awareness of context. I imagine it wouldn't be terribly hard to dump a larger amount of contextual knowledge into a car, probably just attach an expert system and pay a few tens of millions of dollars to have people fill it with data. I'm not sure what the state of machine learning is, but they could perhaps pay owners to upload logs (maybe especially after any accident / close call, or maybe that would give a skewed perception) and then crunch those to find new rules. > Also, we don't have information on how error-prone this system actually is. All we know is that a car it was driving had one accident. We don't know how many times the human driver had to take control because it would have otherwise crashed. Yup, that the humans ever took control at all says it probably isn't quite ready yet. But it looks very promising. If I ever have kids I might not need to deal with them learning to drive, and maybe my next car could let me browse the internet on the way in to socialize (well, why else won't they let me telecommute?) at work.

kijinbear 16y ago

Same as today, I guess -- whoever is responsible for the bug, but only if criminal negligence can be proven.

ajju 16y ago

What happens when this is due to a hardware bug, like with the Toyotas recently? I don't think criminal liability is involved except in special circumstances. Why should software be any different. IANAL however.

alextgordon 16y ago
I think the difference between this and the Toyota incident is that the driver still maintains absolute control over the autonomous google car. So it could be argued that anything the AI does is still ultimately the driver's responsibility.

jfager 16y ago

We deal with that issue today. Who is liable when brakes fail? It's usually determined on a case-by-case basis. Why would this system be any different?

ajays 16y ago
Failure is one aspect. But what if there's a bug? Suppose there's an extreme edge case where a bicyclists fluorescent jacket appears as a road to the car; so the car chases down the cyclist and runs him over. This is not a hardware failure, but an active decision made by the car to cause injury to a person. So in such a situation, who would be responsible? I guess it is a question for technolawyers to decide.
- jfager 16y ago
  How is that not a system failure? Because the car has some kind of intent? Well, we have an analogue for that today, too: who gets punished when a dog attacks someone? I'm not saying there are no details to work out. I'm just saying that it's not some unfathomable chasm we have to cross. We deal with similar questions all the time right now.
Someone 16y ago
I guess this could be different just because 'the public' does not accept it to be the same. That may be irrational, but it also is quite natural. We accept traffic deaths more than plane crashes, invest disproportionately in research on some diseases, etc.

tbrownaw 16y ago

My understanding is that criminal liability almost always requires either intent negligence. If the automated system is less error-prone than a human, the driver can't reasonably be negligent for doing nothing ("it would be incredibly foolish and arrogant for me to think I could handle that situation better than a computer designed by 10 Phds, especially when the insurance company agrees that the computer drives better"). For the manufacturer the reasonable position would be that as long as they followed certain verification/testing standards, and fixed any discovered bugs as fast as reasonably possible (probably not issuing a recall or telling people to turn it off, assuming the system was still safer than a human driver), they should be fine. The industry might need to get together and come up with standards and a certification body for the first part to work, and I have no idea how well the no-recall thing would work without something explicitly permitting it.

alextgordon 16y ago
This is surely the rational way to handle it. But I'm not convinced a judge and jury would see it the same way. The prosecution would likely argue that no computer program can be a match for a human's ability handle new and surprising situations: the "driver" must still pay full attention to the road, whether a computer is driving or not. It would not be enough for a computer to be statistically better at driving, it would also have to not make mistakes that a reasonable person wouldn't make. Also, we don't have information on how error-prone this system actually is. All we know is that a car it was driving had one accident. We don't know how many times the human driver had to take control because it would have otherwise crashed. [obligatory IANAL]
- tbrownaw 16y ago
  > argue that no computer program can be a match for a human's ability handle new and surprising situations I suspect that this ability is due to our having more awareness of context. I imagine it wouldn't be terribly hard to dump a larger amount of contextual knowledge into a car, probably just attach an expert system and pay a few tens of millions of dollars to have people fill it with data. I'm not sure what the state of machine learning is, but they could perhaps pay owners to upload logs (maybe especially after any accident / close call, or maybe that would give a skewed perception) and then crunch those to find new rules. > Also, we don't have information on how error-prone this system actually is. All we know is that a car it was driving had one accident. We don't know how many times the human driver had to take control because it would have otherwise crashed. Yup, that the humans ever took control at all says it probably isn't quite ready yet. But it looks very promising. If I ever have kids I might not need to deal with them learning to drive, and maybe my next car could let me browse the internet on the way in to socialize (well, why else won't they let me telecommute?) at work.