5 ms·

Is the only way we will accept something like autopilot is if it has a perfect driving record? Is it not enough to be better than humans on average? Thousands o

Is the only way we will accept something like autopilot is if it has a perfect driving record? Is it not enough to be better than humans on average? Thousands of people die in automotive accidents, but we don't think twice when a human is behind the wheel.

phpisthebest 4y ago
I have long said that if someone invited the car today it would never be approved for use Society is FAR FAR FAR FAR FAR more risk adverse which IMO is holding us back, we can not advance because perfection is impossible
mechagodzilla 4y ago
Many people are charged with crimes related to automotive accidents (DUI, for instance, or laws related to cell phone usage). We require drivers to carry insurance. We are constantly regulating things to reduce deaths and injuries in automotive accidents (seat belts, crumple zones, airbags, forced recalls, etc.) - we constantly think about automotive accidents when a human is behind the wheel. Why should we just shrug when some Tesla mows down a pedestrian while the driver is watching youtube videos?
- ck2 4y ago
  Always prosecute the driver. There's no such thing as "the car did it". And if someone wanted to intentionally murder someone, seems like a perfect excuse "the car did it". Unless we are going to victim blame.
  - themaninthedark 4y ago
    But the point of the autopilot system is that there doesn't need to be a driver right? So here you are saying, "Hey, we have this great system that is such a better diver than the human. Here are the statistics that show it." and then turning around and saying "Hey human, why were you not a better driver. You need to be paying attention." A bit of wanting to have your cake and eat it too, isn't it?
    - ck2 4y ago
      "autopilot" is just a marketing term for the next decade, it's not fully automatic, it's actually "computer aided driving" (where driving is the human) but that description isn't sexy enough to sell
      - themaninthedark 4y ago
        The GGP talks about "something like autopilot" acceptance and the eventual goal of the system is to cut the human driver out. GP is talking about insurance and how we regulate drivers, stating that we should be looking at the same for autopilot systems. Parent says "Always prosecute the driver. There's no such thing as "the car did it"." At what point does the "autopilot" have any skin in the game so to speak? Is it a binary thing, one day we wake up to an update and the cars can drive themselves with no input? Or will will be forever stuck in a quasi beta test, where the driver is liable for the actions of the system but has no meaningful way for controlling the system(autopilot) or even knowing how the system operates. You can't have it both ways. Either the system is better than people and is going to take responsibility or it isn't. No one will accept a system they have no control over and do not understand but have to accept liability for.
        ck2 4y ago
        There may come a day where someone is truly the passenger in the driver's seat in a car, like a person riding a bus. Today is not that day. Nor is it next year. Or this decade. A whole lot of people are going to die this decade, both drivers and victims hit, because marketing wants people to believe you don't have to pay attention once you turn it on rather it's an assist, not a replacement, for the driver. It's not autopilot right now. It's driver assist. As long as the driver exists, the driver is responsible. In an "old school" car, if an accelerator gets stuck and you mow someone down, then they find spaghetti code in the accelerator or it mechanically got stuck from design failure, you still have liability for killing someone but your prosecution won't be the same as someone who purposely drove over someone they were angry with.
    - drcongo 4y ago
  - falcolas 4y ago
    You are technically correct for tesla vehicles (though there's some grey area around how Tesla markets things - you can't just handwave away legal liability for a death with "but the fine print!"), but not for self-driving vehicles in general. Waymo taxis, for example, explicitly does not have a driver. And Tesla has explicitly advertised that this Waymo scenario is where they're headed, so it's a question that's definitely worth thinking about now. "Blame the driver" isn't looking like a tenable position long term.
- onelovetwo 4y ago
  not sure how carrying insurance is related to accident prevention, Its mandatory in both cases. The amount of cases we charge people for accidents is extremely rare because in most cases its an actual accident. People swerving into oncoming traffic, not checking both sides before making a turn...etc.
dontreact 4y ago
It seems a bare minimum is releasing enough data that the public can assess whether it is better on average. Right now, Tesla releases data that show that when autopilot is engaged, it is safer on average. However, this data is not adjusted for the fact that the miles where users choose to activate it are on average safer. So really some sort of base-risk adjusted comparison would be the bare minimum I would like to see them publish. They really don't share much data.
icyfox 4y ago
Even if we accept some statistical failure rate, we should still investigate lethal failures. Like any good RCA it may uncover some learnings that could help the implementation and/or regulations around self driving.
addicted 4y ago
Yes we do. We have a ton of rules before, during and after humans are behind the wheel. You are correct that we should probably be thinking even more about it, and other countries are doing that, slowing down cars, providing separation from other modes of transport and pedestrians, and reducing car dependence altogether.
sonofhans 4y ago
> Is it not enough to be better than humans on average? No, not at all. The average driver is 1/3 distracted and 1/4 drunk. Autopilot and similar systems must be at least as good as a skilled, attentive, unimpaired driver. > Is the only way we will accept something like autopilot is if it has a perfect driving record? Putting it plainly, this is bullshit. No one -- no one at all -- is asking for autopilot to be perfect. People are asking for it not to drive full-speed into obvious obstacles.
- themaninthedark 4y ago
  >The average driver is 1/3 distracted and 1/4 drunk. Kinda curious where you are getting those numbers from.
- kqr 4y ago
  > People are asking for it not to drive full-speed into obvious obstacles. But isn't this a weird criterion? An obvious obstacle for a human might not be obvious to the computer -- but importantly, it could be the other way around too: the computer detects obstacles that are obvious to it, but a human would miss. Shouldn't the goal be that the computer considers more obstacles obvious than the human? I get the emotional argument that it's hard to convince a person off the street of this, but isn't HN a place for more nuanced debate?
  - sonofhans 4y ago
    It seems like you're debating the meaning of "obvious" -- "What is obvious? And to whom?" My meaning was, "obvious to a skilled, attentive, unimpaired human." > Shouldn't the goal be that the computer considers more obstacles obvious than the human? That would be excellent, as long as it's correct :)
    - hcurtiss 4y ago
      In my view, the question should be whether it's more or less safe than humans driving. If use of automated driving results in less death than humans driving, considered at the broadest possible scale, then that strikes me as an excellent outcome. Instead, each and every death under autopilot results in all the hens clucking as though it's proof the technology has failed. That's ridiculous.
xeromal 4y ago
This is just an investigation which is important in cases like this. They need to determine if Teslas can't see motorcycle riders at 1am in Draper, Utah. There's all kinds of oddities that can happen so these independent investigations are necessary. Side note, imagine if you got bulldozed by a Tesla and died only to have someone write this comment. I know we live in a huge world and we can't think of everyone this way, but there should be a modicum of compassion for the guy who just died. His life has ended for what amounts to a potential programming error. It doesn't matter if another human could do it to. This man is actually dead. It's tragic.
boudin 4y ago
I really doubt Tesla is even close from being better than human for now, seems like it needs to be in really specific and simple situation to act properly. There's countless videos showing how clueless it is as soon as it is in an environment that is a bit more complex, including attempting to run into pedestrians. Most humans don't do that.
tqi 4y ago
For one, "better than average" is hard to measure / subject to lots of confounding factors, so people (rightfully) find those claims hard to believe. For another Autopilot often fails in different ways than human drivers, so it's easy to find examples where a human driver would have been fine, which exacerbates the perception that these are not in fact better.
rurp 4y ago
If you have proof that autopilot is safer I'd love to see it. Cherry picked and non-standard data selectively released from the company selling autopilot for thousands of dollars doesn't count :).
themaninthedark 4y ago
The problem with an autopilot and the reason some of are so worried is because if one fails, the others will fail in the same way. If one driver hits a bicyclist at a stop sign, we can assign the blame to the driver and figure out the cause i.e. driver inattentive, lack of reflective gear on bicycle. We can then take steps to mitigate this, driver education, requiring lights on bicycles. If an automated system hits a bicycle at a stop sign, we can try and do the same analysis but my understanding is, we have to interpret the data and might not actually discover the root cause, in which case we have no way of mitigating the failure. Add in the fact that the explanation is difficult or does not make sense to the general public(i.e. the body of the trailer was white and contrasted with the sky as the car went up the hill(public: anyone can see the difference between a trailer and the sky!)) We understand how and why people fail, we can empathize with them and assign blame to them. We don't understand how the computer is failing and can neither empathize with them nor are we actually assigning blame to them(This system is in beta; You must have you hands on the wheel at all time and be ready to take over; X is not liable for any damage caused by use of this system, the user assumes full liability etc.)
happytoexplain 4y ago
Humans understandably are extremely averse to the idea of dying for a stupid reason, especially if control over the death is shifted away from the person, in which case they are more "furious" than "averse". It's why we hate drunk drivers so much. If I introduce a car that reduces deaths to near zero, but the remaining deaths are caused by a robot arm that is programmed to pop out and stab the driver, people are not going to accept that. If my car reduces car deaths dramatically, but the remaining deaths are caused by the car killing you by accident in an extremely dramatic way that you are confident you would have avoided if you were driving, e.g. swerving off the road and driving at full speed into a median, or into a truck that you can see but the car can't, people are not going to accept that either. You can not shift a person's ability to protect themselves away from themselves and occasionally kill them with your solution simply because your system is better than the average across all humans, even if it's much better. It's frankly a little baffling that this part of human psychology even has to be explained. It's like the trolley problem: There seem to be some people who don't understand why you wouldn't simply pursue the result that holds fewer deaths - who don't understand why the trolley problem exists at all, what the point of it is. I suspect there is a relationship between these people and the people who don't understand how there could be any part of decision-making about autonomous vehicles apart from the raw numeric value of deaths vs non-autonomous vehicles.
- dxscorp 4y ago
  This is the reply among all discussion I've read about autopilot that captures my thoughts best, well said
- hcurtiss 4y ago
  Fair enough, but approaching the problem rationally, is that really how we should measure success? Are you satisfied with a solution that results in objectively more death just because you claim to have some part in creating it? That strikes me as ridiculous, and the outcome is that we're holding autonomous driving to an impossible standard.
  - RhysU 4y ago
    Thought experiments! Cancer is cured but 99% of its death toll is replaced by wholly random spontaneous human combustion. Is that better? What if it wasn't purely random but instead the chance of your spontaneous combustion slightly depended on if your socks are worn? Is that better? I posit both suck.

phpisthebest 4y ago

I have long said that if someone invited the car today it would never be approved for use Society is FAR FAR FAR FAR FAR more risk adverse which IMO is holding us back, we can not advance because perfection is impossible

mechagodzilla 4y ago

Many people are charged with crimes related to automotive accidents (DUI, for instance, or laws related to cell phone usage). We require drivers to carry insurance. We are constantly regulating things to reduce deaths and injuries in automotive accidents (seat belts, crumple zones, airbags, forced recalls, etc.) - we constantly think about automotive accidents when a human is behind the wheel. Why should we just shrug when some Tesla mows down a pedestrian while the driver is watching youtube videos?

ck2 4y ago
Always prosecute the driver. There's no such thing as "the car did it". And if someone wanted to intentionally murder someone, seems like a perfect excuse "the car did it". Unless we are going to victim blame.
- themaninthedark 4y ago
  But the point of the autopilot system is that there doesn't need to be a driver right? So here you are saying, "Hey, we have this great system that is such a better diver than the human. Here are the statistics that show it." and then turning around and saying "Hey human, why were you not a better driver. You need to be paying attention." A bit of wanting to have your cake and eat it too, isn't it?
  - ck2 4y ago
    "autopilot" is just a marketing term for the next decade, it's not fully automatic, it's actually "computer aided driving" (where driving is the human) but that description isn't sexy enough to sell
    - themaninthedark 4y ago
      The GGP talks about "something like autopilot" acceptance and the eventual goal of the system is to cut the human driver out. GP is talking about insurance and how we regulate drivers, stating that we should be looking at the same for autopilot systems. Parent says "Always prosecute the driver. There's no such thing as "the car did it"." At what point does the "autopilot" have any skin in the game so to speak? Is it a binary thing, one day we wake up to an update and the cars can drive themselves with no input? Or will will be forever stuck in a quasi beta test, where the driver is liable for the actions of the system but has no meaningful way for controlling the system(autopilot) or even knowing how the system operates. You can't have it both ways. Either the system is better than people and is going to take responsibility or it isn't. No one will accept a system they have no control over and do not understand but have to accept liability for.
      - ck2 4y ago
        There may come a day where someone is truly the passenger in the driver's seat in a car, like a person riding a bus. Today is not that day. Nor is it next year. Or this decade. A whole lot of people are going to die this decade, both drivers and victims hit, because marketing wants people to believe you don't have to pay attention once you turn it on rather it's an assist, not a replacement, for the driver. It's not autopilot right now. It's driver assist. As long as the driver exists, the driver is responsible. In an "old school" car, if an accelerator gets stuck and you mow someone down, then they find spaghetti code in the accelerator or it mechanically got stuck from design failure, you still have liability for killing someone but your prosecution won't be the same as someone who purposely drove over someone they were angry with.
  - drcongo 4y ago
- falcolas 4y ago
  You are technically correct for tesla vehicles (though there's some grey area around how Tesla markets things - you can't just handwave away legal liability for a death with "but the fine print!"), but not for self-driving vehicles in general. Waymo taxis, for example, explicitly does not have a driver. And Tesla has explicitly advertised that this Waymo scenario is where they're headed, so it's a question that's definitely worth thinking about now. "Blame the driver" isn't looking like a tenable position long term.
onelovetwo 4y ago
not sure how carrying insurance is related to accident prevention, Its mandatory in both cases. The amount of cases we charge people for accidents is extremely rare because in most cases its an actual accident. People swerving into oncoming traffic, not checking both sides before making a turn...etc.

dontreact 4y ago

It seems a bare minimum is releasing enough data that the public can assess whether it is better on average. Right now, Tesla releases data that show that when autopilot is engaged, it is safer on average. However, this data is not adjusted for the fact that the miles where users choose to activate it are on average safer. So really some sort of base-risk adjusted comparison would be the bare minimum I would like to see them publish. They really don't share much data.

icyfox 4y ago

Even if we accept some statistical failure rate, we should still investigate lethal failures. Like any good RCA it may uncover some learnings that could help the implementation and/or regulations around self driving.

addicted 4y ago

Yes we do. We have a ton of rules before, during and after humans are behind the wheel. You are correct that we should probably be thinking even more about it, and other countries are doing that, slowing down cars, providing separation from other modes of transport and pedestrians, and reducing car dependence altogether.

sonofhans 4y ago

> Is it not enough to be better than humans on average? No, not at all. The average driver is 1/3 distracted and 1/4 drunk. Autopilot and similar systems must be at least as good as a skilled, attentive, unimpaired driver. > Is the only way we will accept something like autopilot is if it has a perfect driving record? Putting it plainly, this is bullshit. No one -- no one at all -- is asking for autopilot to be perfect. People are asking for it not to drive full-speed into obvious obstacles.

themaninthedark 4y ago
>The average driver is 1/3 distracted and 1/4 drunk. Kinda curious where you are getting those numbers from.
kqr 4y ago
> People are asking for it not to drive full-speed into obvious obstacles. But isn't this a weird criterion? An obvious obstacle for a human might not be obvious to the computer -- but importantly, it could be the other way around too: the computer detects obstacles that are obvious to it, but a human would miss. Shouldn't the goal be that the computer considers more obstacles obvious than the human? I get the emotional argument that it's hard to convince a person off the street of this, but isn't HN a place for more nuanced debate?
- sonofhans 4y ago
  It seems like you're debating the meaning of "obvious" -- "What is obvious? And to whom?" My meaning was, "obvious to a skilled, attentive, unimpaired human." > Shouldn't the goal be that the computer considers more obstacles obvious than the human? That would be excellent, as long as it's correct :)
  - hcurtiss 4y ago
    In my view, the question should be whether it's more or less safe than humans driving. If use of automated driving results in less death than humans driving, considered at the broadest possible scale, then that strikes me as an excellent outcome. Instead, each and every death under autopilot results in all the hens clucking as though it's proof the technology has failed. That's ridiculous.

xeromal 4y ago

This is just an investigation which is important in cases like this. They need to determine if Teslas can't see motorcycle riders at 1am in Draper, Utah. There's all kinds of oddities that can happen so these independent investigations are necessary. Side note, imagine if you got bulldozed by a Tesla and died only to have someone write this comment. I know we live in a huge world and we can't think of everyone this way, but there should be a modicum of compassion for the guy who just died. His life has ended for what amounts to a potential programming error. It doesn't matter if another human could do it to. This man is actually dead. It's tragic.

boudin 4y ago

I really doubt Tesla is even close from being better than human for now, seems like it needs to be in really specific and simple situation to act properly. There's countless videos showing how clueless it is as soon as it is in an environment that is a bit more complex, including attempting to run into pedestrians. Most humans don't do that.

tqi 4y ago

For one, "better than average" is hard to measure / subject to lots of confounding factors, so people (rightfully) find those claims hard to believe. For another Autopilot often fails in different ways than human drivers, so it's easy to find examples where a human driver would have been fine, which exacerbates the perception that these are not in fact better.

rurp 4y ago

If you have proof that autopilot is safer I'd love to see it. Cherry picked and non-standard data selectively released from the company selling autopilot for thousands of dollars doesn't count :).

themaninthedark 4y ago

The problem with an autopilot and the reason some of are so worried is because if one fails, the others will fail in the same way. If one driver hits a bicyclist at a stop sign, we can assign the blame to the driver and figure out the cause i.e. driver inattentive, lack of reflective gear on bicycle. We can then take steps to mitigate this, driver education, requiring lights on bicycles. If an automated system hits a bicycle at a stop sign, we can try and do the same analysis but my understanding is, we have to interpret the data and might not actually discover the root cause, in which case we have no way of mitigating the failure. Add in the fact that the explanation is difficult or does not make sense to the general public(i.e. the body of the trailer was white and contrasted with the sky as the car went up the hill(public: anyone can see the difference between a trailer and the sky!)) We understand how and why people fail, we can empathize with them and assign blame to them. We don't understand how the computer is failing and can neither empathize with them nor are we actually assigning blame to them(This system is in beta; You must have you hands on the wheel at all time and be ready to take over; X is not liable for any damage caused by use of this system, the user assumes full liability etc.)

happytoexplain 4y ago

Humans understandably are extremely averse to the idea of dying for a stupid reason, especially if control over the death is shifted away from the person, in which case they are more "furious" than "averse". It's why we hate drunk drivers so much. If I introduce a car that reduces deaths to near zero, but the remaining deaths are caused by a robot arm that is programmed to pop out and stab the driver, people are not going to accept that. If my car reduces car deaths dramatically, but the remaining deaths are caused by the car killing you by accident in an extremely dramatic way that you are confident you would have avoided if you were driving, e.g. swerving off the road and driving at full speed into a median, or into a truck that you can see but the car can't, people are not going to accept that either. You can not shift a person's ability to protect themselves away from themselves and occasionally kill them with your solution simply because your system is better than the average across all humans, even if it's much better. It's frankly a little baffling that this part of human psychology even has to be explained. It's like the trolley problem: There seem to be some people who don't understand why you wouldn't simply pursue the result that holds fewer deaths - who don't understand why the trolley problem exists at all, what the point of it is. I suspect there is a relationship between these people and the people who don't understand how there could be any part of decision-making about autonomous vehicles apart from the raw numeric value of deaths vs non-autonomous vehicles.

dxscorp 4y ago
This is the reply among all discussion I've read about autopilot that captures my thoughts best, well said
hcurtiss 4y ago
Fair enough, but approaching the problem rationally, is that really how we should measure success? Are you satisfied with a solution that results in objectively more death just because you claim to have some part in creating it? That strikes me as ridiculous, and the outcome is that we're holding autonomous driving to an impossible standard.
- RhysU 4y ago
  Thought experiments! Cancer is cured but 99% of its death toll is replaced by wholly random spontaneous human combustion. Is that better? What if it wasn't purely random but instead the chance of your spontaneous combustion slightly depended on if your socks are worn? Is that better? I posit both suck.