14 ms·

Toyota Wants Cars to React to the Unexpected, and Then Explain Themselves

29 points by chewymouse 11y ago

ceejayoz 11y ago
"I'm sorry we're falling off a mountain right now, but it was us or that family."
- afarrell 11y ago
  I think that self-driving cars should act in the self interest of the owners. It will encourage people to use self-driving cars and decrease road deaths. Also, if you are at a rail yard and see something hurtling toward 5 workmen and can push a fat man in front to protect them, you should refrain. The workmen had the opportunity and duty to take actions to prepare for and prevent rogue train cars. The fat man should not have a duty to be constantly on guard against assault.
  - mikeash 11y ago
    People make this out to be far more complex than it needs to be. Just give the car a fast connection to the credit bureaus and have it minimize the sum of the FICO scores of those killed.
    - afarrell 11y ago
      ...huh? Oh, this comment is satire. Eh, I really don't think people on HN tend to explicitly value others according to wealth. The failure mode some people have is not being fully aware of the degree that a certain level of wealth or stability during childhood is a prerequisite for all but a very few to get to a solid technical [self-]education.
      - mikeash 11y ago
        It's satire, but not meant to be about HN readers. It's just an amusing way to cut the Gordian Knot and riff on society's treatment of the less well-off.
  - ceejayoz 11y ago
    > I think that self-driving cars should act in the self interest of the owners. To what extent? At some point, we're going to have to decide as a society what call a self-driving car has to make in situations like "I can run down this family in the cross-walk or get rear-ended by that oncoming vehicle".
    - zyxley 11y ago
      I'd rather see a focus on smarter and faster braking then on solving contrived moral dilemmas.
      - infecto 11y ago
        This please.
      - mikeash 11y ago
        I don't understand why this keeps coming up. I have never seen or even heard of a driver getting into a situation where there's any sort of moral dilemma in how they react. There's always an obvious best answer or a bunch of equivalently good answers. For the one-in-a-trillion cases where this is not true, it's OK if the car reacts sub-optimally. It will still be vastly better than human drivers, who frequently react sub-optimally in unambiguous situations.
      - Qworg 11y ago
        There are physical limits on how fast or smart you can make braking. We make thousands of unconscious decisions while driving - many of them moral/ambiguous. "Should I pass this car in the intersection even though it is breaking the law?" "Can I squeeze past this bicyclist even though it forces them towards the curb?" The core of most of these arguments assume a fully autonomous world - and it just won't work like that.
    - deleted 11y ago
      [deleted]
    - intopieces 11y ago
      When autonomous vehicles encounter an obstacle, they brake. Moreover, why in this problem, that appears in absolutely every thread about self-driving cars, is there a family in a crosswalk with no awareness whatsoever of a speeding vehicle? Are they deaf and blind without assistance? This is not a joke. The car will be aware of the presence of the crosswalk. It will slow long before arriving at it. So the deaf and blind family will likely be safe.
- lukaslalinsky 11y ago
  If you are in a situation where there is a risk of falling off a mountain and you are driving fast enough that you can't safely stop if you see people in front of you, you are doing something wrong. A self-driving car should definitely not even get into situations where it's forced to make decisions like that.
  - abakker 11y ago
    I know it is rare, but this situation can arise through no error of the car's controls. Consider the failure of the car's master cylinder. If your Brakes don't work, then this might be the exact choice you make. I don't want to sound overly cynical, but I somehow doubt that self-driving cars are going to suddenly make cars more reliable. Mechanical failure are not a common cause of crashes - http://www-nrd.nhtsa.dot.gov/pubs/812115.pdf http://www-nrd.nhtsa.dot.gov/pubs/812115.pdf - according to the NHTSA there, they account for 2% of crashes between 2005-2007. Still, self driving cars will need to be able to handle situations where there are mechanical failures.
    - cmdrfred 11y ago
      try: brakes() except BrakeFailure: emergency_brakes()
      - abakker 11y ago
        Yes, but have you ever stopped with your eBrake? first, it doesn't usually effect all 4 wheels, and second, it isn't antilock. If, traveling 60mph your master cylinder or brake lines fail, the E Brake might not do the job. I know the E Brake in my Toyota Tundra wouldn't stop me if I were going 60mph. Again, my point was not to say that this was likely to happen, but merely that we should ensure that there is framework for dealing with these kind of scenarios where a car needs to choose. According to the document I shared earlier, 95% of accidents are driver error somehow. Self-driving cars are sure to improve those statistics, but for public acceptance we need to answer the other 5% of accidents too. EDIT: as Saberworks below points out, there is no Emergency Brake in the Tundra, or in fact almost any modern car. I was loosely equating the parking brake with an emergency brake because in many cars there is no "emergency brake", so the parking brake is your only other choice.
shallowpedantic 11y ago
Self-driving cars are going to make people even lazier then they already are. And as for those cars that automatically park themselves - whats going to happen when 5 cars see only one spot, and they all try and go for that same spot?
- zyxley 11y ago
  > whats going to happen when 5 cars see only one spot, and they all try and go for that same spot? What happens when 5 human drivers see only one spot, and they all try and go for that same spot?
ryanmarsh 11y ago
Customers Want Cars to Not Kill Them FTFY
- protomyth 11y ago
  Pedestrians on the sidewalk want the car not run them over to avoid killing the Customer.
  - pc86 11y ago
    Fair point, but if the alternative is hitting something that has an as high or higher likelihood of killing the occupants, I'm not sure what the alternative is (don't say "avoid the situation in the first place," that's obvious). Nobody's going to buy an automated vehicle that might intentionally kill them to save a pedestrian.
    - protomyth 11y ago
      I think the alternative is pretty much a no go. If some poor pedestrian that was on a side gets hit by a car to preserve the owner then the lawsuits will be huge. Killing bystanders is not going to be acceptable. Let's look at a picture[1] from an article[2] that made it to HN[3]. If any program chose A as an acceptable solution, then they should be sued into oblivion. Killing the bystander will kill the driverless car. 1) http://www.technologyreview.com/sites/default/files/styles/view_body_embed/public/images/Ethical%20cars.png?itok=A_wye_WL http://www.technologyreview.com/sites/default/files/styles/v... 2) http://www.technologyreview.com/view/542626/why-self-driving-cars-must-be-programmed-to-kill/ http://www.technologyreview.com/view/542626/why-self-driving... 3) https://news.ycombinator.com/item?id=10439106 https://news.ycombinator.com/item?id=10439106
      - true_religion 11y ago
        I'd say these aren't simple systems. The "kill the pedestrian, not me" example is brought up... but what is a real life situation wherein the only two choices in retrospective was 'die' or kill a pedestrian? Most of the time, you'll avoid killing a pedestrian and it'll result in non-critical injuries to you due to airbags and other safety features. Or it'll result in a dodge or just car damage, since the other cars are more prepared to react to you than a hobbled pedestrian carrying groceries is. I think often times the car will 'choose' to risk injury you rather than absolutely a pedestrian.

29 points by chewymouse 11y ago

ceejayoz 11y ago

"I'm sorry we're falling off a mountain right now, but it was us or that family."

afarrell 11y ago
I think that self-driving cars should act in the self interest of the owners. It will encourage people to use self-driving cars and decrease road deaths. Also, if you are at a rail yard and see something hurtling toward 5 workmen and can push a fat man in front to protect them, you should refrain. The workmen had the opportunity and duty to take actions to prepare for and prevent rogue train cars. The fat man should not have a duty to be constantly on guard against assault.
- mikeash 11y ago
  People make this out to be far more complex than it needs to be. Just give the car a fast connection to the credit bureaus and have it minimize the sum of the FICO scores of those killed.
  - afarrell 11y ago
    ...huh? Oh, this comment is satire. Eh, I really don't think people on HN tend to explicitly value others according to wealth. The failure mode some people have is not being fully aware of the degree that a certain level of wealth or stability during childhood is a prerequisite for all but a very few to get to a solid technical [self-]education.
    - mikeash 11y ago
      It's satire, but not meant to be about HN readers. It's just an amusing way to cut the Gordian Knot and riff on society's treatment of the less well-off.
- ceejayoz 11y ago
  > I think that self-driving cars should act in the self interest of the owners. To what extent? At some point, we're going to have to decide as a society what call a self-driving car has to make in situations like "I can run down this family in the cross-walk or get rear-ended by that oncoming vehicle".
  - zyxley 11y ago
    I'd rather see a focus on smarter and faster braking then on solving contrived moral dilemmas.
    - infecto 11y ago
      This please.
    - mikeash 11y ago
      I don't understand why this keeps coming up. I have never seen or even heard of a driver getting into a situation where there's any sort of moral dilemma in how they react. There's always an obvious best answer or a bunch of equivalently good answers. For the one-in-a-trillion cases where this is not true, it's OK if the car reacts sub-optimally. It will still be vastly better than human drivers, who frequently react sub-optimally in unambiguous situations.
    - Qworg 11y ago
      There are physical limits on how fast or smart you can make braking. We make thousands of unconscious decisions while driving - many of them moral/ambiguous. "Should I pass this car in the intersection even though it is breaking the law?" "Can I squeeze past this bicyclist even though it forces them towards the curb?" The core of most of these arguments assume a fully autonomous world - and it just won't work like that.
  - deleted 11y ago
    [deleted]
  - intopieces 11y ago
    When autonomous vehicles encounter an obstacle, they brake. Moreover, why in this problem, that appears in absolutely every thread about self-driving cars, is there a family in a crosswalk with no awareness whatsoever of a speeding vehicle? Are they deaf and blind without assistance? This is not a joke. The car will be aware of the presence of the crosswalk. It will slow long before arriving at it. So the deaf and blind family will likely be safe.
lukaslalinsky 11y ago
If you are in a situation where there is a risk of falling off a mountain and you are driving fast enough that you can't safely stop if you see people in front of you, you are doing something wrong. A self-driving car should definitely not even get into situations where it's forced to make decisions like that.
- abakker 11y ago
  I know it is rare, but this situation can arise through no error of the car's controls. Consider the failure of the car's master cylinder. If your Brakes don't work, then this might be the exact choice you make. I don't want to sound overly cynical, but I somehow doubt that self-driving cars are going to suddenly make cars more reliable. Mechanical failure are not a common cause of crashes - http://www-nrd.nhtsa.dot.gov/pubs/812115.pdf http://www-nrd.nhtsa.dot.gov/pubs/812115.pdf - according to the NHTSA there, they account for 2% of crashes between 2005-2007. Still, self driving cars will need to be able to handle situations where there are mechanical failures.
  - cmdrfred 11y ago
    try: brakes() except BrakeFailure: emergency_brakes()
    - abakker 11y ago
      Yes, but have you ever stopped with your eBrake? first, it doesn't usually effect all 4 wheels, and second, it isn't antilock. If, traveling 60mph your master cylinder or brake lines fail, the E Brake might not do the job. I know the E Brake in my Toyota Tundra wouldn't stop me if I were going 60mph. Again, my point was not to say that this was likely to happen, but merely that we should ensure that there is framework for dealing with these kind of scenarios where a car needs to choose. According to the document I shared earlier, 95% of accidents are driver error somehow. Self-driving cars are sure to improve those statistics, but for public acceptance we need to answer the other 5% of accidents too. EDIT: as Saberworks below points out, there is no Emergency Brake in the Tundra, or in fact almost any modern car. I was loosely equating the parking brake with an emergency brake because in many cars there is no "emergency brake", so the parking brake is your only other choice.

shallowpedantic 11y ago

Self-driving cars are going to make people even lazier then they already are. And as for those cars that automatically park themselves - whats going to happen when 5 cars see only one spot, and they all try and go for that same spot?

zyxley 11y ago
> whats going to happen when 5 cars see only one spot, and they all try and go for that same spot? What happens when 5 human drivers see only one spot, and they all try and go for that same spot?

ryanmarsh 11y ago

Customers Want Cars to Not Kill Them FTFY

protomyth 11y ago
Pedestrians on the sidewalk want the car not run them over to avoid killing the Customer.
- pc86 11y ago
  Fair point, but if the alternative is hitting something that has an as high or higher likelihood of killing the occupants, I'm not sure what the alternative is (don't say "avoid the situation in the first place," that's obvious). Nobody's going to buy an automated vehicle that might intentionally kill them to save a pedestrian.
  - protomyth 11y ago
    I think the alternative is pretty much a no go. If some poor pedestrian that was on a side gets hit by a car to preserve the owner then the lawsuits will be huge. Killing bystanders is not going to be acceptable. Let's look at a picture[1] from an article[2] that made it to HN[3]. If any program chose A as an acceptable solution, then they should be sued into oblivion. Killing the bystander will kill the driverless car. 1) http://www.technologyreview.com/sites/default/files/styles/view_body_embed/public/images/Ethical%20cars.png?itok=A_wye_WL http://www.technologyreview.com/sites/default/files/styles/v... 2) http://www.technologyreview.com/view/542626/why-self-driving-cars-must-be-programmed-to-kill/ http://www.technologyreview.com/view/542626/why-self-driving... 3) https://news.ycombinator.com/item?id=10439106 https://news.ycombinator.com/item?id=10439106
    - true_religion 11y ago
      I'd say these aren't simple systems. The "kill the pedestrian, not me" example is brought up... but what is a real life situation wherein the only two choices in retrospective was 'die' or kill a pedestrian? Most of the time, you'll avoid killing a pedestrian and it'll result in non-critical injuries to you due to airbags and other safety features. Or it'll result in a dodge or just car damage, since the other cars are more prepared to react to you than a hobbled pedestrian carrying groceries is. I think often times the car will 'choose' to risk injury you rather than absolutely a pedestrian.