6 ms·

While I'm almost always bullish on new tech, self-driving cars are one where I frankly have a pretty hard time buying into the hype. I think when they really st

While I'm almost always bullish on new tech, self-driving cars are one where I frankly have a pretty hard time buying into the hype. I think when they really start to hit the streets, some issues are going to show up. Here's what I mean. Imagine this situation: You're pulling into your driveway in your self-driving car, and one of your kids left a lawn gnome there, blocking it. The car won't go forward because there's an obstacle. Simultaneously, a driver looking at their phone and not paying attention comes careening down the road. They're clearly going to rear-end you, hard. You COULD just pull forward and out of the way.....but the car won't, because of the lawn gnome. It can't know the lawn gnome doesn't matter in comparison to the damage your car is about to suffer. Imagine this flip scenario: You're pulling into your driveway in your self-driving car, and one of your kids, a toddler, is in your way. The car won't go forward because there's an obstacle. Otherwise, same situation as above. Now, you ABSOLUTELY do NOT want to pull forward, because you'd rather get rear-ended than run over your child. Even if you want to nitpick this exact scenario, surely you can think of a hundred similar contexts yourself, where driving involves understanding the value of different pieces of the car's immediate environment. A car cannot know the value of a child versus a lawn gnome, nor can it reliably detect the difference. Even if such a situation only arises once in a thousand days of driving, with hundreds of millions of people on the road everyday, that's thousands of times per day. That's really bad. Is it statistically better than the amount of harm done by human drivers? Possibly. But we all still get in cars believing we'll be better than other drivers. Who's going to get in a car when any given car has an equivalent chance of making a bad decision all on its own? It's just like getting in a car with a drunk driver. Who's going to sign up for that? Not me, that's for sure.

bjtitus 9y ago
I always question whether humans are really any better at this calculation. In the gnome example, many people will probably stop to avoid damaging their property, not see the car behind them careening forward, and end up getting rear ended anyway. In an ideal world, ethical issues like the one you presented would be major hurdles but humans aren't going to be perfectly rational in the moment of many accidents, especially the most dangerous high-speed ones when there is physically not enough time to do ethical math. > we all still get in cars believing we'll be better than other drivers. Who's going to get in a car when any given car has an equivalent chance of making a bad decision all on its own? Given the fact that plenty of people take taxis, rely on Uber and Lyft drivers they've never met, and take public transportation like aircraft and trains with little knowledge of the operator, this seems like a non-issue to me.
- oaktowner 9y ago
  The other thing to bear in mind: I would guess that the vast majority of accidents are caused by people not paying attention -- distracted by a phone, person, great view, or even just because they're looking the wrong direction. Cars don't do any of those things. They will always be paying attention, looking in all directions at once. So if they eliminate all of the accidents that are caused that way, but increase the number of garden-gnome related accidents...wouldn't that be worthwhile? Another, less flippant way to say it: yes, we can imagine scenarios in which the number of accidents will increase. But what is the relative frequency of those types of accidents compared to run-of-the-mill rear-end, red-light run, lane change due to inattentiveness?
  - frenchie4111 9y ago
    And even if the number of accidents increases: Does that increase cause greater monetary loss than the considerable cost savings of not paying drivers.
  - otakucode 9y ago
    If given a choice between changing from what you are used to which has drawbacks, to something new that has drawbacks, even if those drawbacks are markedly milder, almost everyone will choose to stick with where they already are. Consciously choosing a position which has drawbacks that you know about makes you feel personally responsible for those when they occur. That is infinitely worse than suffering the drawbacks of the "default" you stay with which you can give yourself a moral pass on because it wasn't a situation you chose to begin with. There's also the issue of what a self-driving car could do... and what they WILL do. Much technology COULD be amazing... if companies did not function in the ways that they actually do. Because of that, technology often fails to meet any of the promises made. Development is supposed to be cheap and fast, and safety and security get left at the door. No company is held liable for such things, and it would be expensive and bothersome to fix, so they're not about to bother.
- irq11 9y ago
  There's no ethical question here. You either see that it's a gnome, or you see that it's a child. Humans may or may not be less situationally aware overall, but we have the gnome question down cold. Cars don't. They see "obstacle in front of bumper radar." The OPs point is well-taken because it illustrates a technical risk that few are discussing: the "last 1%" of self-driving software may be equivalent to solving AGI.
sbov 9y ago
None of these contrived scenarios compares to the rate at which humans simply fail to stop when they should have stopped. I don't know anyone who got into or was rescued from an accident by pulling one of these maneuvers. I know plenty of people who got into accidents because someone (them or someone else) failed to stop when they should have stopped. Considering humans can barely stop when they should, I don't give them a high likelihood to also be able to have the situational awareness and make the multiple split second decisions required to both decide to, and take action upon, directing their car towards the least-valuable obstacle in an emergency. By the time they decide what to do they will probably be rear ended. Edit: your examples also ignores that with self driving cars, people won't be put in these situations in the first place. A driver not paying attention to the road, looking at their phone would be... not a safety hazard.
- irq11 9y ago
  As long as other cars (or humans, or animals...) are on the road, robot cars will face these scenarios. The OP's scenario is contrived to make a clear example, but it happens regularly when you're driving: you're forced to make a choice between acting and possibly hitting something, or doing nothing and being hit. We don't need self driving cars to solve the 80% case of preventing human drivers from hitting things. We need radar and auto-braking, which already exists. Full autonomy requires solving the harder problems.
joshualross 9y ago
There are 35 thousand deaths per year as a result of car crashes, and over 6 million total crashes[0]. If autonomous cars reduce those deaths to hundreds and overall crashes to thousands, I think you're underestimating the number of people that will sign up for that. [0]https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812318 https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...
giarc 9y ago
MIT (and I assume others) have collected data on these types of scenarios. For example, swerve and kill a bunch of people, or stay the course and kill a mom and baby? http://moralmachine.mit.edu/ http://moralmachine.mit.edu/
dirtyaura 9y ago
There is nothing hard about valuing gnome vs child, if you can identify them. Computers can evaluate huge problem spaces in milliseconds. The hard part is the perception: identifying the child and the gnome. Deep learning has allowed us to make gigantic leaps in the quality of perception but there is a long way to go in uncontrolled environments. I also think people incorrectly think that the moral choice dilemmas that self-driving cars will face are somehow the difficult part. What will happen is that we will provide valuation functions that are somewhat close to our intuitive collective understanding of value of life in different forms. Computers use them to evaluate these situations and we as a society just adapt to the new ethics of the AI age.
maxerickson 9y ago
What if the jackass on the phone was in a self driving car that just stopped?
otakucode 9y ago
I will get excited about a self-driving car on the day that an auto manufacturer fully opens up details about their development process - size of team, experience levels of team, breakdown of salaries, tools they used, methodologies used, etc. I also want a PERSONAL statement from every single engineer who worked on the project saying that they take personal responsibility for the safety of the vehicle and that every single one of their own concerns about safety or foreshortened schedules or testing were addressed to their full satisfaction. I also want to see the CEO, CTO, and other C-level execs of the auto company to release a legally binding agreement holding themselves personally liable for criminal negligence if the vehicle is later found to have obvious flaws - obvious determined by an independent group of several thousand unaffiliated volunteer experienced software engineers. Right now, if a car company slapped a 'self driving' car together that was designed to seek and destroy pre-schoolers, no one would say boo. Certainly not the courts. Toyota killed people by having ludicrously egregiously bad development practices (their developers did not even have access to a bug tracker... they followed 6 of 94 industry standard recommended practices... and worse) - and the courts acquitted them of negligence. That is disgusting, and makes it impossible for any reasonable person to trust any auto company doing development of something as significant and difficult as making a self-driving car. They know that they will not be held liable no matter how bad of a job they do - would you drive across a bridge built by bridge engineers hired, managed, and made to work the way software engineers are hired, managed, and made to work in modern companies?
deleted 9y ago
[deleted]
tigershark 9y ago
Hopefully in a not too far future you and all the others that are convinced that are better drivers will be forcefully signed up so your scenario can't possibly occur given that the other car will be self driving and always paying attention.
inverse_pi 9y ago
You're thinking about the tail end of the distribution. If self driving cars can solve 90% of the mass of the distribution, i.e. driving on highways and urban streets, it would be a significant improvement in terms of efficiency. Of course there will always be morale questions like the one you proposed. But quite frankly, the return on investment of solving the tail end scenarios like that is really small.
- bobsil1 9y ago
  You can't ship without restricting where the car goes, punting edge cases to remote drivers, or actually solving the tail end with AGI. Actual full self-driving is far more satisfying than "you can go from here to the airport" or "you can drive in downtown SF."
  - jvolkman 9y ago
    But actually shipping a product that can operate autonomously in a subset of areas and conditions is more satisfying than not shipping a product at all.
    - bobsil1 9y ago
      True, I’m a Model 3 fan :)
freyir 9y ago
The danger in this contrived scenario is still the human driver, who is distracted and careening down the road. If all the cars were autonomous, it would more likely to be a matter of frustration than safety: the gnome causes a traffic jam because the cars won't endanger the passenger by driving into the opposing lane. Then what? They find an alternate route? They automatically call in a service crew to clear the road? Not the end of the world.

bjtitus 9y ago

I always question whether humans are really any better at this calculation. In the gnome example, many people will probably stop to avoid damaging their property, not see the car behind them careening forward, and end up getting rear ended anyway. In an ideal world, ethical issues like the one you presented would be major hurdles but humans aren't going to be perfectly rational in the moment of many accidents, especially the most dangerous high-speed ones when there is physically not enough time to do ethical math. > we all still get in cars believing we'll be better than other drivers. Who's going to get in a car when any given car has an equivalent chance of making a bad decision all on its own? Given the fact that plenty of people take taxis, rely on Uber and Lyft drivers they've never met, and take public transportation like aircraft and trains with little knowledge of the operator, this seems like a non-issue to me.

oaktowner 9y ago
The other thing to bear in mind: I would guess that the vast majority of accidents are caused by people not paying attention -- distracted by a phone, person, great view, or even just because they're looking the wrong direction. Cars don't do any of those things. They will always be paying attention, looking in all directions at once. So if they eliminate all of the accidents that are caused that way, but increase the number of garden-gnome related accidents...wouldn't that be worthwhile? Another, less flippant way to say it: yes, we can imagine scenarios in which the number of accidents will increase. But what is the relative frequency of those types of accidents compared to run-of-the-mill rear-end, red-light run, lane change due to inattentiveness?
- frenchie4111 9y ago
  And even if the number of accidents increases: Does that increase cause greater monetary loss than the considerable cost savings of not paying drivers.
- otakucode 9y ago
  If given a choice between changing from what you are used to which has drawbacks, to something new that has drawbacks, even if those drawbacks are markedly milder, almost everyone will choose to stick with where they already are. Consciously choosing a position which has drawbacks that you know about makes you feel personally responsible for those when they occur. That is infinitely worse than suffering the drawbacks of the "default" you stay with which you can give yourself a moral pass on because it wasn't a situation you chose to begin with. There's also the issue of what a self-driving car could do... and what they WILL do. Much technology COULD be amazing... if companies did not function in the ways that they actually do. Because of that, technology often fails to meet any of the promises made. Development is supposed to be cheap and fast, and safety and security get left at the door. No company is held liable for such things, and it would be expensive and bothersome to fix, so they're not about to bother.
irq11 9y ago
There's no ethical question here. You either see that it's a gnome, or you see that it's a child. Humans may or may not be less situationally aware overall, but we have the gnome question down cold. Cars don't. They see "obstacle in front of bumper radar." The OPs point is well-taken because it illustrates a technical risk that few are discussing: the "last 1%" of self-driving software may be equivalent to solving AGI.

sbov 9y ago

None of these contrived scenarios compares to the rate at which humans simply fail to stop when they should have stopped. I don't know anyone who got into or was rescued from an accident by pulling one of these maneuvers. I know plenty of people who got into accidents because someone (them or someone else) failed to stop when they should have stopped. Considering humans can barely stop when they should, I don't give them a high likelihood to also be able to have the situational awareness and make the multiple split second decisions required to both decide to, and take action upon, directing their car towards the least-valuable obstacle in an emergency. By the time they decide what to do they will probably be rear ended. Edit: your examples also ignores that with self driving cars, people won't be put in these situations in the first place. A driver not paying attention to the road, looking at their phone would be... not a safety hazard.

irq11 9y ago
As long as other cars (or humans, or animals...) are on the road, robot cars will face these scenarios. The OP's scenario is contrived to make a clear example, but it happens regularly when you're driving: you're forced to make a choice between acting and possibly hitting something, or doing nothing and being hit. We don't need self driving cars to solve the 80% case of preventing human drivers from hitting things. We need radar and auto-braking, which already exists. Full autonomy requires solving the harder problems.

joshualross 9y ago

There are 35 thousand deaths per year as a result of car crashes, and over 6 million total crashes[0]. If autonomous cars reduce those deaths to hundreds and overall crashes to thousands, I think you're underestimating the number of people that will sign up for that. [0]https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/812318 https://crashstats.nhtsa.dot.gov/Api/Public/ViewPublication/...

giarc 9y ago

MIT (and I assume others) have collected data on these types of scenarios. For example, swerve and kill a bunch of people, or stay the course and kill a mom and baby? http://moralmachine.mit.edu/ http://moralmachine.mit.edu/

dirtyaura 9y ago

There is nothing hard about valuing gnome vs child, if you can identify them. Computers can evaluate huge problem spaces in milliseconds. The hard part is the perception: identifying the child and the gnome. Deep learning has allowed us to make gigantic leaps in the quality of perception but there is a long way to go in uncontrolled environments. I also think people incorrectly think that the moral choice dilemmas that self-driving cars will face are somehow the difficult part. What will happen is that we will provide valuation functions that are somewhat close to our intuitive collective understanding of value of life in different forms. Computers use them to evaluate these situations and we as a society just adapt to the new ethics of the AI age.

maxerickson 9y ago

What if the jackass on the phone was in a self driving car that just stopped?

otakucode 9y ago

I will get excited about a self-driving car on the day that an auto manufacturer fully opens up details about their development process - size of team, experience levels of team, breakdown of salaries, tools they used, methodologies used, etc. I also want a PERSONAL statement from every single engineer who worked on the project saying that they take personal responsibility for the safety of the vehicle and that every single one of their own concerns about safety or foreshortened schedules or testing were addressed to their full satisfaction. I also want to see the CEO, CTO, and other C-level execs of the auto company to release a legally binding agreement holding themselves personally liable for criminal negligence if the vehicle is later found to have obvious flaws - obvious determined by an independent group of several thousand unaffiliated volunteer experienced software engineers. Right now, if a car company slapped a 'self driving' car together that was designed to seek and destroy pre-schoolers, no one would say boo. Certainly not the courts. Toyota killed people by having ludicrously egregiously bad development practices (their developers did not even have access to a bug tracker... they followed 6 of 94 industry standard recommended practices... and worse) - and the courts acquitted them of negligence. That is disgusting, and makes it impossible for any reasonable person to trust any auto company doing development of something as significant and difficult as making a self-driving car. They know that they will not be held liable no matter how bad of a job they do - would you drive across a bridge built by bridge engineers hired, managed, and made to work the way software engineers are hired, managed, and made to work in modern companies?

deleted 9y ago

[deleted]

tigershark 9y ago

Hopefully in a not too far future you and all the others that are convinced that are better drivers will be forcefully signed up so your scenario can't possibly occur given that the other car will be self driving and always paying attention.

inverse_pi 9y ago

You're thinking about the tail end of the distribution. If self driving cars can solve 90% of the mass of the distribution, i.e. driving on highways and urban streets, it would be a significant improvement in terms of efficiency. Of course there will always be morale questions like the one you proposed. But quite frankly, the return on investment of solving the tail end scenarios like that is really small.

bobsil1 9y ago
You can't ship without restricting where the car goes, punting edge cases to remote drivers, or actually solving the tail end with AGI. Actual full self-driving is far more satisfying than "you can go from here to the airport" or "you can drive in downtown SF."
- jvolkman 9y ago
  But actually shipping a product that can operate autonomously in a subset of areas and conditions is more satisfying than not shipping a product at all.
  - bobsil1 9y ago
    True, I’m a Model 3 fan :)

freyir 9y ago

The danger in this contrived scenario is still the human driver, who is distracted and careening down the road. If all the cars were autonomous, it would more likely to be a matter of frustration than safety: the gnome causes a traffic jam because the cars won't endanger the passenger by driving into the opposing lane. Then what? They find an alternate route? They automatically call in a service crew to clear the road? Not the end of the world.