7 ms·

How much better than a human driver does a self-driving system need to be before we as a society will accept fully autonomous vehicles? That seems like the key

How much better than a human driver does a self-driving system need to be before we as a society will accept fully autonomous vehicles? That seems like the key question which we are spending very little effort trying to understand. Until we have a national conversation about this, I suspect self-driving will be an intractable problem.

kube-system 4y ago
A human driver is no longer the null hypothesis they have to beat. Most vehicles in the US are now sold with collision avoidance systems. The alternative to an autonomous vehicle is a vehicle with the combined efforts of humans + automated systems. Also, until there's a SAE level 5 vehicle in existence, we're comparing different sample sets. The situations humans crash most often in are many of the same situations where current autonomous vehicles simply won't operate at all. We may see these sample sets get even more skewed as we partially automate driving in levels 3 and 4. Because humans will have to drive in the more demanding scenarios where automated systems refuse to do so.
- TrevorJ 4y ago
  What I'm 'driving at' (hehe) here is that there's a lot of daylight between what the math says, and what people are likely to accept. We are, for better or worse, somewhat comfortable with the fact that a human driven semi truck will occasionally wipe out a minivan. It's not clear to me at all that we as a society are, or will ever be comfortable with a computer-operated semi-truck doing the same. No amount of "Well, the number say this truck is statistically safer when driven by a computer..." will move the needle on that, IMO.
  - kube-system 4y ago
    Well, there are different failure modes that we care about too. People think about first-party losses much differently than third-party losses. And, people currently take actions to mitigate those losses. Preventable crashes can be avoidable, by avoiding the situations in which humans engage in dangerous behavior. And the factor of control is why people are more comfortable with that failure mode, than a failure mode which may be more unpredictable. For instance: If I'm worried about killing my child in a first-party drunk driving crash, the average crash rate is irrelevant, because I can decrease the risk to zero by not drinking and driving. If I'm worried about third-party risk of a drunk driving crash, I can greatly mitigate that risk by choosing when and where I drive with my child in the car. > No amount of "Well, the number say this truck is statistically safer when driven by a computer..." will move the needle on that And I think the reason for that is that people subconsciously understand that the people who would be interested in (or financially capable of) purchasing an autonomous vehicle for the safety features are the same people who have lower accident rates than average.
    - Brendinooo 4y ago
      Nah, I think it’s more that people like to be in control. Planes are safer than cars, but plenty of people are afraid of flying. Maybe there’s something psychological about going hundreds of miles per hour thousands of feet in the air, but what I usually hear as the reason is that people don’t like that they’re not in control of the systems. (Also that media plays up plane crashes, but that would almost certainly be true of autonomous crashes as well.)
      - kube-system 4y ago
        That's what I meant by > the factor of control is why people are more comfortable with that failure mode, than a failure mode which may be more unpredictable. There are situations in which control mitigates risk, and there are situations in which it doesn't. While the average crash rate is relevant to public health, it's really irrelevant to someone who is dead.
CoastalCoder 4y ago
> Until we have a national conversation about this, I suspect self-driving will be an intractable problem. I hate to be a pessimist, but I'm not sure "national conversations" are all that useful in the U.S. Consider the "national conversation" leading up to Obamacare. NPR (at least) reported on the known pros/cons of various national healthcare systems around the world. AFAIK some of them were simply better than what we had, regardless of whether one looked at them through a conservative or liberal lens. And yet we still ended up with a system that was only marginally better IMHO.
- beebmam 4y ago
  For the ACA, whether it's better for you or worse for you depends on what income bracket you're in, whether you have preexisting conditions, and what kind of health care your job previously offered. Whether it's better or worse for the country is a complicated question that I don't have an answer to, and no one does. Because it's a nuanced answer, if you're being serious about quantifying "better". Unfortunately most people, across both political factions in the US, would rather adhere to their ideologies than think critically about things in this era.
- TrevorJ 4y ago
  I think it's easy to be biased towards agreeing with your point here because failures are much more visible in this realm than the successes. There's a whole host of things that we've 'worked out' on a national level. Schools, National Parks, Interstate Highway Systems, Fire Departments. All these things either didn't exist, or took hugely different forms only a couple generations ago, and now they are almost entirely accepted in their present form. We just don't think much about it now because this stuff is so accepted, even though at one point it was a whole big complicated thing that needed to get worked out and agreed on.
robonerd 4y ago
Beating the national average never should have been seen as the finish line. The average is dragged down into the gutter by a minority of drivers who choose to engage in very poor behavior, like driving drunk, speeding, making unnecessary trips at night and bad weather, street racing, etc. By simply avoiding these few behaviors, it's easy to put yourself well above the average. (inb4 "everybody thinks they're better than the average"; most people who think this are correct. Bad driving doesn't have a normal distribution.) If you want to impose self-driving on everybody, it should be at least as good as the best drivers. It would be a grave injustice to force it on the best drivers if it can't match their performance. As for the bar to pass for simply allowing, not mandating, self driving: I think they should first exceed the skill of a student driver. If they can't then they don't belong on the road without supervision, just like a student driver.
- dkarl 4y ago
  > If you want to impose self-driving on everybody, it should be at least as good as the best drivers. I would gladly have self-driving imposed on me if it were also imposed on drunks, teenagers, and the elderly. Sounds like a great trade-off.
- kube-system 4y ago
  You can see this in actuarial data that the IIHS posts. Often, cars that perform comparably in crash tests have wildly different real world loss rates. The difference is likely explainable by differences in driver behavior. For example, look at any car also sold under a hybrid trim model, the hybrid almost always has lower rates of injury. For an identical chassis. Or, consider the brands that have good crash test rating, but target subprime buyers.... in comparison to brands that have similar crash performance but target higher incomes and/or higher credit profiles. Nissan and Kia vehicles often have good crash test ratings, but they consistently have bad real-world crash data. https://www.iihs.org/ratings/insurance-losses-by-make-and-model https://www.iihs.org/ratings/insurance-losses-by-make-and-mo...
  - alphabettsy 4y ago
    I noticed the same thing. It seems pretty reasonable that the kind of person who drives a 350hp Volvo hybrid wagon is most often not the same kind of person who drives a 350hp Mustang.
polytely 4y ago
It also isn't a problem that's binary, I think we could very easily end up in a situation where the AI is a better driver 99% of the time but will occasionally cause an easily preventable accident. Who will be liable once your self-driving car runs over a kid because it mistook them for road markings? I have the feeling that people are going to be more OK with explainable human accidents than unexplainable AI accidents.
- TrevorJ 4y ago
  I agree with this 100%. To compound this issue is the fact that when I, as a person, make a mistake on the road, the problem space is limited to me. When a self driving car makes a mistake, every other vehicle on the road running that software/hardware is implicated as well. Imagine having to decide if all humans should have their license suspended until root cause analysis can be completed every single time a human is involved in a car crash. That is the problem we will need to grapple with when it comes to self driving cars.
kodah 4y ago
Personally, I'm kind of done having "national conversations", I don't think having all these fruitful perspectives has really driven us anywhere but mad. The people at the NHTSA need to be non-appointed SME's that heavily prioritize safety and balance efficiency and innovation below the safety threshold. The government then needs to respect them as an institution that protects people within its domain (drivers and pedestrians) and they need to either guide, force, or destroy violators.
r00fus 4y ago
I don't think real L4+ self-driving will be feasible until we change how the roads are designed to support such vehicles. Having clear indicators and markings that are maintained properly will not only help L4 but also L2 and even human drivers. So yes, it would require a standards-based solution - and that's where government involvement is essential.
- Robotbeat 4y ago
  A grade-separated (ie tunnel) system for autonomous vehicles would be one such method. Government can certainly assist in this, but industry is usually better at coming up with better standards which the government can then deploy
- jowday 4y ago
  Requiring dedicated infrastructure eliminates the point of self driving cars - you might as well just build public transit infrastructure. On top of that, this won’t eliminate any of the real edge cases - humans and other cars will still violate these rules and drive or walk across these roads, at which point the AV will still need to know how to handle this novelty. These are the main reasons AV companies don’t consider approaches like this.
  - r00fus 4y ago
    Nothing of what I said indicates dedicated infrastructure. In fact, I don't think that would work (as you say why not I'm thinking more like expanding priority bus-lanes (if you have visited regions outside the US these are common) and ensuring there are markers that are abundantly clear for everyone and more usable for self-driving vehicles.
    - jowday 4y ago
      I mean I’d count adding lane markers as building infrastructure - and the same caveat of people disobeying them applies. Not to mention you wouldn’t be able to add dedicated lanes outside of main thoroughfares and highways.
paxys 4y ago
The problem with this approach is that the average human driver includes teenagers, very old people who shouldn't be driving, drunk drivers, distracted drivers, speeders, reckless drivers and lots more categories whose accident risk is a LOT higher than the baseline. As someone who doesn't fall into any of these, I wouldn't accept a self driving system that's merely better than average, it has to be better than ME.
jowday 4y ago
This is something that robotaxi companies have already spent a ton of time figuring out - there are a ton of different metrics the cars have to hit before they remove the driver (as companies have done in Phoenix, San Francisco, Miami, and Austin). There are already robotaxis on the road right now.
jjav 4y ago
> How much better than a human driver does a self-driving system need to be before we as a society will accept fully autonomous vehicles? If you watch the many selfdriving videos online, it is clear the current state is worse than a drunken teenager on their first week with a license.

kube-system 4y ago

A human driver is no longer the null hypothesis they have to beat. Most vehicles in the US are now sold with collision avoidance systems. The alternative to an autonomous vehicle is a vehicle with the combined efforts of humans + automated systems. Also, until there's a SAE level 5 vehicle in existence, we're comparing different sample sets. The situations humans crash most often in are many of the same situations where current autonomous vehicles simply won't operate at all. We may see these sample sets get even more skewed as we partially automate driving in levels 3 and 4. Because humans will have to drive in the more demanding scenarios where automated systems refuse to do so.

TrevorJ 4y ago
What I'm 'driving at' (hehe) here is that there's a lot of daylight between what the math says, and what people are likely to accept. We are, for better or worse, somewhat comfortable with the fact that a human driven semi truck will occasionally wipe out a minivan. It's not clear to me at all that we as a society are, or will ever be comfortable with a computer-operated semi-truck doing the same. No amount of "Well, the number say this truck is statistically safer when driven by a computer..." will move the needle on that, IMO.
- kube-system 4y ago
  Well, there are different failure modes that we care about too. People think about first-party losses much differently than third-party losses. And, people currently take actions to mitigate those losses. Preventable crashes can be avoidable, by avoiding the situations in which humans engage in dangerous behavior. And the factor of control is why people are more comfortable with that failure mode, than a failure mode which may be more unpredictable. For instance: If I'm worried about killing my child in a first-party drunk driving crash, the average crash rate is irrelevant, because I can decrease the risk to zero by not drinking and driving. If I'm worried about third-party risk of a drunk driving crash, I can greatly mitigate that risk by choosing when and where I drive with my child in the car. > No amount of "Well, the number say this truck is statistically safer when driven by a computer..." will move the needle on that And I think the reason for that is that people subconsciously understand that the people who would be interested in (or financially capable of) purchasing an autonomous vehicle for the safety features are the same people who have lower accident rates than average.
  - Brendinooo 4y ago
    Nah, I think it’s more that people like to be in control. Planes are safer than cars, but plenty of people are afraid of flying. Maybe there’s something psychological about going hundreds of miles per hour thousands of feet in the air, but what I usually hear as the reason is that people don’t like that they’re not in control of the systems. (Also that media plays up plane crashes, but that would almost certainly be true of autonomous crashes as well.)
    - kube-system 4y ago
      That's what I meant by > the factor of control is why people are more comfortable with that failure mode, than a failure mode which may be more unpredictable. There are situations in which control mitigates risk, and there are situations in which it doesn't. While the average crash rate is relevant to public health, it's really irrelevant to someone who is dead.

CoastalCoder 4y ago

> Until we have a national conversation about this, I suspect self-driving will be an intractable problem. I hate to be a pessimist, but I'm not sure "national conversations" are all that useful in the U.S. Consider the "national conversation" leading up to Obamacare. NPR (at least) reported on the known pros/cons of various national healthcare systems around the world. AFAIK some of them were simply better than what we had, regardless of whether one looked at them through a conservative or liberal lens. And yet we still ended up with a system that was only marginally better IMHO.

beebmam 4y ago
For the ACA, whether it's better for you or worse for you depends on what income bracket you're in, whether you have preexisting conditions, and what kind of health care your job previously offered. Whether it's better or worse for the country is a complicated question that I don't have an answer to, and no one does. Because it's a nuanced answer, if you're being serious about quantifying "better". Unfortunately most people, across both political factions in the US, would rather adhere to their ideologies than think critically about things in this era.
TrevorJ 4y ago
I think it's easy to be biased towards agreeing with your point here because failures are much more visible in this realm than the successes. There's a whole host of things that we've 'worked out' on a national level. Schools, National Parks, Interstate Highway Systems, Fire Departments. All these things either didn't exist, or took hugely different forms only a couple generations ago, and now they are almost entirely accepted in their present form. We just don't think much about it now because this stuff is so accepted, even though at one point it was a whole big complicated thing that needed to get worked out and agreed on.

robonerd 4y ago

Beating the national average never should have been seen as the finish line. The average is dragged down into the gutter by a minority of drivers who choose to engage in very poor behavior, like driving drunk, speeding, making unnecessary trips at night and bad weather, street racing, etc. By simply avoiding these few behaviors, it's easy to put yourself well above the average. (inb4 "everybody thinks they're better than the average"; most people who think this are correct. Bad driving doesn't have a normal distribution.) If you want to impose self-driving on everybody, it should be at least as good as the best drivers. It would be a grave injustice to force it on the best drivers if it can't match their performance. As for the bar to pass for simply allowing, not mandating, self driving: I think they should first exceed the skill of a student driver. If they can't then they don't belong on the road without supervision, just like a student driver.

dkarl 4y ago
> If you want to impose self-driving on everybody, it should be at least as good as the best drivers. I would gladly have self-driving imposed on me if it were also imposed on drunks, teenagers, and the elderly. Sounds like a great trade-off.
kube-system 4y ago
You can see this in actuarial data that the IIHS posts. Often, cars that perform comparably in crash tests have wildly different real world loss rates. The difference is likely explainable by differences in driver behavior. For example, look at any car also sold under a hybrid trim model, the hybrid almost always has lower rates of injury. For an identical chassis. Or, consider the brands that have good crash test rating, but target subprime buyers.... in comparison to brands that have similar crash performance but target higher incomes and/or higher credit profiles. Nissan and Kia vehicles often have good crash test ratings, but they consistently have bad real-world crash data. https://www.iihs.org/ratings/insurance-losses-by-make-and-model https://www.iihs.org/ratings/insurance-losses-by-make-and-mo...
- alphabettsy 4y ago
  I noticed the same thing. It seems pretty reasonable that the kind of person who drives a 350hp Volvo hybrid wagon is most often not the same kind of person who drives a 350hp Mustang.

polytely 4y ago

It also isn't a problem that's binary, I think we could very easily end up in a situation where the AI is a better driver 99% of the time but will occasionally cause an easily preventable accident. Who will be liable once your self-driving car runs over a kid because it mistook them for road markings? I have the feeling that people are going to be more OK with explainable human accidents than unexplainable AI accidents.

TrevorJ 4y ago
I agree with this 100%. To compound this issue is the fact that when I, as a person, make a mistake on the road, the problem space is limited to me. When a self driving car makes a mistake, every other vehicle on the road running that software/hardware is implicated as well. Imagine having to decide if all humans should have their license suspended until root cause analysis can be completed every single time a human is involved in a car crash. That is the problem we will need to grapple with when it comes to self driving cars.

kodah 4y ago

Personally, I'm kind of done having "national conversations", I don't think having all these fruitful perspectives has really driven us anywhere but mad. The people at the NHTSA need to be non-appointed SME's that heavily prioritize safety and balance efficiency and innovation below the safety threshold. The government then needs to respect them as an institution that protects people within its domain (drivers and pedestrians) and they need to either guide, force, or destroy violators.

r00fus 4y ago

I don't think real L4+ self-driving will be feasible until we change how the roads are designed to support such vehicles. Having clear indicators and markings that are maintained properly will not only help L4 but also L2 and even human drivers. So yes, it would require a standards-based solution - and that's where government involvement is essential.

Robotbeat 4y ago
A grade-separated (ie tunnel) system for autonomous vehicles would be one such method. Government can certainly assist in this, but industry is usually better at coming up with better standards which the government can then deploy
jowday 4y ago
Requiring dedicated infrastructure eliminates the point of self driving cars - you might as well just build public transit infrastructure. On top of that, this won’t eliminate any of the real edge cases - humans and other cars will still violate these rules and drive or walk across these roads, at which point the AV will still need to know how to handle this novelty. These are the main reasons AV companies don’t consider approaches like this.
- r00fus 4y ago
  Nothing of what I said indicates dedicated infrastructure. In fact, I don't think that would work (as you say why not I'm thinking more like expanding priority bus-lanes (if you have visited regions outside the US these are common) and ensuring there are markers that are abundantly clear for everyone and more usable for self-driving vehicles.
  - jowday 4y ago
    I mean I’d count adding lane markers as building infrastructure - and the same caveat of people disobeying them applies. Not to mention you wouldn’t be able to add dedicated lanes outside of main thoroughfares and highways.

paxys 4y ago

The problem with this approach is that the average human driver includes teenagers, very old people who shouldn't be driving, drunk drivers, distracted drivers, speeders, reckless drivers and lots more categories whose accident risk is a LOT higher than the baseline. As someone who doesn't fall into any of these, I wouldn't accept a self driving system that's merely better than average, it has to be better than ME.

jowday 4y ago

This is something that robotaxi companies have already spent a ton of time figuring out - there are a ton of different metrics the cars have to hit before they remove the driver (as companies have done in Phoenix, San Francisco, Miami, and Austin). There are already robotaxis on the road right now.

jjav 4y ago

> How much better than a human driver does a self-driving system need to be before we as a society will accept fully autonomous vehicles? If you watch the many selfdriving videos online, it is clear the current state is worse than a drunken teenager on their first week with a license.