5 ms·

The six levels of vehicle autonomy are marketing bullshit. They imply failure of automation is acceptable and the human can be left to pick up the slack. In re

by yholio 6y ago

The six levels of vehicle autonomy are marketing bullshit. They imply failure of automation is acceptable and the human can be left to pick up the slack. In reality, there is no spectrum of automated driving, with the human doing less and less. Driving is not an instantaneous activity like swimming, it requires planing and strategizing, the decisions made in the past influence situational awareness. I plan an overtake maneuver based on a myriad factors and have escape routes already planned if things don't go well. When the automated driver fails a human driver cannot be expected to just drop in and correct the mistakes - by that time, they are uncorectable and the ramp-up time of the human driver far exceeds the available time in most real life situations. In the overtake example, if the autopilot fails when a 20 ton truck is approaching, I have no recourse, I have no idea if an attempt to brake and regroup will be successful because I did not plan the maneuver. Therefore, computer drivers cannot be allowed to fail in such a scenario. In practice, there are only two real levels: 1. autonomous; and 0. non-autonomous, with various automation that helps the driver, while he remains fully in control and the sole decision maker. What can constitute a self-driving spectrum is the type of road where full automation is expected to work: from restricted and instrumented roads, where only other similar vehicles are allowed, for the dumbest self-driving modes, and up to full self-driving on general purpose roads mixed with human drivers and pedestrians. But the idea that you can achieve level 5 self driving by incrementally improving the systems designed to aid the driver is a dangerous pipe dream.

bob1029 6y ago
I still think general AI (i.e. AI so advanced that we would consider protecting each instance as we would a human life) is a prerequisite for truly autonomous driving on our complex roadways today. This is the kind of game theory that would make me feel more comfortable sharing the road with these kinds of cars. I completely agree with the assertion that the grey area in between is where a bunch of people are going to wind up dead.
- Aeolun 6y ago
  > I completely agree with the assertion that the grey area in between is where a bunch of people are going to wind up dead. But will more people end up dead than in the scenario where we had no ‘self-driving’ cars at all? Humans are much more accident prone than robots in all the research on this I’ve seen.
  - yholio 6y ago
    Truth is, we will never accept robots on the roads that are as deadly as human drivers. Especially since they will fail in specific, surprising and hard to comprehend ways, see here. Robotic drivers will probably need to demonstrate at least an order of magnitude improvement in safety before they can be accepted by the public.
  - toast0 6y ago
    The thing is, most of the deaths from self driving will be seen as preventable, if only a human was driving. That's going to make it look bad, even if the numbers are less. If you want to reduce deaths, I think you really want to invest in things like automatic emergency braking, monitoring driver attention, and safely stopping the vehicle if the driver is incapacitated. Having the computer supervise the human means the human is engaged in driving and aware of the circumstances for the most part; a computer supervising a human can act with great speed if the situation warrants, but a human supervising a computer is likely not to react so quickly.
    - deleted 6y ago
      [deleted]
  - j7ake 6y ago
    Calculating just deaths / DrivingDistance is easy but overly simplifies the complicated comparison between human- vs self-driving. Comparing normalized deaths is not sufficient to persuade real people to choose self-driving over human-driving. Real humans and society take into account many more factors to accurately compare the two driving modes. Take a hypothetical scenario where self-driving cars reduce deaths by 10x compared to now, but the deaths caused by self-driving cars are all small children and people with darker skin in human-preventable ways (you can imagine if cameras are used why this could happen). I doubt society would accept this tradeoff. The safer future is to have humans still drive, but be enhanced by computers and cameras that are installed in the car.
- yholio 6y ago
  Maybe not a prerequisite, but certainly necessary to prevent certain type of long-tailed events. For example, I can anticipate by the look of a neighborhood that it's more likely that there would be jaywalking and poor infrastructure, I can feel that the driver riding ahead is drunk or on drugs and keep my distance etc. It remains to be seen if a automated drivers can make-up for increased risk in such scenarios with good performance in areas they excel, such as reaction time, highly distributive attention, near perfect physical predictions and decisions made on that basis, vehicle-to-vehicle coordination and vehicle to road communication etc. What is clear though is that any practical system in the near term (non-general AI) will approach driving quite differently than a human driver. Hence the jury is still out of lidar vs camera-only approaches (Waymo vs Tesla).
- dboreham 6y ago
  You can probably achieve a safer-than-humans solution without strong/general AI, provided it only operates on a special kind of road (in-highway sensors, no pedestrians, all other vehicles are also self-driving).
slashink 6y ago
I hadn’t considered this before but you’re absolutely right. Driving is a continuous operation so it’s almost impossible to pick up without the prior context.
tantalor 6y ago
Your dichotomy may make sense in theory, but in practice where the average driver makes a fatal mistake at rate X, a level-4 autonomous vehicle with a failure rate Y<X (including when human override is required) is an win.
- darkerside 6y ago
  I think your claim is far more theoretical in nature. In a vacuum, sure, fewer deaths is better. In the real world, where people aren't satisfied unless someone can be held accountable in a clear way, we are far better off with humans driving.
  - Retric 6y ago
    The maker and or operator of a self driving car can be held accountable depending on the specifics. IMO, we have this odd social expectation that humans killing people with cars or pollution is basically acceptable and nothing else really gets that pass. IMO, killing someone with a car or a gun is a direct equivalent.
    - darkerside 6y ago
      It's equivalent, yes. When someone recklessly kills someone with a car, it's called vehicular manslaughter and they are punished for it. It's not considered acceptable. Blurring the lines of accountability is an anti-feature. The killer feature for humans is that we can be held accountable.
      - Retric 6y ago
        That’s assuming it’s prosecuted which is rare. However, just looking at the laws directly you notice the odd exclusion. In California for example vehicular manslaughter is specifically excluded from involuntary manslaughter and carries a lower penalty. It is also frequently just a misdemeanor.
deleted 6y ago
[deleted]
goodluckchuck 6y ago
This idea needs a little more fleshing out, but is very good. The Uber vehicle IIRC didn’t abandon control when it saw the pedestrian, it did so (if ever) when it determined that it couldn’t avoid them... So the vehicle had more and better information about the situation. “Failing to the driver only works when it’s not helpful.“
PiRho3141 6y ago
Here's the problem. The person was being paid to pay attention to the road. According to the below news article, the person was watching the voice on their mobile phone while the car was driving. Perfectly acceptable if the vehicle has already been rated to be fully autonomous but, in this case, they are still testing the automation and the car detected the bicyclists 5.6 seconds prior to the accident. Uber should be partially at fault since they deactivated the automatic emergency braking. Yes, I agree when vehicles become fully autonomous, people will be less attuned to the environment, but in this case, the person was hired specifically to pay attention at all times. https://news.sky.com/story/uber-safety-driver-charged-over-death-of-pedestrian-who-was-struck-by-self-driving-car-12072968 https://news.sky.com/story/uber-safety-driver-charged-over-d...
- runamok 6y ago
  FYI it was a bicyclist (singular) but they were walking their bike across a crosswalk at 10pm. Not to victim blame but if they were not wearing lights or a reflective vest I think a human could have made the same tragic error. As you point out the system actually DID detect them.
  - curiousgal 6y ago
    "Herzberg was crossing Mill Avenue (North) from west to east, approximately 360 feet (110 m) south of the intersection with Curry Road, outside the designated pedestrian crosswalk"
anticensor 6y ago
So, you think SAE J3016 (document that defines this 6-level terminology) is pointless and we could just get away without it? You need to prove your case further.
btbuildem 6y ago
Not everybody drives the same way -- from how you describe it, to you driving is an all-encompassing activity, requiring presence, planning, attention etc. Don't get me wrong, that's ideal. Unfortunately most people don't drive like that. For many people, it's an incidental activity, a means to an end, done with the bare minimum of resources required to get by. That's why there are so many "accidents" and why you see so much stupidity on the roads.

by yholio 6y ago

bob1029 6y ago

I still think general AI (i.e. AI so advanced that we would consider protecting each instance as we would a human life) is a prerequisite for truly autonomous driving on our complex roadways today. This is the kind of game theory that would make me feel more comfortable sharing the road with these kinds of cars. I completely agree with the assertion that the grey area in between is where a bunch of people are going to wind up dead.

Aeolun 6y ago
> I completely agree with the assertion that the grey area in between is where a bunch of people are going to wind up dead. But will more people end up dead than in the scenario where we had no ‘self-driving’ cars at all? Humans are much more accident prone than robots in all the research on this I’ve seen.
- yholio 6y ago
  Truth is, we will never accept robots on the roads that are as deadly as human drivers. Especially since they will fail in specific, surprising and hard to comprehend ways, see here. Robotic drivers will probably need to demonstrate at least an order of magnitude improvement in safety before they can be accepted by the public.
- toast0 6y ago
  The thing is, most of the deaths from self driving will be seen as preventable, if only a human was driving. That's going to make it look bad, even if the numbers are less. If you want to reduce deaths, I think you really want to invest in things like automatic emergency braking, monitoring driver attention, and safely stopping the vehicle if the driver is incapacitated. Having the computer supervise the human means the human is engaged in driving and aware of the circumstances for the most part; a computer supervising a human can act with great speed if the situation warrants, but a human supervising a computer is likely not to react so quickly.
  - deleted 6y ago
    [deleted]
- j7ake 6y ago
  Calculating just deaths / DrivingDistance is easy but overly simplifies the complicated comparison between human- vs self-driving. Comparing normalized deaths is not sufficient to persuade real people to choose self-driving over human-driving. Real humans and society take into account many more factors to accurately compare the two driving modes. Take a hypothetical scenario where self-driving cars reduce deaths by 10x compared to now, but the deaths caused by self-driving cars are all small children and people with darker skin in human-preventable ways (you can imagine if cameras are used why this could happen). I doubt society would accept this tradeoff. The safer future is to have humans still drive, but be enhanced by computers and cameras that are installed in the car.
yholio 6y ago
Maybe not a prerequisite, but certainly necessary to prevent certain type of long-tailed events. For example, I can anticipate by the look of a neighborhood that it's more likely that there would be jaywalking and poor infrastructure, I can feel that the driver riding ahead is drunk or on drugs and keep my distance etc. It remains to be seen if a automated drivers can make-up for increased risk in such scenarios with good performance in areas they excel, such as reaction time, highly distributive attention, near perfect physical predictions and decisions made on that basis, vehicle-to-vehicle coordination and vehicle to road communication etc. What is clear though is that any practical system in the near term (non-general AI) will approach driving quite differently than a human driver. Hence the jury is still out of lidar vs camera-only approaches (Waymo vs Tesla).
dboreham 6y ago
You can probably achieve a safer-than-humans solution without strong/general AI, provided it only operates on a special kind of road (in-highway sensors, no pedestrians, all other vehicles are also self-driving).

slashink 6y ago

I hadn’t considered this before but you’re absolutely right. Driving is a continuous operation so it’s almost impossible to pick up without the prior context.

tantalor 6y ago

Your dichotomy may make sense in theory, but in practice where the average driver makes a fatal mistake at rate X, a level-4 autonomous vehicle with a failure rate Y<X (including when human override is required) is an win.

darkerside 6y ago
I think your claim is far more theoretical in nature. In a vacuum, sure, fewer deaths is better. In the real world, where people aren't satisfied unless someone can be held accountable in a clear way, we are far better off with humans driving.
- Retric 6y ago
  The maker and or operator of a self driving car can be held accountable depending on the specifics. IMO, we have this odd social expectation that humans killing people with cars or pollution is basically acceptable and nothing else really gets that pass. IMO, killing someone with a car or a gun is a direct equivalent.
  - darkerside 6y ago
    It's equivalent, yes. When someone recklessly kills someone with a car, it's called vehicular manslaughter and they are punished for it. It's not considered acceptable. Blurring the lines of accountability is an anti-feature. The killer feature for humans is that we can be held accountable.
    - Retric 6y ago
      That’s assuming it’s prosecuted which is rare. However, just looking at the laws directly you notice the odd exclusion. In California for example vehicular manslaughter is specifically excluded from involuntary manslaughter and carries a lower penalty. It is also frequently just a misdemeanor.

deleted 6y ago

[deleted]

goodluckchuck 6y ago

This idea needs a little more fleshing out, but is very good. The Uber vehicle IIRC didn’t abandon control when it saw the pedestrian, it did so (if ever) when it determined that it couldn’t avoid them... So the vehicle had more and better information about the situation. “Failing to the driver only works when it’s not helpful.“

PiRho3141 6y ago

Here's the problem. The person was being paid to pay attention to the road. According to the below news article, the person was watching the voice on their mobile phone while the car was driving. Perfectly acceptable if the vehicle has already been rated to be fully autonomous but, in this case, they are still testing the automation and the car detected the bicyclists 5.6 seconds prior to the accident. Uber should be partially at fault since they deactivated the automatic emergency braking. Yes, I agree when vehicles become fully autonomous, people will be less attuned to the environment, but in this case, the person was hired specifically to pay attention at all times. https://news.sky.com/story/uber-safety-driver-charged-over-death-of-pedestrian-who-was-struck-by-self-driving-car-12072968 https://news.sky.com/story/uber-safety-driver-charged-over-d...

runamok 6y ago
FYI it was a bicyclist (singular) but they were walking their bike across a crosswalk at 10pm. Not to victim blame but if they were not wearing lights or a reflective vest I think a human could have made the same tragic error. As you point out the system actually DID detect them.
- curiousgal 6y ago
  "Herzberg was crossing Mill Avenue (North) from west to east, approximately 360 feet (110 m) south of the intersection with Curry Road, outside the designated pedestrian crosswalk"

anticensor 6y ago

So, you think SAE J3016 (document that defines this 6-level terminology) is pointless and we could just get away without it? You need to prove your case further.

btbuildem 6y ago

Not everybody drives the same way -- from how you describe it, to you driving is an all-encompassing activity, requiring presence, planning, attention etc. Don't get me wrong, that's ideal. Unfortunately most people don't drive like that. For many people, it's an incidental activity, a means to an end, done with the bare minimum of resources required to get by. That's why there are so many "accidents" and why you see so much stupidity on the roads.