5 ms·

When are we going to admit that the current level of understanding and technical capabilities are not sufficient for safe autonomous driving?

fallingknife 2y ago
First we need to admit that the current level of human capabilities are not sufficient for safe driving.
arethuza 2y ago
To be fair, I've seen many human drivers doing equally stupid things.
- Timshel 2y ago
  Yes tired, drunk or raging people do stupid shit. But the goal is not exactly to have autonomous driving at their level ...
  - danielbarla 2y ago
    There are a few more categories of people who often do stupid shit, many of which are fairly mundane and far less easy to disregard (such as: momentarily distracted / semi-permanently fiddling with their phone / dangerously incompetent). I don't think it's unfair to point out that humans, quite often, also suck quite badly at driving.
    - Timshel 2y ago
      In general, I don't think human sucks at driving, but there are many times when any driver can be extremely dangerous for many reasons. Many of those reasons are illegal (alcohol, fiddling with your phone) so it's not a question of fairness an automated program was supposed to be better than that. I don't see anything unfair in requiring that automated driving which is supposed to not be afflicted from our human failing to not perform illegal and dangerous driving ... Especially since autonomous driving might bring more car not less (similar to Uber and Lift impact https://www.nature.com/articles/s41893-020-00678-z#Sec8 https://www.nature.com/articles/s41893-020-00678-z#Sec8 )
      - danielbarla 2y ago
        My take is that both have spectacular edge cases / failure modes that are essentially laughable from the other's point of view. It's hard to compare the two, when all we're doing is cherry picking scenarios. I could point to a double pedestrian crossing near my place where cars routinely speed. Various people have told me that the drivers cannot see the pedestrians, for a variety of reasons (buildings, cars, and especially a large tree near one corner), yet the human drivers are confident to go 10-20 kph over the speed limit through the area, despite this lack of information. I can give many similar examples - people really do suck at driving, they just tend to get away with it most of the time because rare events are, well, rare. AI sucks for a variety of other reasons. The only real question is whether on average, over all scenarios and ratios of humans to AIs populating the roads, one sucks more than the other. It's something I don't think we have the answer to.
  - arethuza 2y ago
    The point I was trying to make, clearly not very well, was that nobody would be surprised by a human making a mistake like this so why are we expecting perfection from a software controlled car? They will make mistakes but I'd also expect these mistakes to become less and less common. I'd only worry if the rate of change of performance isn't fairly positive...
    - Timshel 2y ago
      Don't know even by low human standard it's bad. I would not expect a human to drive two blocks while failing to go back to the correct lane; I would expect him to realize after 50m that there are too many and slow down and go back behind the uni cycle. In the same way I do not expect a human driver to stop after hitting someone then start again and drag the body with it. The problem is that it's easy to pump the statistic while driving miles without issue then it utterly fails when the situation is not well-defined anymore.
    - _xivi 2y ago
      > They will make mistakes Machines don't make "mistakes". A mistake require intent. In case of autonomous driving, any problem is a defect in the system that the whole fleet share. > I'd also expect these mistakes to become less and less common So the people should just tolerate it in the mean time "for the greater good"? oh please, it's an enterprise.
- logifail 2y ago
  > I've seen many human drivers doing equally stupid things Just yesterday I got yelled at by an Italian police officer who was directing traffic at a busy junction (I was in a rental car). [Full disclosure: I stopped, wound my window down, we had a polite conversation, everything turned out fine...] The key difference is that if I screw up badly enough when driving, I risk a monetary fine and/or losing my licence and/or getting sent to prison. For most human drivers these represent clear incentives not to screw up when driving. Questions: Have we agreed on suitable penalties when an autonomous vehicle screws up, given there isn't a single human driver to sanction? Could the fleet get grounded? Could the company get fined? Could relevant executives go to prison?
  - kalleboo 2y ago
    > Could the fleet get grounded? Cruise's permit was suspended after their accident
    - logifail 2y ago
      There wasn't just the one Cruise incident, though, was there?
- blibble 2y ago
  I'd expect to be prosecuted if I was caught driving like this, and probably lose my license as a result what is the threshold for permanently terminating Waymo's license? - an example like this of dangerous driving? - 1 death by dangerous driving? - 10 deaths by dangerous driving? (say it veered right in this video into the other road users) - 100 deaths by dangerous driving? (say it veers into a sidewalk full of pedestrians)
  - miki123211 2y ago
    The problem with these self-driving companies is that they drive a lot more miles than a human ever could. Even if their error rates were 1/10th of the most cautious human drivers, if we held them to the same standards as we hold humans, they'd lose their licenses in a week. This is due to the sheer number of cars they have and miles they drive. This is a variation on the trolley problem. Would you rather have roads that are 10 times safer, with 10 times fewer accidents and 10 times fewer deaths, but where the accidents are completely unexplainable and where nobody can be held guilty, or would you rather have the world stay as it is?
    - peteradio 2y ago
      If Waymo sold the software/hardware without onerous licensing costs I would rather live in world #2. If instead they become an incredibly wealthy lobbying engine based on extraction for a few rather than true wealth creation then I'd rather still live in world #1.
  - acimim_ha 2y ago
    Negative publicity is reflected on their equity evaluation. They have great incentives. The system works. I think with 100 deaths the company would get liquidated. This incident alone probably caused more damage than a cost of one single driver's license suspension.
  - viraptor 2y ago
    Why would you expect losing a license? The only actual issue I can see is crossing the double line which is just one penalty point if caught https://www.shouselaw.com/ca/defense/vehicle-code/21460/ https://www.shouselaw.com/ca/defense/vehicle-code/21460/ (Possibly also not using the indicator which is another point, but I don't think we can be 100% sure it wasn't used before the change started - California seems to require it only before not during lane change if I read it correctly)
Mashimo 2y ago
That's why they still are in research and not widely available.
- NewJazz 2y ago
  Why are they conducting experimental research on public roads in densely populated cities?
  - Mashimo 2y ago
    Probably because it's cheap and gives the most feedback.
  - dageshi 2y ago
    Why do we allow human trials of new drugs?
    - interloxia 2y ago
      https://en.m.wikipedia.org/wiki/Human_challenge_study https://en.m.wikipedia.org/wiki/Human_challenge_study
    - deleted 2y ago
      [deleted]
    - NewJazz 2y ago
      Consent
      - dageshi 2y ago
        The people surrounding the trial participant didn't give consent, if the drug causes a violent psychosis which in turn causes people to be injured by the participant, how does that differ?
        NewJazz 2y ago
        You are quickly moving the goalposts I see. Have fun with that exercise.
dotancohen 2y ago
Probably about the time that we realize 100% safety is an impossible goal, and better-than-human safety is achievable. I fully admit to having done things in the car far dumber and more dangerous than this when I was young. I'm not excusing the Waymo, rather I'm putting it in perspective.
rufus_foreman 2y ago
One alternative to autonomous driving in San Francisco: https://www.youtube.com/watch?v=9HoqFlDuuBI https://www.youtube.com/watch?v=9HoqFlDuuBI
scoofy 2y ago
I've said it over and over: the American automobile transit system is inherently unsafe. That's why I've been a self driving skeptic. We have a system where there drivers must break the rules and/or drive in an unsafe manner regularly. This involves: excessive speed when others are driving too fast, entering the oncoming lane when needing to pass someone double parked, making blind turns because a big truck is blocking your view, not knowing when a pedestrian will enter a crosswalk and needing to slam on the brakes, having to yield when crossing a lane of traffic... the list goes on and on. There isn't an actual safe set of rules, and the rules we have are so commonly broken that a learning algorithm won't follow them. I'd put this inherent danger at somewhere around 0.1% of the time. I'd say I'm forced to drive dangerously about once every 10 miles of driving, but the fact that people must drive dangerously occasionally to use the system means that a machine can't ever learn to use it safely.

fallingknife 2y ago

First we need to admit that the current level of human capabilities are not sufficient for safe driving.

arethuza 2y ago

To be fair, I've seen many human drivers doing equally stupid things.

Timshel 2y ago
Yes tired, drunk or raging people do stupid shit. But the goal is not exactly to have autonomous driving at their level ...
- danielbarla 2y ago
  There are a few more categories of people who often do stupid shit, many of which are fairly mundane and far less easy to disregard (such as: momentarily distracted / semi-permanently fiddling with their phone / dangerously incompetent). I don't think it's unfair to point out that humans, quite often, also suck quite badly at driving.
  - Timshel 2y ago
    In general, I don't think human sucks at driving, but there are many times when any driver can be extremely dangerous for many reasons. Many of those reasons are illegal (alcohol, fiddling with your phone) so it's not a question of fairness an automated program was supposed to be better than that. I don't see anything unfair in requiring that automated driving which is supposed to not be afflicted from our human failing to not perform illegal and dangerous driving ... Especially since autonomous driving might bring more car not less (similar to Uber and Lift impact https://www.nature.com/articles/s41893-020-00678-z#Sec8 https://www.nature.com/articles/s41893-020-00678-z#Sec8 )
    - danielbarla 2y ago
      My take is that both have spectacular edge cases / failure modes that are essentially laughable from the other's point of view. It's hard to compare the two, when all we're doing is cherry picking scenarios. I could point to a double pedestrian crossing near my place where cars routinely speed. Various people have told me that the drivers cannot see the pedestrians, for a variety of reasons (buildings, cars, and especially a large tree near one corner), yet the human drivers are confident to go 10-20 kph over the speed limit through the area, despite this lack of information. I can give many similar examples - people really do suck at driving, they just tend to get away with it most of the time because rare events are, well, rare. AI sucks for a variety of other reasons. The only real question is whether on average, over all scenarios and ratios of humans to AIs populating the roads, one sucks more than the other. It's something I don't think we have the answer to.
- arethuza 2y ago
  The point I was trying to make, clearly not very well, was that nobody would be surprised by a human making a mistake like this so why are we expecting perfection from a software controlled car? They will make mistakes but I'd also expect these mistakes to become less and less common. I'd only worry if the rate of change of performance isn't fairly positive...
  - Timshel 2y ago
    Don't know even by low human standard it's bad. I would not expect a human to drive two blocks while failing to go back to the correct lane; I would expect him to realize after 50m that there are too many and slow down and go back behind the uni cycle. In the same way I do not expect a human driver to stop after hitting someone then start again and drag the body with it. The problem is that it's easy to pump the statistic while driving miles without issue then it utterly fails when the situation is not well-defined anymore.
  - _xivi 2y ago
    > They will make mistakes Machines don't make "mistakes". A mistake require intent. In case of autonomous driving, any problem is a defect in the system that the whole fleet share. > I'd also expect these mistakes to become less and less common So the people should just tolerate it in the mean time "for the greater good"? oh please, it's an enterprise.
logifail 2y ago
> I've seen many human drivers doing equally stupid things Just yesterday I got yelled at by an Italian police officer who was directing traffic at a busy junction (I was in a rental car). [Full disclosure: I stopped, wound my window down, we had a polite conversation, everything turned out fine...] The key difference is that if I screw up badly enough when driving, I risk a monetary fine and/or losing my licence and/or getting sent to prison. For most human drivers these represent clear incentives not to screw up when driving. Questions: Have we agreed on suitable penalties when an autonomous vehicle screws up, given there isn't a single human driver to sanction? Could the fleet get grounded? Could the company get fined? Could relevant executives go to prison?
- kalleboo 2y ago
  > Could the fleet get grounded? Cruise's permit was suspended after their accident
  - logifail 2y ago
    There wasn't just the one Cruise incident, though, was there?
blibble 2y ago
I'd expect to be prosecuted if I was caught driving like this, and probably lose my license as a result what is the threshold for permanently terminating Waymo's license? - an example like this of dangerous driving? - 1 death by dangerous driving? - 10 deaths by dangerous driving? (say it veered right in this video into the other road users) - 100 deaths by dangerous driving? (say it veers into a sidewalk full of pedestrians)
- miki123211 2y ago
  The problem with these self-driving companies is that they drive a lot more miles than a human ever could. Even if their error rates were 1/10th of the most cautious human drivers, if we held them to the same standards as we hold humans, they'd lose their licenses in a week. This is due to the sheer number of cars they have and miles they drive. This is a variation on the trolley problem. Would you rather have roads that are 10 times safer, with 10 times fewer accidents and 10 times fewer deaths, but where the accidents are completely unexplainable and where nobody can be held guilty, or would you rather have the world stay as it is?
  - peteradio 2y ago
    If Waymo sold the software/hardware without onerous licensing costs I would rather live in world #2. If instead they become an incredibly wealthy lobbying engine based on extraction for a few rather than true wealth creation then I'd rather still live in world #1.
- acimim_ha 2y ago
  Negative publicity is reflected on their equity evaluation. They have great incentives. The system works. I think with 100 deaths the company would get liquidated. This incident alone probably caused more damage than a cost of one single driver's license suspension.
- viraptor 2y ago
  Why would you expect losing a license? The only actual issue I can see is crossing the double line which is just one penalty point if caught https://www.shouselaw.com/ca/defense/vehicle-code/21460/ https://www.shouselaw.com/ca/defense/vehicle-code/21460/ (Possibly also not using the indicator which is another point, but I don't think we can be 100% sure it wasn't used before the change started - California seems to require it only before not during lane change if I read it correctly)

Mashimo 2y ago

That's why they still are in research and not widely available.

NewJazz 2y ago
Why are they conducting experimental research on public roads in densely populated cities?
- Mashimo 2y ago
  Probably because it's cheap and gives the most feedback.
- dageshi 2y ago
  Why do we allow human trials of new drugs?
  - interloxia 2y ago
    https://en.m.wikipedia.org/wiki/Human_challenge_study https://en.m.wikipedia.org/wiki/Human_challenge_study
  - deleted 2y ago
    [deleted]
  - NewJazz 2y ago
    Consent
    - dageshi 2y ago
      The people surrounding the trial participant didn't give consent, if the drug causes a violent psychosis which in turn causes people to be injured by the participant, how does that differ?
      - NewJazz 2y ago
        You are quickly moving the goalposts I see. Have fun with that exercise.

dotancohen 2y ago

Probably about the time that we realize 100% safety is an impossible goal, and better-than-human safety is achievable. I fully admit to having done things in the car far dumber and more dangerous than this when I was young. I'm not excusing the Waymo, rather I'm putting it in perspective.

rufus_foreman 2y ago

One alternative to autonomous driving in San Francisco: https://www.youtube.com/watch?v=9HoqFlDuuBI https://www.youtube.com/watch?v=9HoqFlDuuBI

scoofy 2y ago

I've said it over and over: the American automobile transit system is inherently unsafe. That's why I've been a self driving skeptic. We have a system where there drivers must break the rules and/or drive in an unsafe manner regularly. This involves: excessive speed when others are driving too fast, entering the oncoming lane when needing to pass someone double parked, making blind turns because a big truck is blocking your view, not knowing when a pedestrian will enter a crosswalk and needing to slam on the brakes, having to yield when crossing a lane of traffic... the list goes on and on. There isn't an actual safe set of rules, and the rules we have are so commonly broken that a learning algorithm won't follow them. I'd put this inherent danger at somewhere around 0.1% of the time. I'd say I'm forced to drive dangerously about once every 10 miles of driving, but the fact that people must drive dangerously occasionally to use the system means that a machine can't ever learn to use it safely.