4 ms·

My general expectation is that Autonomous vehicles will at some point become safer than human drivers. However, even when that happens, when they make errors t

My general expectation is that Autonomous vehicles will at some point become safer than human drivers. However, even when that happens, when they make errors they will be errors that seem stupid and incomprehensible to humans. This will be because the failure mode of an ML driving model will be utterly different to the failure mode of human beings. This will lead to a lot of fear and uncertainty, humans want to comprehend what might kill them, and autonomous driving, while safer will kill them in bizarre ways.

Whatarethese 7y ago
I have the same assumption when a dumb human is tailgating and rear ends the car in from of them. That has been solved with frontal radar and emergency braking.
- sokoloff 7y ago
  “been solved”? What percentage of the overall fleet is equipped for automated frontal collision avoidance? Under 5%? Under 2%?
  - standyro 7y ago
    Every Model 3 has a forward collision avoidance that has a better reaction time than I do. I know because it's saved my life already when I narrowly avoided a suddenly stopped car in front of me. Tesla's Autopilot (not full self driving) has its issues, mainly with vague lane markings, but collision avoidance isn't one of them. I know plenty of drivers who fall victim and cause accidents because of the same issues that Autopilot has: poor lane markings, not seeing cars that are immobile on the side of the highway.
    - sokoloff 7y ago
      There are over 1000 times more "not Model 3"s in the US than Model 3s. That was my only point about "this has been solved". It's not a knock against the Model 3.
wruza 7y ago
Communication plays a substantial role in traffic situations. If self-driven cars could signal whether they “see” you or not, it would be safer for pedestrians to cooperate. E.g. when you take a crosswalk, it could shine green for braking before you, or red/yellow for accelerating without seeing any obstacles. It would also be good, if a green laser projected its suggested path with a bold stop line right on the road (or fading red dot pattern if it doesn’t).
- beerandt 7y ago
  This is a good point- I'm thinking about how many times pedestrians will wait for eye contact with a driver before crossing. And also the ones that see you, but cross anyway, assuming you'll stop. There's a lot of subtle interaction that I have yet to see how ai driving cars will replicate, or at least account for.
- leggomylibro 7y ago
  Pedestrians have the right of way, though; they don't need to cooperate. A self-driving car shouldn't be allowed to run someone down just because they were jaywalking and didn't correlate the red light speeding towards them with an autonomous death machine. We generally don't let human drivers get away with mowing down pedestrians or hit-and-runs if they get caught, so why would it be an acceptable failure mode for a machine? Who is responsible for that failure and that death, in the eyes of the court? If the answer is 'nobody', then self-driving cars will have a lot of difficulty finding acceptance. Also, many states have regulations surrounding which colors of lights can go where on a car for these sorts of reasons - unambiguous and predictable communication is key to driving, and putting what looks like a brake light on the front of your car is confusing.
  - Sohcahtoa82 7y ago
    > Pedestrians have the right of way, though; they don't need to cooperate. I'd change "don't need to" with "shouldn't have to". There are a lot of dead pedestrians who didn't look before crossing a street because they knew they had the right of way. It's just another form of "trust but verify". I know when I'm crossing a busy intersection, even if the cross-traffic has a red light and I have a walk sign, I'm closely watching the on-coming cars to make sure none of them are running the red light. I'm extra vigilant for the people making a right turn on red.
beerandt 7y ago
I'm waiting for the first race related incident to blow this topic up. A Tesla swerving around a white pedestrian but at the expense of hitting a black pedestrian will put an end to any justification of "costs one life to save others." Edit: I'm not saying ai/ml will be racist (although it hasn't shown a great track record so far). I'm saying public sentiment, given the current status of media alarmism, is going to jump at the first hint of any sort of detection bias, whether it actually exists or not.
- zaroth 7y ago
  You'll be waiting a long time. The systems simply do not operate at this level of awareness or reasoning, and likely never will.
  - sokoloff 7y ago
    I don't think GP is suggesting a conscious or intentional bias would be present. Imagine that a car swerves around a pedestrian A who happens to be in light blue jeans and a white jacket and strikes an undetected pedestrian B who happens to be in black pants and a black jacket. It seems easy to imagine how that could happen to any driver, right? Now, imagine that it's an autonomous car, pedestrian A happens to be white, and pedestrian B happens to be black. Boom, there's your scandalous story: an autonomous car just swerved to avoid a white pedestrian and struck a black pedestrian.
    - beerandt 7y ago
      This is exactly what I meant, thank you. But there is a bit more to it. The A or B side-by-side is the most simple comparison to make the point. Some of the same dangers that are in play now might still be risks as AI cars become more common. For instance, minority neighborhoods tend to not have as many street lights at night. The cars might be smart enough to compensate. But if they aren't, AI has the potential of falling victim to statistical correlation. Headline by click bait reporter: "Studies show self-driving cars kill minorities at higher than average rates" It doesn't matter if the real cause was street lights.
  - 7y ago

Whatarethese 7y ago

I have the same assumption when a dumb human is tailgating and rear ends the car in from of them. That has been solved with frontal radar and emergency braking.

sokoloff 7y ago
“been solved”? What percentage of the overall fleet is equipped for automated frontal collision avoidance? Under 5%? Under 2%?
- standyro 7y ago
  Every Model 3 has a forward collision avoidance that has a better reaction time than I do. I know because it's saved my life already when I narrowly avoided a suddenly stopped car in front of me. Tesla's Autopilot (not full self driving) has its issues, mainly with vague lane markings, but collision avoidance isn't one of them. I know plenty of drivers who fall victim and cause accidents because of the same issues that Autopilot has: poor lane markings, not seeing cars that are immobile on the side of the highway.
  - sokoloff 7y ago
    There are over 1000 times more "not Model 3"s in the US than Model 3s. That was my only point about "this has been solved". It's not a knock against the Model 3.

wruza 7y ago

Communication plays a substantial role in traffic situations. If self-driven cars could signal whether they “see” you or not, it would be safer for pedestrians to cooperate. E.g. when you take a crosswalk, it could shine green for braking before you, or red/yellow for accelerating without seeing any obstacles. It would also be good, if a green laser projected its suggested path with a bold stop line right on the road (or fading red dot pattern if it doesn’t).

beerandt 7y ago
This is a good point- I'm thinking about how many times pedestrians will wait for eye contact with a driver before crossing. And also the ones that see you, but cross anyway, assuming you'll stop. There's a lot of subtle interaction that I have yet to see how ai driving cars will replicate, or at least account for.
leggomylibro 7y ago
Pedestrians have the right of way, though; they don't need to cooperate. A self-driving car shouldn't be allowed to run someone down just because they were jaywalking and didn't correlate the red light speeding towards them with an autonomous death machine. We generally don't let human drivers get away with mowing down pedestrians or hit-and-runs if they get caught, so why would it be an acceptable failure mode for a machine? Who is responsible for that failure and that death, in the eyes of the court? If the answer is 'nobody', then self-driving cars will have a lot of difficulty finding acceptance. Also, many states have regulations surrounding which colors of lights can go where on a car for these sorts of reasons - unambiguous and predictable communication is key to driving, and putting what looks like a brake light on the front of your car is confusing.
- Sohcahtoa82 7y ago
  > Pedestrians have the right of way, though; they don't need to cooperate. I'd change "don't need to" with "shouldn't have to". There are a lot of dead pedestrians who didn't look before crossing a street because they knew they had the right of way. It's just another form of "trust but verify". I know when I'm crossing a busy intersection, even if the cross-traffic has a red light and I have a walk sign, I'm closely watching the on-coming cars to make sure none of them are running the red light. I'm extra vigilant for the people making a right turn on red.

beerandt 7y ago

I'm waiting for the first race related incident to blow this topic up. A Tesla swerving around a white pedestrian but at the expense of hitting a black pedestrian will put an end to any justification of "costs one life to save others." Edit: I'm not saying ai/ml will be racist (although it hasn't shown a great track record so far). I'm saying public sentiment, given the current status of media alarmism, is going to jump at the first hint of any sort of detection bias, whether it actually exists or not.

zaroth 7y ago
You'll be waiting a long time. The systems simply do not operate at this level of awareness or reasoning, and likely never will.
- sokoloff 7y ago
  I don't think GP is suggesting a conscious or intentional bias would be present. Imagine that a car swerves around a pedestrian A who happens to be in light blue jeans and a white jacket and strikes an undetected pedestrian B who happens to be in black pants and a black jacket. It seems easy to imagine how that could happen to any driver, right? Now, imagine that it's an autonomous car, pedestrian A happens to be white, and pedestrian B happens to be black. Boom, there's your scandalous story: an autonomous car just swerved to avoid a white pedestrian and struck a black pedestrian.
  - beerandt 7y ago
    This is exactly what I meant, thank you. But there is a bit more to it. The A or B side-by-side is the most simple comparison to make the point. Some of the same dangers that are in play now might still be risks as AI cars become more common. For instance, minority neighborhoods tend to not have as many street lights at night. The cars might be smart enough to compensate. But if they aren't, AI has the potential of falling victim to statistical correlation. Headline by click bait reporter: "Studies show self-driving cars kill minorities at higher than average rates" It doesn't matter if the real cause was street lights.
- 7y ago