4 ms·

> dead Uber pedestrian Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have p

by keanzu 7y ago

> dead Uber pedestrian Elaine Herzberg was in dark clothing crossing a dark street well away from any crosswalk or street lighting. Would a human driver have performed better? From the footage I saw she was nearly invisible, I would have hit her too. https://www.youtube.com/watch?v=ufNNuafuU7M https://www.youtube.com/watch?v=ufNNuafuU7M This was not a hard fail for the AI.

sebcat 7y ago
The exposure time and the dynamic range of the sensor affects the visibility of the person in that video - it is very likely that a non-distracted human would have performed better. The vehicle was equipped with both radar and lidar. The victim was detected as an unknown object six seconds prior to impact, and at 1.3 seconds prior to impact when the victim entered the road the system determined that emergency braking by the driver was required, however the (distracted) driver was not alerted of this.
- leereeves 7y ago
  > the system determined that emergency braking by the driver was required, however the (distracted) driver was not alerted of this. Why would the system notify the driver that emergency braking was required instead of simply braking?
  - keanzu 7y ago
    "the nascent computerized braking technology was disabled the day of the crash" https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg
kgabis 7y ago
The footage shows how car's cameras saw the accident. Human eyes have much greater dynamic range than that, so I wouldn't assume that a human driver would not perform better. Something appearing pitch black on this footage could be well recognisable to you. Also, this car's lidar also failed to recognise the pedestrian so if this isn't an AI fail then I don't know what is.
- keanzu 7y ago
  This is absolutely a fail, a woman died. The question is whether or not this incident is an example of "fails harder than most average drivers" or a hard fail. With the sensor array available to it the car should have done better, no question. But to make the claim "fails harder" I would be looking for a clear cut situation where a human would almost definitely have outperformed the AI. Human eyes do have miraculous dynamic range so we would likely see more. Can we say with 90% certainty that a human would have saved the situation?
  - Piskvorrr 7y ago
    "There's something out in front of me, an unclear shape right in my path, relatively far, 6 seconds out. I will drive straight through it instead of slowing down, because...[fill in your Voight-Kampff test response]". Well? Is that at least 90% human? Moreover, try this dashcam video: https://youtu.be/typj1asf1EM https://youtu.be/typj1asf1EM It's 10 seconds long, and makes the pedestrian look almost invisible except for the soles. However, when I took that video, both the crossing pedestrians were clearly visible, not vague shapes that you only notice when you're told they exist. So much for video feed fidelity.
  - pja 7y ago
    Can we say with 90% certainty that a human would have saved the situation? Yes, given the misleading nature of the dashcam video I think we can. This was not a pitch dark road lit only by headlights where an obstacle "appeared out of no-where". This was a well-lit main street, with good visibility in all directions. An ordinary human driver would have had no problem identifying Elaine as a hazard and taking the appropriate avoiding action, which was simply to slow down sufficiently to allow her to cross the road. The backup driver in the car was apparently looking at their phone or some other device and not watching the road at the time.
    - keanzu 7y ago
      Based on the evidence you have put forth your conclusion is logical and reasonable. You have convinced me that my statement was in error.
michaelt 7y ago
LIDAR would have picked that up dead easily. Just like LIDAR would have picked up https://www.extremetech.com/extreme/297901-ntsb-autopilot-design-flaw-inattentive-driver-led-to-tesla-firetruck-crash https://www.extremetech.com/extreme/297901-ntsb-autopilot-de... And just like LIDAR would have picked up https://youtu.be/-2ml6sjk_8c?t=17 https://youtu.be/-2ml6sjk_8c?t=17 And just like LIDAR would have picked up https://youtu.be/fKyUqZDYwrU https://youtu.be/fKyUqZDYwrU And just like LIDAR would have picked up https://www.bbc.co.uk/news/technology-50713716 https://www.bbc.co.uk/news/technology-50713716 These accidents are 100% due to the decision to use a janky vision system to avoid spending $2000 on lidar; and that janky vision system failing.
- keanzu 7y ago
  "Brad Templeton, who provided consulting for autonomous driving competitor Waymo, noted the car was equipped with advanced sensors, including radar and LiDAR, which would not have been affected by the darkness." https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg The car had LIDAR.
  - deleted 7y ago
    [deleted]
  - Brave-Steak 7y ago
    It had lidar and ignored an unknown object it was tracking. If that's not damning for the whole field, I don't know what is.
  - michaelt 7y ago
    Yep, and it detected Herzberg in the roadway with plenty of time to spare. "the car detected the pedestrian as early as 6 seconds before the crash" [...] "Uber told the NTSB that “emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior,” in other words, to ensure a smooth ride. “The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.”" [1] 'Oh well it was very dark' is not a factor in the crash that killed Herzberg [1] https://techcrunch.com/2018/05/24/uber-in-fatal-crash-detected-pedestrian-but-had-emergency-braking-disabled/ https://techcrunch.com/2018/05/24/uber-in-fatal-crash-detect...
Piskvorrr 7y ago
Nice story...if only it would match the evidence. The algorithm has detected the person 6 seconds before the crash, but as it didn't match any of the object classes conclusively, it took no action. You read that right: "there's something at 12 o'clock. Bike? Person? Unknown? Who cares, proceed through it!" If that's not a hard fail, IDK what would.
- dkonofalski 7y ago
  That's not what happened. The AI was not programmed to drive through anything. It was incorrectly programmed to dump previous data when the categorization changed. It correctly identified at each point that it was meant to slow down and/or stop but, by the time it determined what the obstacle was, the previous data was thrown out and it didn't have enough time to stop properly. In your example, it was more like "There's something that 12 o'clock. Bike? Person? Unknown? Stop!!" just before actually hitting the person.
  - Piskvorrr 7y ago
    The car did "see" an obstacle for over 6 seconds and did not brake for it, now someone is dead. You are haggling over semantics to make it look like this did not happen and/or this is not a bug. Atrocious. (Or, more charitably, "oops, somebody forgot that object persistence is a thing" does not excuse the result)
    - dkonofalski 7y ago
      What? That's not at all what I'm doing and you're being extremely disingenuous to suggest that. I'm simply correcting misinformation. The car wasn't programmed to drive through anything. It was programmed to throw away information. Either way, it's an atrocious mistake and I've even said, elsewhere in these comments, that the people responsible for that code should be held liable for criminal negligence. There's no need to lie about my point or my position to defend yourself. That's just silly.
      - Piskvorrr 7y ago
pja 7y ago
The video released by Uber was extremely misleading. Here is a video on YouTube of the same stretch of road taken by someone with an ordinary phone camera shortly after Elaine’s death: https://www.youtube.com/watch?v=CRW0q8i3u6E https://www.youtube.com/watch?v=CRW0q8i3u6E It’s clear that a) the road is well lit and b) visibility is far, far better than the Uber video would suggest. An ordinary human driver would have seen Elaine & taken evasive action. This death is absolutely Uber’s responsibility.
- keanzu 7y ago
  > An ordinary human driver would have seen Elaine & taken evasive action. Looks like this was a hard fail for the AI then. We can say with better than 90% certainty that a human would have saved the situation, probably would have stopped or avoided easily. My mistake.
- greatpatton 7y ago
  "May have seen" is more appropriate as every day pedestrian get killed on well lit road by human drivers.
  - keanzu 7y ago
    The assumption was for an ordinary driver, the expectation is that given sufficient lighting the vast majority of drivers would see and avoid a pedestrian. Most of the millions of pedestrian vehicle interactions daily go by without incident, one or the other party giving way, so this would be the normal expectation for an ordinary driver. We can reasonably assume that pja is aware of the existence of abysmal drivers and fatal crashes that should not have happened. I doubt their intent was for "would" to be interpreted as "100%".
  - Piskvorrr 7y ago
    Which is also true. This is perhaps the underlying issue: "we expect cars to be safe, while also expecting driving fast in inherently unsafe conditions." In other words, the actual driving risk appetite is atrocious, but nobody's willing to admit it when human drivers are in the equation. SDVs are disruptive to this open secret.
deleted 7y ago
[deleted]

by keanzu 7y ago

sebcat 7y ago

The exposure time and the dynamic range of the sensor affects the visibility of the person in that video - it is very likely that a non-distracted human would have performed better. The vehicle was equipped with both radar and lidar. The victim was detected as an unknown object six seconds prior to impact, and at 1.3 seconds prior to impact when the victim entered the road the system determined that emergency braking by the driver was required, however the (distracted) driver was not alerted of this.

leereeves 7y ago
> the system determined that emergency braking by the driver was required, however the (distracted) driver was not alerted of this. Why would the system notify the driver that emergency braking was required instead of simply braking?
- keanzu 7y ago
  "the nascent computerized braking technology was disabled the day of the crash" https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg

kgabis 7y ago

The footage shows how car's cameras saw the accident. Human eyes have much greater dynamic range than that, so I wouldn't assume that a human driver would not perform better. Something appearing pitch black on this footage could be well recognisable to you. Also, this car's lidar also failed to recognise the pedestrian so if this isn't an AI fail then I don't know what is.

keanzu 7y ago
This is absolutely a fail, a woman died. The question is whether or not this incident is an example of "fails harder than most average drivers" or a hard fail. With the sensor array available to it the car should have done better, no question. But to make the claim "fails harder" I would be looking for a clear cut situation where a human would almost definitely have outperformed the AI. Human eyes do have miraculous dynamic range so we would likely see more. Can we say with 90% certainty that a human would have saved the situation?
- Piskvorrr 7y ago
  "There's something out in front of me, an unclear shape right in my path, relatively far, 6 seconds out. I will drive straight through it instead of slowing down, because...[fill in your Voight-Kampff test response]". Well? Is that at least 90% human? Moreover, try this dashcam video: https://youtu.be/typj1asf1EM https://youtu.be/typj1asf1EM It's 10 seconds long, and makes the pedestrian look almost invisible except for the soles. However, when I took that video, both the crossing pedestrians were clearly visible, not vague shapes that you only notice when you're told they exist. So much for video feed fidelity.
- pja 7y ago
  Can we say with 90% certainty that a human would have saved the situation? Yes, given the misleading nature of the dashcam video I think we can. This was not a pitch dark road lit only by headlights where an obstacle "appeared out of no-where". This was a well-lit main street, with good visibility in all directions. An ordinary human driver would have had no problem identifying Elaine as a hazard and taking the appropriate avoiding action, which was simply to slow down sufficiently to allow her to cross the road. The backup driver in the car was apparently looking at their phone or some other device and not watching the road at the time.
  - keanzu 7y ago
    Based on the evidence you have put forth your conclusion is logical and reasonable. You have convinced me that my statement was in error.

michaelt 7y ago

LIDAR would have picked that up dead easily. Just like LIDAR would have picked up https://www.extremetech.com/extreme/297901-ntsb-autopilot-design-flaw-inattentive-driver-led-to-tesla-firetruck-crash https://www.extremetech.com/extreme/297901-ntsb-autopilot-de... And just like LIDAR would have picked up https://youtu.be/-2ml6sjk_8c?t=17 https://youtu.be/-2ml6sjk_8c?t=17 And just like LIDAR would have picked up https://youtu.be/fKyUqZDYwrU https://youtu.be/fKyUqZDYwrU And just like LIDAR would have picked up https://www.bbc.co.uk/news/technology-50713716 https://www.bbc.co.uk/news/technology-50713716 These accidents are 100% due to the decision to use a janky vision system to avoid spending $2000 on lidar; and that janky vision system failing.

keanzu 7y ago
"Brad Templeton, who provided consulting for autonomous driving competitor Waymo, noted the car was equipped with advanced sensors, including radar and LiDAR, which would not have been affected by the darkness." https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg The car had LIDAR.
- deleted 7y ago
  [deleted]
- Brave-Steak 7y ago
  It had lidar and ignored an unknown object it was tracking. If that's not damning for the whole field, I don't know what is.
- michaelt 7y ago
  Yep, and it detected Herzberg in the roadway with plenty of time to spare. "the car detected the pedestrian as early as 6 seconds before the crash" [...] "Uber told the NTSB that “emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior,” in other words, to ensure a smooth ride. “The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.”" [1] 'Oh well it was very dark' is not a factor in the crash that killed Herzberg [1] https://techcrunch.com/2018/05/24/uber-in-fatal-crash-detected-pedestrian-but-had-emergency-braking-disabled/ https://techcrunch.com/2018/05/24/uber-in-fatal-crash-detect...

Piskvorrr 7y ago

Nice story...if only it would match the evidence. The algorithm has detected the person 6 seconds before the crash, but as it didn't match any of the object classes conclusively, it took no action. You read that right: "there's something at 12 o'clock. Bike? Person? Unknown? Who cares, proceed through it!" If that's not a hard fail, IDK what would.

dkonofalski 7y ago
That's not what happened. The AI was not programmed to drive through anything. It was incorrectly programmed to dump previous data when the categorization changed. It correctly identified at each point that it was meant to slow down and/or stop but, by the time it determined what the obstacle was, the previous data was thrown out and it didn't have enough time to stop properly. In your example, it was more like "There's something that 12 o'clock. Bike? Person? Unknown? Stop!!" just before actually hitting the person.
- Piskvorrr 7y ago
  The car did "see" an obstacle for over 6 seconds and did not brake for it, now someone is dead. You are haggling over semantics to make it look like this did not happen and/or this is not a bug. Atrocious. (Or, more charitably, "oops, somebody forgot that object persistence is a thing" does not excuse the result)
  - dkonofalski 7y ago
    What? That's not at all what I'm doing and you're being extremely disingenuous to suggest that. I'm simply correcting misinformation. The car wasn't programmed to drive through anything. It was programmed to throw away information. Either way, it's an atrocious mistake and I've even said, elsewhere in these comments, that the people responsible for that code should be held liable for criminal negligence. There's no need to lie about my point or my position to defend yourself. That's just silly.
    - Piskvorrr 7y ago

pja 7y ago

The video released by Uber was extremely misleading. Here is a video on YouTube of the same stretch of road taken by someone with an ordinary phone camera shortly after Elaine’s death: https://www.youtube.com/watch?v=CRW0q8i3u6E https://www.youtube.com/watch?v=CRW0q8i3u6E It’s clear that a) the road is well lit and b) visibility is far, far better than the Uber video would suggest. An ordinary human driver would have seen Elaine & taken evasive action. This death is absolutely Uber’s responsibility.

keanzu 7y ago
> An ordinary human driver would have seen Elaine & taken evasive action. Looks like this was a hard fail for the AI then. We can say with better than 90% certainty that a human would have saved the situation, probably would have stopped or avoided easily. My mistake.
greatpatton 7y ago
"May have seen" is more appropriate as every day pedestrian get killed on well lit road by human drivers.
- keanzu 7y ago
  The assumption was for an ordinary driver, the expectation is that given sufficient lighting the vast majority of drivers would see and avoid a pedestrian. Most of the millions of pedestrian vehicle interactions daily go by without incident, one or the other party giving way, so this would be the normal expectation for an ordinary driver. We can reasonably assume that pja is aware of the existence of abysmal drivers and fatal crashes that should not have happened. I doubt their intent was for "would" to be interpreted as "100%".
- Piskvorrr 7y ago
  Which is also true. This is perhaps the underlying issue: "we expect cars to be safe, while also expecting driving fast in inherently unsafe conditions." In other words, the actual driving risk appetite is atrocious, but nobody's willing to admit it when human drivers are in the equation. SDVs are disruptive to this open secret.

deleted 7y ago

[deleted]