3 ms·

The problem is, they can't do this today with the necessary classification quality. Elaine was killed, because someone at Uber wanted these cars to drive so bad

The problem is, they can't do this today with the necessary classification quality. Elaine was killed, because someone at Uber wanted these cars to drive so badly. It also is about culture and prediction. If the other participants behave differently to your prediction, it can get hazardous quickly at highway speeds. I can't find a better link [0], but maybe it's good enough to get the point across. The term is "warning dilemma" and means that you only have perfect information at the time of impact. Before, the classification if there will be a problem or not is worse, the earlier you ask. But asking early is necessary for the human to have a chance to react. [0] https://www.researchgate.net/publication/326568066_Towards_an_Assistance_Strategy_That_Reduces_Unnecessary_Collision_Alarms_An_Examination_of_the_Driver's_Perceived_Need_for_Assistance https://www.researchgate.net/publication/326568066_Towards_a...

Aeolun 6y ago
From what I remember, Elaine was mostly killed because she stepped in front of a moving car in the middle of the night. I’m not convinced that’s the best example, since the AI might have actually done better if it had full control of the car instead of defaulting to ‘please take over now’ mode.
- mbreese 6y ago
  If I remember correctly, the cars built in system for emergency braking in this scenario was disabled. So, any other car of that type would have avoided the accident. However, because the AI apparently knew better, it was disabled and someone died. AI has the problem that not only does it need to observe the environment and determine what to do, but it has to predict the actions of non-rational actor (humans). We aren’t always predictable. AI isn’t always predictable either. The woman who died may have assumed that a car driven by a person would have stopped.
- jeffreygoesto 6y ago
  Right. I could have stated it better, what I meant was that the (safety) driver was not paying (enough) attention to the road and was basically driving Level 3 (the car drives and it will beep whenever I need to take over, so just relax and browse the web). It was impossible to react to the short notice and the warning was _not_ given 10 seconds ago. I personally think the system had sensed her as a succession of standing targets and never predicted any velocity vector. Braking for static radar targets is not happening, because of the clutter you get everywhere you would not drive at all. So the misclassification and misprediction was not detectable for the system (in it's world model, everything looked consistent) and it warned too late.

Aeolun 6y ago

From what I remember, Elaine was mostly killed because she stepped in front of a moving car in the middle of the night. I’m not convinced that’s the best example, since the AI might have actually done better if it had full control of the car instead of defaulting to ‘please take over now’ mode.

mbreese 6y ago
If I remember correctly, the cars built in system for emergency braking in this scenario was disabled. So, any other car of that type would have avoided the accident. However, because the AI apparently knew better, it was disabled and someone died. AI has the problem that not only does it need to observe the environment and determine what to do, but it has to predict the actions of non-rational actor (humans). We aren’t always predictable. AI isn’t always predictable either. The woman who died may have assumed that a car driven by a person would have stopped.
jeffreygoesto 6y ago
Right. I could have stated it better, what I meant was that the (safety) driver was not paying (enough) attention to the road and was basically driving Level 3 (the car drives and it will beep whenever I need to take over, so just relax and browse the web). It was impossible to react to the short notice and the warning was _not_ given 10 seconds ago. I personally think the system had sensed her as a succession of standing targets and never predicted any velocity vector. Braking for static radar targets is not happening, because of the clutter you get everywhere you would not drive at all. So the misclassification and misprediction was not detectable for the system (in it's world model, everything looked consistent) and it warned too late.