4 ms·
The Uber crash was not a failure of the AI, it was a failure of the UX. The car saw the obstacle and detected a collision, but there was no warning system to a
by hannasanarion 8y ago
The Uber crash was not a failure of the AI, it was a failure of the UX.
The car saw the obstacle and detected a collision, but there was no warning system to alert the driver about it.
Meanwhile, the driver was looking at a laptop and taking notes on the system logs (which they were told to do), not watching the road.
- cozzyd 8y agoThe car saw the obstacle and detected a collision, but did nothing, presumably due to the high false positive rate of such things.
- aetherson 8y agoYeah, I feel like your parent poster is ignoring the implications of the high false positive rate. Like, let's say I had a grammar check. But the grammar check was bad. It flags every other sentence as needing a grammar change, anything other than a very simple "subject-verb-object" sentence structure looks wrong to it. For every correct grammar problem it identifies, it finds 20 incorrect grammar problems. As a result, I turn off the UI for the grammar check. Then I write a document that has a grammar problem. It seems stupid to say that this is either a failure of my use of the software, or a UI failure of the software. The problem is that the fucking software doesn't work, specifically that it fails in a way that makes it too "noisy" to use. And sure, if my job is to catch grammar errors, then it's also my fault. I'm supposed to be backing up the system. But we also know that I won't be perfect and will have failures of attention sometimes. But regardless of that, the grammar checker just didn't work. And, in this analogy, bad grammar kills people.
- voidmain 8y agoIn this case, as I understand it, they also disabled the emergency braking feature built into the Volvo. Which wouldn't be obviously crazy if they had better, adequately tested emergency braking integrated with the "autonomy" system. But to disable the existing safety feature in favor of their own system, and also disable their own system due to a high false positive rate, seems negligent on the face of it.
- joejerryronnie 8y agoAnd you have to correct the bad grammer in 0.5 seconds.
- hannasanarion 8y agoThe danger of false positives does not justify having no mechanism to alert the driver. The system is relying on the driver to take over in exceptional circumstances, but gives no warning when an exception occurs.
- _jal 8y agoIf you focus on the crash itself, yes, this semi-autonomous murderbot failed to alert the human. But the failure that killed Elaine Herzberg happened well before then, when the decision to disable the emergency brakes was made. It appears they disabled other safeties as well. I don't disagree that playing a little loose with allowing new tech makes sense. We need to understand what we don't understand about them, accidents of course will happen, etc. But this looks a lot like foreseeable negligent homicide to me.
- jacquesm 8y agoI'd argue it was a corporate culture failure. A company that can put a device like that on the road is failing on many levels, that there is a contributory issue with the AI or the UX has nothing to do with the root cause.
- w_t_payne 8y agoYou probably have something here. Culture is sooo important.