3 ms·

I think the disturbing thing to me is that there wasn't any sort of layered reliability approach to the testing process itself. When you know you're going to be

I think the disturbing thing to me is that there wasn't any sort of layered reliability approach to the testing process itself. When you know you're going to be testing changes to a very complex, multi-sensor real-world system, it would be a standard approach to lay in a more foundational set of technology and practices to the system tests to serve as a conservative safety net. For example, an overriding set of heuristics to brake if needed could be one precaution, and in terms of practices, the human monitor, should have been swapped out at high intervals to alleviate attention fatigue. One can go back and forth on the details, but the need for a separate and independent treatment of safety reviews from the primary product development activity in this kind of endeavour should have been drawn from any of a number of existing practices in other industries. And this is something that a lot of the casual tech articles completely miss - criticizing the primary autonomy tech which of course will have faults, but if you know you will have faults, then where is the serious plan to operate safely knowing the dev system is anticipated to have faults of many types.

mnm1 8y ago
You mean they shouldn't have disabled the car's own autonomous, non AI braking system and kept it as a backup for when theirs failed? Yeah, but that would take a responsible, caring player and we're talking about uber here, a company built on ignoring laws, best practices, and common sense. This can easily be a case of manslaughter, but their cozy relationship with the az government will probably prevent that from happening despite uber's egregious recklessness.
- digikata 8y ago
  > You mean they shouldn't have disabled the car's own autonomous, non AI braking system and kept it as a backup for when theirs failed? I admit that was a thought, but I'd hold back from to saying that simple specific thing was the missed item, or even if that was a realistic expectation to maintain that setup. Without a built up safety culture that looks at the issue from multiple directions in detail, and has the authority to halt testing if needed, then it could have easily been something else. And there are multiple industries from civil/military aviation, medical, architecture, as well as many industrial occupations from which one could have formed a safety framework upon.

mnm1 8y ago

You mean they shouldn't have disabled the car's own autonomous, non AI braking system and kept it as a backup for when theirs failed? Yeah, but that would take a responsible, caring player and we're talking about uber here, a company built on ignoring laws, best practices, and common sense. This can easily be a case of manslaughter, but their cozy relationship with the az government will probably prevent that from happening despite uber's egregious recklessness.

digikata 8y ago
> You mean they shouldn't have disabled the car's own autonomous, non AI braking system and kept it as a backup for when theirs failed? I admit that was a thought, but I'd hold back from to saying that simple specific thing was the missed item, or even if that was a realistic expectation to maintain that setup. Without a built up safety culture that looks at the issue from multiple directions in detail, and has the authority to halt testing if needed, then it could have easily been something else. And there are multiple industries from civil/military aviation, medical, architecture, as well as many industrial occupations from which one could have formed a safety framework upon.