4 ms·
They disabled a system we know works, for one that failed. No redundancy. If that is the case, then Uber can absolutely be held accountable for not taking enoug
by shakna 9y ago
They disabled a system we know works, for one that failed. No redundancy. If that is the case, then Uber can absolutely be held accountable for not taking enough safety precautions.
These aren't just test cars. Uber and Volvo's deal was around Uber selling 'enhanced' XC90s. Why not design around the product you purchased?
> Can you imagine testing a self-driving system when another system is also controlling the car?
You mean external forces such as a driver? Which is expected and required at this point?
- icebraining 9y agoI don't pretend to be an expert, but generally adding multiple layers of control on the same axis adds its own failure points. Having a human that can override the automatic system is not an argument for leaving that enabled - it's an argument against it! Personally, I think Uber simply shouldn't have tested it on public roads at this point.
- shakna 9y ago> generally adding multiple layers of control on the same axis They wouldn't be. Subsumption architecture is the usual method to couple these things together, since the 80s. Individual reactionary modules, that may be 'dumb', like the AutoBreak, that have a hierarchy of response. Redundancy is essential for safety.
- icebraining 9y agoOK, let's say the Volvo system gets precedence. How will the Uber system ever learn how to behave when the Volvo system is no longer in place? If we were talking about a production system, I would agree, but I don't think this is appropriate for the training phase. Redundancy is only essential if the potential for failure is unacceptable. If you remove that (by not driving in public streets), it isn't.
- shakna 9y ago> Redundancy is only essential if the potential for failure is unacceptable. If you remove that (by not driving in public streets), it isn't. In which case we wouldn't be having this conversation, Uber wouldn't be under investigation, and someone wouldn't be dead.
- leereeves 9y agoSaving lives is more important than training the self-driving system. Disabling safety systems for the sake of training isn't acceptable. That said, how does Uber improve the self-driving system in response to any failure? Perhaps by training on recorded data? They must have ways to train the system in addition to waiting and hoping the car encounters the same situation again.
- azernik 9y agoIt will learn how to behave by treating an activation of the redundancy systems as a failure. One of the big lessons of the Therac fiasco - your interlocks need to log every time they are activated.