"What if the neural net or the system used to detect obstacles didn't see it because the precise configuration of the data fooled it?" I think the answer is th

"What if the neural net or the system used to detect obstacles didn't see it because the precise configuration of the data fooled it?" I think the answer is that all state must be represented as a probability over states and the controller must be set up to be a robust controller under this ambiguity of state. Then the problem becomes one of making sure that probability is well calibrated so that you get the appropriate control response. The whole software stack for autonomous driving is also a black box, that is just as big an issue. If some parts are sophisticated but are being fed into others that are naive we have no idea. Every sensor has issues, I wouldn't be so quick to worry about the neural networks in particular.