Unit tests are not sufficient for neural networks. Say the network takes 2D images of typical 224x224 pixels with 3 channels (RGB) of 8 bit values, this input s

Unit tests are not sufficient for neural networks. Say the network takes 2D images of typical 224x224 pixels with 3 channels (RGB) of 8 bit values, this input space has 256^(224x244x3) = 3.5×10^362507 possibilities. Billions of years to test them all. This is before we consider stereo vision, 3D images and state over time. How to know which subset of these inputs are necessary to give reasonable coverage? Right now I don't think we have very good answers to these things. Of course one can always add a regression test (with some K mutations) when someone crashes. It is better than nothing, but hardly good assurance that something like this will never happen again. The entire area of safety and quality assurance with neural networks is still in active research actively, from multiple angles. For some examples of how chaotic neural networks can be, look up 'adversarial examples'.