Gradient descent is done in batches and over the aggregate collection of the training data so it's pretty lossy. Usually the error is more of the fault of the l

Gradient descent is done in batches and over the aggregate collection of the training data so it's pretty lossy. Usually the error is more of the fault of the learned model itself, and then the culprit is attributed to neuron "mis"-activations post-hoc. Often, problems take the form of generalization error instead. And yes, all of these problems are still quite unsolved, which is why we don't have end-to-end neural network-based self-driving cars, we just have rule based systems with a perception framework that just uses convnets for vision.