Interestingly deep networks aren't ever really learned by gradient descent. No! Really ! They are learned by interaction with a network selection process of v

Interestingly deep networks aren't ever really learned by gradient descent. No! Really ! They are learned by interaction with a network selection process of varying degrees of hokeyness that involves decisions being taken by humans about the characteristics of the network, training data, training process and testing processes. At the end -> the model! Gradient descent is v.critical to this, but it's not the only thing going on by a long way. Also dropout.