I believe that has to do with the vanishing gradient issue which has been studied. As for why nets converge at all, there's this paper whichI believe tries to

I believe that has to do with the vanishing gradient issue which has been studied. As for why nets converge at all, there's this paper whichI believe tries to establish some theory why bigger nets don't have such a big problem with local optima: http://arxiv.org/abs/1412.0233 http://arxiv.org/abs/1412.0233