>an infinitely deep net (whatever that means) would not generalize on little data, and would get even harder to train the deeper it gets. Not with proper prior

>an infinitely deep net (whatever that means) would not generalize on little data, and would get even harder to train the deeper it gets. Not with proper priors/regularization. >You've argued some ideas from intuition. There is little theoretical rigor around this, however. There's this paper which goes more into theoretical depth on the idea: http://arxiv.org/abs/1412.0233 http://arxiv.org/abs/1412.0233