3 ms·
I thought it was already proven that a neural network with two hidden layers is as powerful as any neural network with an arbitrary number of layers?
by oh_sigh 12y ago
I thought it was already proven that a neural network with two hidden layers is as powerful as any neural network with an arbitrary number of layers?
- ivan_ah 12y agoRight, but the narrative associated with this result is that the single-layer might require a lot more units to achieve the same performance as a deep architecture. It seems this is not necessarily the case---based on the empirical results shown.
- beagle3 12y agoWhat I'm familiar with is described here: http://neuron.eng.wayne.edu/tarek/MITbook/chap2/2_3.html http://neuron.eng.wayne.edu/tarek/MITbook/chap2/2_3.html Basically, one hidden layer networks are already universal approximator; however, it's a nonconstructive result. This article is semi-constructive, - they train more complicated model, then emulate it with a simpler model, but do not know how to train the simple model directly.