4 ms·
Lost me at "Traditional neural nets have two input layers."
by TheDudeMan 5y ago
Lost me at "Traditional neural nets have two input layers."
- named-user 5y agoYes, the input layer and .... The input layer?
- candiodari 5y agoThere used to be a time, long ago, when neural networks had 2 layers: the input layer and the output layer, with full connections between the two. There's an old proof that, if you don't have nonlinearities there's an equivalent 2-layer net for any arbitrarily deep neural net. So the real difference between modern "deep neural networks" and "neural networks" is not actually the depth, even though it is. It's just that neural networks, old definition, a 2 layer net can match any depth net, so you wouldn't use more. The real difference isn't the depth, but the nonlinearity (the tanh/sigmoid/relu operation).