5 ms·
While in practice this is true, a lot of work is being done to try to figure out how to make shallow neural networks train as fast as deep ones. We know by the
by Der_Einzige 6y ago
While in practice this is true, a lot of work is being done to try to figure out how to make shallow neural networks train as fast as deep ones. We know by the UAT that in principle a shallow network with similar paramaters counts to a deep one should be capable of learning the same decision boundary...
https://en.m.wikipedia.org/wiki/Universal_approximation_theorem https://en.m.wikipedia.org/wiki/Universal_approximation_theo...
- aborsy 6y agoIt depends on the function that you are trying to approximate. I can give you a function that a shallow nnet would approximate better and functions that deep nets approximate exponentially better even with one more layer (in terms of number of neurons n). In the limit n->\infty, both reach arbitrary small errors (obviously often with different number of parameters).
- falcor84 6y agoIs there a particular theorem that you're invoking here?