3 ms·
Right, the whole point is that neural networks let you apply tricks like convolution or recurrence to capture a lot of the structure in the model domain and exp
by ummonk 7y ago
Right, the whole point is that neural networks let you apply tricks like convolution or recurrence to capture a lot of the structure in the model domain and exponentially reduce the search space for your model when dealing with large inputs like detailed images or waveforms.
- enriquto 7y agoThis has to do with the symmetry of the input to translations, that allows a convolutional neural network. But neural networks need not be convolutional.
- posterboy 7y agoCould one say, in a sense, that recurrence is essentially differential equations and convolution essentially more complicated operations than those of arithmetic polynomials? This might sound like nonsense. On the one hand, most trivial convolutions use trivial operators; "polynomials" might include higher operations anyhow, or approximate some of the more important ones, none of which is appealing if simplicity equals efficiency. On the other hand, I never really understood diff-eqs; ODEs seem like polynomials over self similar polynomials, to me, hence "recurrent"; All the other diff-eqs I can't begin to fathom.
- apl 7y agoThere's many perspectives on everything. Deep ConvNets, for instance, can be expressed as a continuously evolving ODE. Here's a fantastic paper on this view: https://papers.nips.cc/paper/7892-neural-ordinary-differential-equations https://papers.nips.cc/paper/7892-neural-ordinary-differenti...
- ummonk 7y agoYeah, that's a reasonable way of looking at it. Convolution would be a partial differential equation in this view.