3 ms·
I don’t blame you, things are pretty diversified so I’m white knuckling it through my own subfield. But as a simple example, convolution would be a little tedi
by janalsncm 3y ago
I don’t blame you, things are pretty diversified so I’m white knuckling it through my own subfield.
But as a simple example, convolution would be a little tedious to describe in the general notation you wrote above without getting into the image dimensions, stride, padding etc., not to mention residual layers or norm layers that are commonly used. Then there are things like stop gradient, dropout or even other training targets which are only used during training but not inference.
- OkayPhysicist 3y agoConvolution actually can be pretty elegantly translated into multiplication by a matrix with symmetries. It's been a few years, but that was an example we had to work out by hand in my Deep Learning course at university. It's pretty much the quintessential example of an enforced symmetry, in that it introduces a symmetry against translation.