3 ms·
How do you einsum convolution? Arguably the single most important linear operation in deep learning?
by TTPrograms 8y ago
How do you einsum convolution? Arguably the single most important linear operation in deep learning?
- krastanov 8y agoThe smart-aleck answer would be "with a Fourier transformation". But yeah, I think you have a point.
- ssivark 8y agoConvolution is a linear operator (with a certain special structure). So convolutions are represented just like any matrix-vector multiplication. Given the special structure of the convolution operator, it can be represented in a sparse manner, and what's more, the actual computation can also be implemented in an efficient manner, compared to a naive matrix-vector multiplication. TL;DR: It can be represented easily using Einstein notation. Einstein notation just does not capture the sparsity properties we like; it represents the transformation properties quite nicely.