3 ms·

Have yet to see an illustration that grasps multichannel convolution filters (MCCF) concept clearly. Why those channel stack size keep growing? How are they act

by Dzugaru 10y ago

Have yet to see an illustration that grasps multichannel convolution filters (MCCF) concept clearly. Why those channel stack size keep growing? How are they actually connected? The thing that each conv filter consists of kernels in multiple channels (that's why first layer filter visualisations are colored btw - color image is a "3-dimensional" image) - and we convolve each kernel with corresponding input channel, then sum (that's the key) the responses. Then having multiple MCCF (usually more at each layer) yields a new multi-channel image (say, 16 channels) and we apply new set of (say, 32) 16-channeled MCCFs to it (which we cannot visualise by themselves anymore, we need a 16-dimensional image for each filter) yielding 32-channel image. That sort of thing is almost never explained properly.

napperjabber 10y ago
Care to chime in with some samples?
deleted 10y ago
[deleted]
limefrog 10y ago
Ever seen "word2vec"? Each word is embedded in a high-dimensional "concept" space, where words that are similar to each other will be close by. A CNN does this for localized sections of the images. Each layer looks over a wider area of the original image (because of the convolution) and embeds the "concepts" in that area of the image in a higher-dimensional space than the layer before it.
jhj 10y ago
Valid-only convolution (in the MATLAB sense) by itself reduces the dimensionality of the input; for images, it will go from (h x w) to (h - kh + 1) x (w - kw + 1) per each plane. You can think of a convnet as a series of feature transformations, consisting of a normalization/whitening stage, a filter bank that is a projection into a higher dimension (on an overcomplete basis), non-linear operation in the higher dimensional space, and then possibly pooling to a lower dimensional space. The “filter bank” (aka convolution) and non-linearity produce a non-linear embedding of the input in a higher dimension; in convnets, the “filter bank” itself is learned. Classes or features are easier to separate in the higher dimensional space. There are some still-developing ideas on how all this stuff is connected to wavelet theory on firmer mathematical ground and the like, but for the most part, it just works "really well". For an image network, at each layer there are (input planes x output planes) convolution kernels of size (kh x kw). Each output plane `j` is a sum over all input planes `i` individually convolved using the filter (i, j); the reduction dimension is the input plane.
- jhj 10y ago
  see https://github.com/facebook/fbcunn/blob/master/test/ReferenceConvolutions.cpp#L192 https://github.com/facebook/fbcunn/blob/master/test/Referenc... for a loop nest that shows what the forward pass of a 2-d image convnet convolution module does. That's gussied up with convolution stride and padding and a bunch of C++11 mumbo jumbo, but you should be able to see what it is doing.

by Dzugaru 10y ago

napperjabber 10y ago

Care to chime in with some samples?

deleted 10y ago

[deleted]

limefrog 10y ago

Ever seen "word2vec"? Each word is embedded in a high-dimensional "concept" space, where words that are similar to each other will be close by. A CNN does this for localized sections of the images. Each layer looks over a wider area of the original image (because of the convolution) and embeds the "concepts" in that area of the image in a higher-dimensional space than the layer before it.

jhj 10y ago

Valid-only convolution (in the MATLAB sense) by itself reduces the dimensionality of the input; for images, it will go from (h x w) to (h - kh + 1) x (w - kw + 1) per each plane. You can think of a convnet as a series of feature transformations, consisting of a normalization/whitening stage, a filter bank that is a projection into a higher dimension (on an overcomplete basis), non-linear operation in the higher dimensional space, and then possibly pooling to a lower dimensional space. The “filter bank” (aka convolution) and non-linearity produce a non-linear embedding of the input in a higher dimension; in convnets, the “filter bank” itself is learned. Classes or features are easier to separate in the higher dimensional space. There are some still-developing ideas on how all this stuff is connected to wavelet theory on firmer mathematical ground and the like, but for the most part, it just works "really well". For an image network, at each layer there are (input planes x output planes) convolution kernels of size (kh x kw). Each output plane `j` is a sum over all input planes `i` individually convolved using the filter (i, j); the reduction dimension is the input plane.

jhj 10y ago
see https://github.com/facebook/fbcunn/blob/master/test/ReferenceConvolutions.cpp#L192 https://github.com/facebook/fbcunn/blob/master/test/Referenc... for a loop nest that shows what the forward pass of a 2-d image convnet convolution module does. That's gussied up with convolution stride and padding and a bunch of C++11 mumbo jumbo, but you should be able to see what it is doing.