3 ms·
The trick is that convolutional neural nets operate on multichannel images. Your intuition that the synaptic weights form point spread functions is correct, but
by QuadmasterXLII 8y ago
The trick is that convolutional neural nets operate on multichannel images. Your intuition that the synaptic weights form point spread functions is correct, but you have to remember that the activation of each neuron depends on a sum of point spread functions, one for each channel in the previous layer.
I like to think of 1x1 convolutions as pixel wise dense layers
Here’s a toy example of a useful 1x1 convolution: you could convert a color image to greyscale by doing a 1x1 convolution with the kernel (.33, .33, .33)
- DoctorOetker 8y agothank you very much, I suppose we can all agree that 1x1x3 (Width x Height x Color) convolution would have been a lot clearer. again, thanks for succinct and clear explanation!
- daenz 8y agoJust to elaborate a bit more, 1x1 convolution is often used to reduce the number of parameters from the previous layer to keep the total number of parameters manageable.
- DoctorOetker 8y agoyes, thank you, I understand the motivation, the reason I never understood before was quite simple, the description "1x1 convolution" (correctly) described that they were not doing convolutions across pixels. It is good to point out what they aren't doing, they should also point out what they are doing. Humor me by considering: "Today I did not go to the zoo, I did not go to school, ..., and I did not go by foot, nor by vehicle without weels, nor by a vehicle with only one wheel, nor by a vehicle that had 3 or more wheels, and the vehicle was not motorised, and I did not need to stand up on the vehicle" While true, and possibly important to point out what I didn't do, it's generally more helpful to describe what I am doing, like "I rode my bicycle to the supermarket"