3 ms·
The structure of convolutional neural nets specifies much of the prior knowledge necessary for learning. In other words, the design of these neural nets makes a
by cs702 9y ago
The structure of convolutional neural nets specifies much of the prior knowledge necessary for learning. In other words, the design of these neural nets makes a lot of correct assumptions about the nature of images (stationarity of pixel statistics, locality of pixel dependencies, and so on).
- nerfhammer 9y agois the structure of these CNNs learned or designed? Do they run some kind of optimizer to learn the optimal CNN structure or does some person sit down and pick structures to include in it?
- cs702 9y agoConvolutional layers are designed, by and large, and they're mostly the same everywhere. Yann Le Cun came up with them in the mid-90's, but their academic origins go back to at least the 50's and 60's.
- nerfhammer 9y agothen the results of this paper are sort of surprising that it's that good, if the structure is just relatively old, hand-crafted and traditional.
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]
- mvij 9y agoHandcrafted, but auto optimizing is a hot research topic right now with DeepMind et al. Needs to be evolved at deeper level than what they’re doing now so architecture is discovered, not just optimized.
- ska 9y agoWhich in turn relies on older work. I wonder if anyone has looked at what Mumford-Shaw implications would look like projected onto a CNN?
- dwaltrip 9y agoBy structure, we are simply referring to the number of layers, the number of neurons in each layer, and the specific connections between neurons in each pair of neighboring layers, right? So in this paper, they carefully chose a certain structure, set the weights randomly, and then what happened after that? I understand that they did not then train it with a training data set, but I'm not quite getting what they did with the single distorted image.
- candiodari 9y agoWell given that it's CNNs, you're leaving out weight sharing. So by structure you should also include the demand that the prediction of any NxN patch of the image should be roughly equal to the prediction of any other NxN patch of the image.
- fizixer 9y agoBy structure, do you mean the architecture of the NN? Meaning number of hidden layers, nodes per layer, and their connectivity with each other and with the input and output layers?
- cs702 9y agoNo, the paper means primarily the weight sharing in each kernel filter within each convolutional layer, and the stacking of these layers in deep networks.