6 ms·

If the weight is positive that feature helps the neuron fire and vice versa. If the neural network is processing images, the weights form and image too. You ca

If the weight is positive that feature helps the neuron fire and vice versa. If the neural network is processing images, the weights form and image too. You can tell what a lot of the units are "looking" for by plotting the weights as an image. page 14, Figure 3: jmlr.org/papers/volume11/erhan10a/erhan10a.pdf You can clearly see various digit deformations in the weights. (white is positive weight, black is negative weight typically) Lots of papers do this, this one was happened to be the first one I managed to google. So the first layer is normally readable because weights are in the same space as the input feature space. The second layer is normally a jumble as its randomly initialised before convergence. But in the paper's example you can imagine the 2nd layer output unit representing a final classification of a 4, is probably summing up all the 4 deformation detectors in layer 1 and negatively summing up everything else.

tlarkworthy 13y ago
Actually I just read that paper properly and I see they did visualise the higher level layer weights too: "For visualizing what units do on the 2nd and 3rd layer, we used the activation maxi- mization technique described by Erhan et al. (2009)"[1] So there you have it! You can visualise all of a neural network! But you have to implement some specific numerics to get the data out. Reading the first layer is trivial, and subsequent layers need some analysis. [1] Erhan 2009 Visualizing higher layer features of a deep network (good find, thanks for asking the right questions!)
foobarqux 13y ago
> If the weight is positive that feature helps the neuron fire and vice versa. But that doesn't tell you much does it? It's effect could be reversed or minimized in subsequent layers. Haven't read it yet but thanks for the paper.
- tlarkworthy 13y ago
  Just look at the one figure in the paper and you will see. Because your final layer is positively encoded, that polarity trickles down through back propogation. I also think the positive weights thing is to do with the way features normally work. It makes sense to look for corners, not anti corners for object detection.

tlarkworthy 13y ago

Actually I just read that paper properly and I see they did visualise the higher level layer weights too: "For visualizing what units do on the 2nd and 3rd layer, we used the activation maxi- mization technique described by Erhan et al. (2009)"[1] So there you have it! You can visualise all of a neural network! But you have to implement some specific numerics to get the data out. Reading the first layer is trivial, and subsequent layers need some analysis. [1] Erhan 2009 Visualizing higher layer features of a deep network (good find, thanks for asking the right questions!)

foobarqux 13y ago

> If the weight is positive that feature helps the neuron fire and vice versa. But that doesn't tell you much does it? It's effect could be reversed or minimized in subsequent layers. Haven't read it yet but thanks for the paper.

tlarkworthy 13y ago
Just look at the one figure in the paper and you will see. Because your final layer is positively encoded, that polarity trickles down through back propogation. I also think the positive weights thing is to do with the way features normally work. It makes sense to look for corners, not anti corners for object detection.