8 ms·

New Theory Cracks Open the Black Box of Deep Neural Networks

36 points by breck 9y ago

mrfusion 9y ago
How big of a deal is this from 1-10?
- gcoda 9y ago
  Like any other theory in science. Nuclear power or glow in the dark toys, usually both, if enough time pass
mrfusion 9y ago
So there’s not chance the brain does back propagation right? I wonder what it might do instead?
- carlmr 9y ago
  I wouldn't say not a chance, we have no clue what the brain does. Only because a brain can learn building on old concepts doesn't mean anything about the basic algorithm being different. The brain is a much bigger and longer living neural network. What they were explaining with the child learning character recognition with the knowledge about strokes. Why would you think our brain doesn't simply connect some of the stroke recognizing neural networks to a higher level neural network built for character recognition. You can increase the depth of the network basically indefinitely. Stack various neural networks to produce more complex ones, etc.
  - Shikadi 9y ago
    I don't think it's a good idea to be calling the human brain a neural network in this context, as cnns/dnns only sort of vaguely resemble brain neurons. The differences are massive, both due to different neurotransmitters as well as different connections being formed, and other stuff
    - carlmr 9y ago
      That doesn't have much to do with my point, also the brain is the original neural network. If anything you should argue to call artificial neural networks by a different name.
notaboutdave 9y ago
Waste of a read! Reasoning that going from many input nodes to few output nodes results in compression (or shedding of nonessential data) isn't a brilliant theory, it's common sense. The author even beats the dead horse of AI being a potential existential threat.
- Zee2 9y ago
  That's exactly what I was thinking. Wow, a neural network discards information at each layer by reducing the number of cells in each matrix? Water is wet. How else would a network make a final decision? If it never discarded any information, it would be just as heterogeneous of a dataset as when it started. It would be as if a linear regression algorithm "fit" a dataset by connecting the dots, except even more useless. I truly don't understand what this "new" theory is trying to say.
  - carlmr 9y ago
    It's so basic I was thinking they could have called me, someone who studied neural networks only for half a year in uni, ten years ago. I could have told them that. Either the journalist compressed the information a bit too much and lost the essential amazing insight. Or it's just not amazing.
  - nerdponx 9y ago
    Everything is obvious in hindsight. Too bad you didn't write up and publish your brilliant thoughts before this guy did. The actual talk is pretty technical and sophisticated. The conclusion is not just that neural networks discard information; it's a rigorous derivation of exactly how they do so.
- Nomentatus 9y ago
  The insight may be to show why image recognition has turned out to be a fairly easy task for deep learning - because just discarding pixels (the right pixels, but still) gets you a very long way toward your solution. Other problems that a brain handles quite well may not yield to such a simple process/analysis.
- 9y ago
rdlecler1 9y ago
NN look like black boxes because we’re not stripping out spurious axons in the network to understand the computational circuit. If you can remove a weight (w_ij -> O) from a NN with no change in the network’s function across a broad spectrum of inputs then it clearly wasn’t necessary in the first place as that weighted edge was not involved in the information processing. Training networks will not only tune certain weights to dial up on signal but they will also (effectively) zero out certain weights that don’t carry information (if they’re not effectively zeroed our then they introduce noise and compromise the network function), in effect reducing the noise and focusing on signal. For anyone interested I wrote a paper in Natures Molecular Systems Biology in 2008 which goes into this. It describes artificial gene regulatory networks but my background was in AI and the NN math is the same. I believe the results are universally applicable across a broad range of NN-class network. Lots of relevant remarks in the supplementary as well. This includes a description of an algorithm to prune the network of spurious interactions. http://m.msb.embopress.org/content/4/1/213.abstract http://m.msb.embopress.org/content/4/1/213.abstract

36 points by breck 9y ago

mrfusion 9y ago

How big of a deal is this from 1-10?

gcoda 9y ago
Like any other theory in science. Nuclear power or glow in the dark toys, usually both, if enough time pass

mrfusion 9y ago

So there’s not chance the brain does back propagation right? I wonder what it might do instead?

carlmr 9y ago
I wouldn't say not a chance, we have no clue what the brain does. Only because a brain can learn building on old concepts doesn't mean anything about the basic algorithm being different. The brain is a much bigger and longer living neural network. What they were explaining with the child learning character recognition with the knowledge about strokes. Why would you think our brain doesn't simply connect some of the stroke recognizing neural networks to a higher level neural network built for character recognition. You can increase the depth of the network basically indefinitely. Stack various neural networks to produce more complex ones, etc.
- Shikadi 9y ago
  I don't think it's a good idea to be calling the human brain a neural network in this context, as cnns/dnns only sort of vaguely resemble brain neurons. The differences are massive, both due to different neurotransmitters as well as different connections being formed, and other stuff
  - carlmr 9y ago
    That doesn't have much to do with my point, also the brain is the original neural network. If anything you should argue to call artificial neural networks by a different name.

notaboutdave 9y ago

Waste of a read! Reasoning that going from many input nodes to few output nodes results in compression (or shedding of nonessential data) isn't a brilliant theory, it's common sense. The author even beats the dead horse of AI being a potential existential threat.

Zee2 9y ago
That's exactly what I was thinking. Wow, a neural network discards information at each layer by reducing the number of cells in each matrix? Water is wet. How else would a network make a final decision? If it never discarded any information, it would be just as heterogeneous of a dataset as when it started. It would be as if a linear regression algorithm "fit" a dataset by connecting the dots, except even more useless. I truly don't understand what this "new" theory is trying to say.
- carlmr 9y ago
  It's so basic I was thinking they could have called me, someone who studied neural networks only for half a year in uni, ten years ago. I could have told them that. Either the journalist compressed the information a bit too much and lost the essential amazing insight. Or it's just not amazing.
- nerdponx 9y ago
  Everything is obvious in hindsight. Too bad you didn't write up and publish your brilliant thoughts before this guy did. The actual talk is pretty technical and sophisticated. The conclusion is not just that neural networks discard information; it's a rigorous derivation of exactly how they do so.
Nomentatus 9y ago
The insight may be to show why image recognition has turned out to be a fairly easy task for deep learning - because just discarding pixels (the right pixels, but still) gets you a very long way toward your solution. Other problems that a brain handles quite well may not yield to such a simple process/analysis.
9y ago

rdlecler1 9y ago

NN look like black boxes because we’re not stripping out spurious axons in the network to understand the computational circuit. If you can remove a weight (w_ij -> O) from a NN with no change in the network’s function across a broad spectrum of inputs then it clearly wasn’t necessary in the first place as that weighted edge was not involved in the information processing. Training networks will not only tune certain weights to dial up on signal but they will also (effectively) zero out certain weights that don’t carry information (if they’re not effectively zeroed our then they introduce noise and compromise the network function), in effect reducing the noise and focusing on signal. For anyone interested I wrote a paper in Natures Molecular Systems Biology in 2008 which goes into this. It describes artificial gene regulatory networks but my background was in AI and the NN math is the same. I believe the results are universally applicable across a broad range of NN-class network. Lots of relevant remarks in the supplementary as well. This includes a description of an algorithm to prune the network of spurious interactions. http://m.msb.embopress.org/content/4/1/213.abstract http://m.msb.embopress.org/content/4/1/213.abstract