4 ms·
That's exactly what I was thinking. Wow, a neural network discards information at each layer by reducing the number of cells in each matrix? Water is wet. How
by Zee2 9y ago
That's exactly what I was thinking. Wow, a neural network discards information at each layer by reducing the number of cells in each matrix? Water is wet.
How else would a network make a final decision? If it never discarded any information, it would be just as heterogeneous of a dataset as when it started. It would be as if a linear regression algorithm "fit" a dataset by connecting the dots, except even more useless.
I truly don't understand what this "new" theory is trying to say.
- carlmr 9y agoIt's so basic I was thinking they could have called me, someone who studied neural networks only for half a year in uni, ten years ago. I could have told them that. Either the journalist compressed the information a bit too much and lost the essential amazing insight. Or it's just not amazing.
- nerdponx 9y agoEverything is obvious in hindsight. Too bad you didn't write up and publish your brilliant thoughts before this guy did. The actual talk is pretty technical and sophisticated. The conclusion is not just that neural networks discard information; it's a rigorous derivation of exactly how they do so.