3 ms·
This is not compression. The author focuses on the small number of nodes and calls it compression but it's the edges that encode the information and there are
by jcoffland 11y ago
This is not compression. The author focuses on the small number of nodes and calls it compression but it's the edges that encode the information and there are a lot of them.
- tgflynn 11y agoYou can get (lossy) compression by using an autoencoder with a hidden layer that has fewer nodes than the number of inputs, since then you only need to store the hidden layer excitations for each example. The weights and network architecture itself are part of the compressor, not the compressed data. As for how well this works compared to other approaches, I don't know, and this blog post certainly doesn't address the question.
- zardeh 11y ago>If you count the number of connections on that last picture you will notice that there are fewer than there were in the first network. There were 55=25 at first, now there are 52*2=20. That reduction would be larger if I had more input and output nodes. For 10 nodes that reduction would be from 100 connections down to 40 when inserting two nodes in the middle. That’s where the compression in the title of this blog post comes from.