3 ms·
I believe this is in reference to the information bottleneck. This is an idea that there is a limit to the performance of a particular network. From an inform
by sjg007 9y ago
I believe this is in reference to the information bottleneck. This is an idea that there is a limit to the performance of a particular network. From an information theory point of view, a deep neural network can be viewed as compressing the input and subsequently generating a code on the other side. So the network is seen as encoding the data and because of that we can bound it using some tools based in information theory.