3 ms·
To me the fact that more information is transmitted with an intermediate number of connections than with a strongly connected network wasn't immediately obvious
by anonymousDan 4y ago
To me the fact that more information is transmitted with an intermediate number of connections than with a strongly connected network wasn't immediately obvious at first glance. I guess there is a link to entropy, i.e. how surprised can you be by the information received at one end of the network given its connectivity.
- visarga 4y agoI think it relates to sparsity. Rarely a fully connected network is better, they need more data to train and have less inductive bias.