I don't have a good intuition of neurons in terms of entropy, but I disagree with the conclusion they draw from their interpretability/importance graph - "Surpr

I don't have a good intuition of neurons in terms of entropy, but I disagree with the conclusion they draw from their interpretability/importance graph - "Surprisingly, we found that there was little relationship between selectivity and importance." While highly selective neurons are not more or less important than generic neurons, the highly important neurons are always very non-selective. I think you call them high entropy neurons? "So the fact that they're saying they have the same effect as more random ones is interesting". Yes, they are saying that, but their graph doesn't show that. I think their graph supports your intuition, that if you ever plan to delete neurons from a NN, while minimizing the degradation, you should start with the low entropy neurons first.