3 ms·
Wonderful paper, and also makes sense intuitively that models that perform well on out of training samples, would have a more distributed internal representatio
by mark_l_watson 9y ago
Wonderful paper, and also makes sense intuitively that models that perform well on out of training samples, would have a more distributed internal representation, and single neurons would be less important. The “money plot” in the article shows that the most damaging neurons to remove are the least specific.
More training data, make the size of the model as small as possible - something I have done since the late 1980s. Not only are neural network engineering techniques rapidly improving, but so is our intuition into how they work.