3 ms·
Not quite. This and dropout are a good idea to analyze together, but this is not the same thing. Dropout is done _during training_ of the neural network. In ot
by pure-awesome 9y ago
Not quite. This and dropout are a good idea to analyze together, but this is not the same thing.
Dropout is done _during training_ of the neural network. In other words, for a given training example / batch of examples, some of the neurons are deactivated. Those that remain are forced to "pick up the slack" of the missing ones, and so this ends up preventing over-fitting.
What this is, is taking a neural network which has _already been trained_ and seeing how the output differs if we now drop some of the neurons.
It is true that neural networks that have been trained with dropout will probably fare much better under these circumstances.
- mabbo 9y agoIf I'm understanding you, you're saying that this is more of a way to evaluate whether a network has generalized rather than a method to help it generalize. Is that right?
- ashraymalhotra 9y agoThis paper basically proves that neurons which people understand better are not any more "important" to the network than the ones we do not understand as well. It is absolutely true that a generalised network will have much lesser drop off in accuracy as compared to a network which basically just stored the input data.
- bcheung 9y agoIs this a metric that is being used any where? Would be interesting to have 'resilency' metric in addition to the normal 'accuracy' and 'loss' metrics.
- mlazos 9y agoThis just seems true by definition. If you have a learning algorithm that has generalized well, then adding noise to its input shouldn’t result in a large degradation in accuracy.