3 ms·
I think that you could consider the co-adaption as a form of greedy local optima (I think this is a very handwavy explanation, though, and I'm having trouble fo
by zardeh 10y ago
I think that you could consider the co-adaption as a form of greedy local optima (I think this is a very handwavy explanation, though, and I'm having trouble forming it better, so take with that what you will). Dropout prevents that by not allowing the 'greedily optimal paths' to form, since sometimes they don't exist, so you can't rely on them too much.
- brianchu 10y agoSure, but dropout actually increases training error (makes you less likely to find the globally optimal training error), with (sometimes) a decrease in generalization error (test error). So any connection is very thin.
- zardeh 10y agoOh actually that's a really good point and I think makes the difference clear (there isn't such a train/test divide in the driving example). Thank you :)