3 ms·
Dropout is traditionally only applied at training time: c.f. "At test time the weights are multiplied by their probability of their associated units’ dropout."
by cr4zy 8y ago
Dropout is traditionally only applied at training time: c.f. "At test time the weights are multiplied by their probability of their associated units’ dropout." from the medium post you linked. I _suspect_ that leaving it on during test has been accidentally tried a bunch of times though :) - so probably is not effective, although it could be interesting to experiment more intentionally with test-time noise. Our brains do not separate so cleanly between train and test, and so it seems more natural for the noise to be always on (if that's even the case as it seems hard to know what's noise when you haven't completely decoded the brain's activity). Decoding the brain seems to be super promising though and I hope we get good enough so as to upload our minds someday. Here's an interesting related article I read recently towards that end: http://www.foldl.me/uploads/papers/ccn2018.pdf http://www.foldl.me/uploads/papers/ccn2018.pdf