Using dropout with deep neural networks is an (extremely cheap) way to gain many of the benefits of using an ensemble. If you want a great overview of the dropo

Using dropout with deep neural networks is an (extremely cheap) way to gain many of the benefits of using an ensemble. If you want a great overview of the dropout technique, watch this tech talk by Hinton: http://www.youtube.com/watch?v=DleXA5ADG78 http://www.youtube.com/watch?v=DleXA5ADG78 Also, the recent maxout algorithm (from Montreal group, authors of Theano) that got state of the art results on several datasets is essentially just an algorithm designed to do particularly well with dropout (as far as I understand it).