I wrote a bit about the theoretical reasons DNNs work here: http://lesswrong.com/lw/m9p/approximating_solomonoff_induction/ http://lesswrong.com/lw/m9p/approxim

I wrote a bit about the theoretical reasons DNNs work here: http://lesswrong.com/lw/m9p/approximating_solomonoff_induction/ http://lesswrong.com/lw/m9p/approximating_solomonoff_inducti... There's some more interesting discussion in the comments there.