I wrote a better explanation for why neural nets work here: http://houshalter.tumblr.com/post/120134087595/approximating-solomonoff-induction http://houshalter.

I wrote a better explanation for why neural nets work here: http://houshalter.tumblr.com/post/120134087595/approximating-solomonoff-induction http://houshalter.tumblr.com/post/120134087595/approximating... The answer is that neural networks are a somewhat crude approximation of Solomonoff induction. Solomonoff induction is an ideal, perfect machine learning method. It only assumes that the universe is a random computer program, and tries to infer which one.