3 ms·

> the combination of neural network topology + the training procedure sets up an implicit prior [...] That's what I got from the koan. Sussman thinks that a ra

> the combination of neural network topology + the training procedure sets up an implicit prior [...] That's what I got from the koan. Sussman thinks that a randomly wired NN has no prior, but that's false. Same as Minsky "thinks" that the room is empty when he closes his eyes. EDIT: actually if you read the source (spetharrific.tumblr.com) it spells this out.

kgwgk 9y ago
Out of sight, out of mind.
Bartweiss 9y ago
That's certainly the moral. I think what's changed with the new deep learning paradigm is a willingness to accept that implicit prior, not so much because it's good as because we don't know how to improve it consistently.