5 ms·
I still remember a story I heard Gerry Sussman tell about Marvin, from when Gerry was a grad student. Gerry was programming a neural net (or something like it;
by ScottBurson 13y ago
I still remember a story I heard Gerry Sussman tell about Marvin, from when Gerry was a grad student. Gerry was programming a neural net (or something like it; I forget exactly what) and told Marvin he was planning to use a random number generator to initialize the weights in the network "so it won't have any biases". Marvin replied, in his careful, laconic way, "Of course it will have biases. You just won't know what they are."
This comment has never seemed as stunningly insightful to me as it evidently did to Gerry, but it's probably one of those things that is obvious once stated, but not before.
Anyway I'm glad to see Marvin winning this prize.
- xtacy 13y agoThe story is in "The Hacker Koan:" http://en.wikipedia.org/wiki/Hacker_koan http://en.wikipedia.org/wiki/Hacker_koan
- eli_gottlieb 13y agoYou know, I just now finally understood what was going on there: a randomized map matches no territory. Enlightenment achieved.
- Houshalter 13y agoBut it does match a territory. It's just a random one
- eli_gottlieb 13y agoOk, let me put it this way: if there happens to exist some territory matched by your randomly-generated map, you still have the problem that a randomly-generated map was not in any way caused by the territory. Of course, that's just unpacking the word "match".
- Double_Cast 13y agoLet's suppose you're lost in the Amazon. Would you not prefer a professionally-drawn map of the Amazon over a "map" I whimsically scribbled while blindfolded? After all, my map might not correspond to the Amazon, but it must correspond to something.
- Houshalter 13y agoI'm not sure if you are responding the the right comment or if you understood what I said. The point is when you initialize weights randomly, you aren't starting with a blank map and then adding stuff to it. You are starting with a random map and then trying to correct it. Random bias is still bias.
- Double_Cast 13y agoMy bad. The way I interpreted your comment was > But (a random map is useful because) it does match a territory (somewhere out in the actual universe), it's just a (location that's hard to find). I don't know what I was thinking.
- bch 13y agoNot necessarily. See also: "when the map and the terrain differ, trust the terrain."
- jes5199 13y agoI've seen some suggestions that the human brain is full of random pattern generators, and that some kinds of learning are selecting which pre-existing patterns most reliably match the external stimuli