3 ms·

You're training it on data already, which by extension will contain information about the game. Training a randomly wired net is like pretending that the train

You're training it on data already, which by extension will contain information about the game. Training a randomly wired net is like pretending that the training data won't give it pre-conceptions, when the training data is what drives the model. (Pretending that closing ones eyes, removes all items from existence)

ninjin 2y ago
My interpretation has always been that any wiring of the network carries its own bias. Thus, a random wiring is biased as well. Your point still stands perfectly on its own though and now I am less sure about the interpretation I have had over all these years.
- quuxplusone 2y ago
  While I certainly see how the story could work as a direct allegory (contents of room = weights; closing eyes = wiring randomly; believing the room empty = believing there are no weights)... FWIW, I always took it much less literally. Sussman has written a whole program to play Tic-Tac-Toe: two players, a 3x3 grid, players alternate turns, the game ends under such-and-such conditions, etc. etc.; and then he thinks that by such a superficial gesture as wiring the net randomly he somehow creates a tabula rasa with no "preconceptions of how to play." Minsky points out, by an equally superficial gesture, that giving up control over one minor aspect of the program (the wiring, the ability to see with one's eyes) really has nothing to do with the issue at hand (how much "understanding" of Tic-Tac-Toe has been baked into the program, how many things exist in the room). Or, to put it back into koan form: One day, Sussman wrote a program that had no preconceptions of how to play Tic-Tac-Toe. The following day, Sussman wrote a program that had no preconceptions of how to play chess. His two programs were not the same.
- Dylan16807 2y ago
  I'm sure you're right. "Preconception" is more specific than bias and would apply before the training starts. Something it learns from the training is not a preconception it had. And I wouldn't assume there even is "training data" that existed when the net was initialized.

ninjin 2y ago

My interpretation has always been that any wiring of the network carries its own bias. Thus, a random wiring is biased as well. Your point still stands perfectly on its own though and now I am less sure about the interpretation I have had over all these years.

quuxplusone 2y ago
While I certainly see how the story could work as a direct allegory (contents of room = weights; closing eyes = wiring randomly; believing the room empty = believing there are no weights)... FWIW, I always took it much less literally. Sussman has written a whole program to play Tic-Tac-Toe: two players, a 3x3 grid, players alternate turns, the game ends under such-and-such conditions, etc. etc.; and then he thinks that by such a superficial gesture as wiring the net randomly he somehow creates a tabula rasa with no "preconceptions of how to play." Minsky points out, by an equally superficial gesture, that giving up control over one minor aspect of the program (the wiring, the ability to see with one's eyes) really has nothing to do with the issue at hand (how much "understanding" of Tic-Tac-Toe has been baked into the program, how many things exist in the room). Or, to put it back into koan form: One day, Sussman wrote a program that had no preconceptions of how to play Tic-Tac-Toe. The following day, Sussman wrote a program that had no preconceptions of how to play chess. His two programs were not the same.
Dylan16807 2y ago
I'm sure you're right. "Preconception" is more specific than bias and would apply before the training starts. Something it learns from the training is not a preconception it had. And I wouldn't assume there even is "training data" that existed when the net was initialized.