4 ms·

I am reminded of this immortal koan: Sussman attains enlightenment In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP

I am reminded of this immortal koan: Sussman attains enlightenment In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. “What are you doing?”, asked Minsky. “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. “Why is the net wired randomly?”, asked Minsky. “I do not want it to have any preconceptions of how to play”, Sussman said. Minsky then shut his eyes. “Why do you close your eyes?”, Sussman asked his teacher. “So that the room will be empty.” At that moment, Sussman was enlightened.

0x1ceb00da 2y ago
I didn't understand.
- dbetteridge 2y ago
  You're training it on data already, which by extension will contain information about the game. Training a randomly wired net is like pretending that the training data won't give it pre-conceptions, when the training data is what drives the model. (Pretending that closing ones eyes, removes all items from existence)
  - ninjin 2y ago
    My interpretation has always been that any wiring of the network carries its own bias. Thus, a random wiring is biased as well. Your point still stands perfectly on its own though and now I am less sure about the interpretation I have had over all these years.
    - quuxplusone 2y ago
      While I certainly see how the story could work as a direct allegory (contents of room = weights; closing eyes = wiring randomly; believing the room empty = believing there are no weights)... FWIW, I always took it much less literally. Sussman has written a whole program to play Tic-Tac-Toe: two players, a 3x3 grid, players alternate turns, the game ends under such-and-such conditions, etc. etc.; and then he thinks that by such a superficial gesture as wiring the net randomly he somehow creates a tabula rasa with no "preconceptions of how to play." Minsky points out, by an equally superficial gesture, that giving up control over one minor aspect of the program (the wiring, the ability to see with one's eyes) really has nothing to do with the issue at hand (how much "understanding" of Tic-Tac-Toe has been baked into the program, how many things exist in the room). Or, to put it back into koan form: One day, Sussman wrote a program that had no preconceptions of how to play Tic-Tac-Toe. The following day, Sussman wrote a program that had no preconceptions of how to play chess. His two programs were not the same.
    - Dylan16807 2y ago
      I'm sure you're right. "Preconception" is more specific than bias and would apply before the training starts. Something it learns from the training is not a preconception it had. And I wouldn't assume there even is "training data" that existed when the net was initialized.

0x1ceb00da 2y ago

I didn't understand.

dbetteridge 2y ago
You're training it on data already, which by extension will contain information about the game. Training a randomly wired net is like pretending that the training data won't give it pre-conceptions, when the training data is what drives the model. (Pretending that closing ones eyes, removes all items from existence)
- ninjin 2y ago
  My interpretation has always been that any wiring of the network carries its own bias. Thus, a random wiring is biased as well. Your point still stands perfectly on its own though and now I am less sure about the interpretation I have had over all these years.
  - quuxplusone 2y ago
    While I certainly see how the story could work as a direct allegory (contents of room = weights; closing eyes = wiring randomly; believing the room empty = believing there are no weights)... FWIW, I always took it much less literally. Sussman has written a whole program to play Tic-Tac-Toe: two players, a 3x3 grid, players alternate turns, the game ends under such-and-such conditions, etc. etc.; and then he thinks that by such a superficial gesture as wiring the net randomly he somehow creates a tabula rasa with no "preconceptions of how to play." Minsky points out, by an equally superficial gesture, that giving up control over one minor aspect of the program (the wiring, the ability to see with one's eyes) really has nothing to do with the issue at hand (how much "understanding" of Tic-Tac-Toe has been baked into the program, how many things exist in the room). Or, to put it back into koan form: One day, Sussman wrote a program that had no preconceptions of how to play Tic-Tac-Toe. The following day, Sussman wrote a program that had no preconceptions of how to play chess. His two programs were not the same.
  - Dylan16807 2y ago
    I'm sure you're right. "Preconception" is more specific than bias and would apply before the training starts. Something it learns from the training is not a preconception it had. And I wouldn't assume there even is "training data" that existed when the net was initialized.