3 ms·

Haven't heard the term "freely wired" before, but FAIR released an exploration of randomly wired neural networks which seems conceptually similar. https://arxiv

Haven't heard the term "freely wired" before, but FAIR released an exploration of randomly wired neural networks which seems conceptually similar. https://arxiv.org/abs/1904.01569 https://arxiv.org/abs/1904.01569

monocasa 6y ago
Randomly wired networks reminds me of a classic AI koan: > In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. > “What are you doing?”, asked Minsky. > “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. > “Why is the net wired randomly?”, asked Minsky. > “I do not want it to have any preconceptions of how to play”, Sussman said. > Minsky then shut his eyes. > “Why do you close your eyes?”, Sussman asked his teacher. > “So that the room will be empty.” > At that moment, Sussman was enlightened.
- optimalsolver 6y ago
  Except random search was found to outperform state-of-the-art neural architecture search algorithms: https://arxiv.org/abs/1902.08142 https://arxiv.org/abs/1902.08142
  - greatgoat420 6y ago
    The story is more about how a random set of weights WILL have preconceptions of how to play. If you look at condition numbers or spectra of random normal matrices they are very much not random.
- ad404b8a372f2b9 6y ago
  I don't see how that relates. You don't stick to a single random wiring, you only make the wiring random so that the space of possible wirings of the network is not constrained by the classical sequential paradigm.
ttul 6y ago
Back in engineering school, a million years ago, my partner and I used simulated annealing to “design” a digital circuit implementation in CMOS (a 32 bit CRC), minimizing various parameters like wire length to optimize its function. It worked shockingly well when simulated. I am a huge fan of using randomized starting states and then allowing the computer to discover the best architecture. It produces, if nothing else, surprising results.

monocasa 6y ago

Randomly wired networks reminds me of a classic AI koan: > In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6. > “What are you doing?”, asked Minsky. > “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied. > “Why is the net wired randomly?”, asked Minsky. > “I do not want it to have any preconceptions of how to play”, Sussman said. > Minsky then shut his eyes. > “Why do you close your eyes?”, Sussman asked his teacher. > “So that the room will be empty.” > At that moment, Sussman was enlightened.

optimalsolver 6y ago
Except random search was found to outperform state-of-the-art neural architecture search algorithms: https://arxiv.org/abs/1902.08142 https://arxiv.org/abs/1902.08142
- greatgoat420 6y ago
  The story is more about how a random set of weights WILL have preconceptions of how to play. If you look at condition numbers or spectra of random normal matrices they are very much not random.
ad404b8a372f2b9 6y ago
I don't see how that relates. You don't stick to a single random wiring, you only make the wiring random so that the space of possible wirings of the network is not constrained by the classical sequential paradigm.

ttul 6y ago

Back in engineering school, a million years ago, my partner and I used simulated annealing to “design” a digital circuit implementation in CMOS (a 32 bit CRC), minimizing various parameters like wire length to optimize its function. It worked shockingly well when simulated. I am a huge fan of using randomized starting states and then allowing the computer to discover the best architecture. It produces, if nothing else, surprising results.