> You can start with a random neural net. It's not exactly empty, but it's ok. One interesting thing about random NNs is how much they can already do. For exam

> You can start with a random neural net. It's not exactly empty, but it's ok. One interesting thing about random NNs is how much they can already do. For example, you can do single-shot image inpainting or superresolution with an untrained randomly-wired CNN: https://dmitryulyanov.github.io/deep_image_prior https://dmitryulyanov.github.io/deep_image_prior In RL, you can use randomly sampled NNs to create various artificial arbitrary 'reward functions' to force an RL agent to explore an environment & learn the dynamics, and then when you give it the real reward function, it learns much faster how to optimize it. Similarly, you can sample random NNs to execute for entire trajectories for 'deep exploration', providing demonstrations of potentially long-range strategies much more efficiently than simple random-action strategies. In 'reservoir sampling', as I understand it, you don't even bother training the NN, you just randomly initialize it and train a simple model on the outputs, assuming that some of the random highly nonlinear relationships encoded into the NN will turn out to be useful, which sounds crazy but apparently works. Makes one think about Tegmark's interpretations.