3 ms·
People have known that training NNs (for any purpose) using evolution works well since the 1990s. The rise of the NN frameworks has made doing differentiation
by mdda 10y ago
People have known that training NNs (for any purpose) using evolution works well since the 1990s. The rise of the NN frameworks has made doing differentiation much easier now than it was before (and having gradient hints is intuitively a good idea). But for OpenAI to allow their PR people to declare this as a novel advance is ... surprising.
- p1esk 10y agoCitation for training NN for image classification task where evolution works well? Let's say you want to use a genetic algorithm to find a good set of weights: you generate, mutate, combine and select many random networks, and repeat this process many times. How many networks and how many times? That depends on the length of your chromosome and complexity of the task. Networks that work well for image classification need at least a million weights. The entire set of weights is a single chromosome. You realize now how computationally intractable this task is on modern hardware?
- mdda 10y ago> NN for image classification task You've created your own straw man here. > "You realize now how computationally intractable this task is on modern hardware?" Here are the people that prove it isn't computationally intractable : https://blog.openai.com/evolution-strategies/ https://blog.openai.com/evolution-strategies/ - but to say they've discovered a new breakthrough method is over-selling the result.
- p1esk 10y agoYou said: "training NNs (for any purpose) using evolution works well". I gave you an example of a purpose where it does not work well. So, let me ask you again: can you give an example of evolutionary methods that work well when applied to training NNs, other than this breakthrough by OpenAI, which only works for RL?