3 ms·
What if you abandoned gradients/back-prop entirely? They knew that for reinforcement learning this was good clear back in 2017! https://openai.com/blog/evoluti
by Der_Einzige 4y ago
What if you abandoned gradients/back-prop entirely? They knew that for reinforcement learning this was good clear back in 2017!
https://openai.com/blog/evolution-strategies/ https://openai.com/blog/evolution-strategies/
Put engineering effort into creating fast, GPU powered (tensorflow/pytorch) algorithms for neuroevolution of the weights of a neural network. I'm still not convinced that properly leveraged gradient free/Feed-forward only optimizers have been tried yet by researchers - namely because they never actually wrote fast gradient free optimizers!
- pchiusano 4y agoYou might like these posts on gradient free optimization: http://pchiusano.github.io/2019-07-06/learning-without-a-gradient.html http://pchiusano.github.io/2019-07-06/learning-without-a-gra... http://pchiusano.github.io/2020-10-12/learning-without-a-gradient2.html http://pchiusano.github.io/2020-10-12/learning-without-a-gra... I think you’re right that the area is underexplored.
- eismcc 4y agoHas anyone tried using another NN as the “gradient” function? With the hope of making it less “blind?”
- rolisz 4y agoDeepMind has a thing called synthetic gradients: https://www.deepmind.com/blog/decoupled-neural-interfaces-using-synthetic-gradients#:~:text=The%20synthetic%20gradient%20model%20takes,with%20respect%20to%20the%20activations https://www.deepmind.com/blog/decoupled-neural-interfaces-us....
- erwincoumans 4y agoBlack box optimization such as CMA-ES or Augmented Random Search (ars) are used by researchers, for example: CMA-ES: https://blog.otoro.net/2017/10/29/visual-evolution-strategies https://blog.otoro.net/2017/10/29/visual-evolution-strategie... ARS: https://arxiv.org/abs/1803.07055 https://arxiv.org/abs/1803.07055 Those optimizers are efficient up to certain neural network sizes.