3 ms·
You might like these posts on gradient free optimization: http://pchiusano.github.io/2019-07-06/learning-without-a-gradient.html http://pchiusano.github.io/201
by pchiusano 4y ago
You might like these posts on gradient free optimization:
http://pchiusano.github.io/2019-07-06/learning-without-a-gradient.html http://pchiusano.github.io/2019-07-06/learning-without-a-gra...
http://pchiusano.github.io/2020-10-12/learning-without-a-gradient2.html http://pchiusano.github.io/2020-10-12/learning-without-a-gra...
I think you’re right that the area is underexplored.
- eismcc 4y agoHas anyone tried using another NN as the “gradient” function? With the hope of making it less “blind?”
- rolisz 4y agoDeepMind has a thing called synthetic gradients: https://www.deepmind.com/blog/decoupled-neural-interfaces-using-synthetic-gradients#:~:text=The%20synthetic%20gradient%20model%20takes,with%20respect%20to%20the%20activations https://www.deepmind.com/blog/decoupled-neural-interfaces-us....