4 ms·
New release of Gradient-Free-Optimizers with two new evolutionary algorithms
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- 528491 2y agoThe new release adds the Genetic Algorithm and Differential Evolution. Also check out the documentation for the new optimization algorithms: https://simonblanke.github.io/gradient-free-optimizers-documentation/1.5/optimizers/ https://simonblanke.github.io/gradient-free-optimizers-docum...
- spencerchubb 2y agothe title mentions two new evolutionary algorithms, and I think it would be good if the title clarifies which two are new. it seems the new ones are the genetic algorithm and differential evolution i find it interesting that Gradient-Free-Optimizers is used in a library for hyperparameter optimization. so in essence using a gradient-free approach to optimize a gradient-based approach
- 528491 2y agoThe title was to long, but I edited my comment for clarification. Hyperparameter optimization is normally done with gradient-free-optimization techniques, since a gradient cannot be computed. This is opposed to the optimization of parameters during training (e.g. neural network weights).
- nickpsecurity 2y agoI was looking for papers on this topic a while back. Here’s one which lists many derivative-free methods: https://arxiv.org/abs/1904.11585 https://arxiv.org/abs/1904.11585
- spencerchubb 2y agothat's quite the comprehensive paper! it's almost like a mini textbook
- jglamine 2y agoAre there any advantages of this over scipy.optimize? That's what I've used in the past. Trying to understand if it's worth switching. It looks like they have many of the same algorithms.
- ris 2y agoI've previously used pagmo2 for this kind of thing with some amount of success. Might be worth giving this one a try as pagmo2's c++ patterns can be something of a mindfuck.
- Matumio 2y agoWhen I read "evolution strategy" I was pretty sure to find some variant of the Canonical Evolution Strategy (as in https://arxiv.org/pdf/1802.08842 https://arxiv.org/pdf/1802.08842) or maybe CMA-ES or something related. But the implementation looks like a GA. Maybe the term means different things to different people...?
- 528491 2y agoThanks for pointing that out. The current implementation is not self-adapting the parameters (like mutation strength) of the individuals in the population: https://github.com/SimonBlanke/Gradient-Free-Optimizers/issues/50 https://github.com/SimonBlanke/Gradient-Free-Optimizers/issu...