8 ms·
I'm also curious about gradient-less algorithms For non deep learning applications, Nelder-Mead saved my butt a fees times
by VHRanger 2y ago
I'm also curious about gradient-less algorithms
For non deep learning applications, Nelder-Mead saved my butt a fees times
- amelius 2y agoChatGPT also advised me to use NM a couple of times, which was neat.
- imurray 2y agoNelder–Mead has often not worked well for me in moderate to high dimensions. I'd recommend trying Powell's method if you want to quickly converge to a local optimum. If you're using scipy's wrappers, it's easy to swap between the two: https://docs.scipy.org/doc/scipy/reference/optimize.html#local-multivariate-optimization https://docs.scipy.org/doc/scipy/reference/optimize.html#loc... For nastier optimization problems there are lots of other options, including evolutionary algorithms and Bayesian optimization: https://facebookresearch.github.io/nevergrad/ https://facebookresearch.github.io/nevergrad/ https://github.com/facebook/Ax https://github.com/facebook/Ax
- woadwarrior01 2y agoLook into zeroth-order optimizers and CMA-ES.
- analog31 2y agoIt's with the utmost humility that I confess to falling back on "just use Nelder-Mead" in *scipy.optimize* when something is ill behaved. I consider it to be a sign that I'm doing something wrong, but I certainly respect its use.
- kernc 2y agoSAMBO does a good job of finding the global optimum in a black-box manner even compared to Nelder-Mead, according to its own benchmark ... https://sambo-optimization.github.io https://sambo-optimization.github.io
- deleted 2y ago[deleted]