3 ms·

I love gradient-free optimisers - there are so many interesting problems where there aren't analytical gradients available. An I right in thinking this library

by twic 6y ago

I love gradient-free optimisers - there are so many interesting problems where there aren't analytical gradients available. An I right in thinking this library doesn't contain a simplex / Nelder-Mead optimiser? To me that's the bread and butter of gradient-free optimisation. Also no Powell algorithms like BOBYQA? nlopt has these and many more, and a Python wrapper: https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/ https://nlopt.readthedocs.io/en/latest/NLopt_Algorithms/ So, are those omitted because this library is modern and those are now considered out of date?

simonblanke 6y ago
I am currently working on the Nelder-Mead optimiser. I will also look into "BOBYQA". I am always searching for interesting algorithms that my users need. If you have more suggestions you can open an issue in the repository.
- twic 6y ago
  That's good to hear. I don't have any other specific requests - for me it seems impossible to know what will work on my problems, so i just try things at random to see what works! I don't know the architecture of your project, but is there any way it could include NLopt, and so make all of its algorithms available? Some of them could be useful, and i doubt it's a good use of your time to reimplement them all from scratch.
  - simonblanke 6y ago
    Unfortunately NLopt cannot be included into Gradient-Free-Optimizers. But i would like to implement multiple algorithms that are also preset in NLopt. I think it will help my understanding of optimization algorithms if i implement them myself. I currently have my eyes on the Downhill-simplex, Direct and Powell's Method. But they are quite different from what i have already implemented, so this could take some time.
- sleavey 6y ago
  I've used this, and it works nicely: https://github.com/numericalalgorithmsgroup/pybobyqa https://github.com/numericalalgorithmsgroup/pybobyqa. I'd be happy if it were added to your project, then I could just use yours and have access to a bunch of alternatives with the same API.
bloaf 6y ago
Came here to ask about the Powell method. I find Powell is typically what engineers come up with if asked to optimize something "by hand" without a mathematical model of the system they are optimizing. https://en.wikipedia.org/wiki/Powell%27s_method https://en.wikipedia.org/wiki/Powell%27s_method

by twic 6y ago

simonblanke 6y ago

I am currently working on the Nelder-Mead optimiser. I will also look into "BOBYQA". I am always searching for interesting algorithms that my users need. If you have more suggestions you can open an issue in the repository.

twic 6y ago
That's good to hear. I don't have any other specific requests - for me it seems impossible to know what will work on my problems, so i just try things at random to see what works! I don't know the architecture of your project, but is there any way it could include NLopt, and so make all of its algorithms available? Some of them could be useful, and i doubt it's a good use of your time to reimplement them all from scratch.
- simonblanke 6y ago
  Unfortunately NLopt cannot be included into Gradient-Free-Optimizers. But i would like to implement multiple algorithms that are also preset in NLopt. I think it will help my understanding of optimization algorithms if i implement them myself. I currently have my eyes on the Downhill-simplex, Direct and Powell's Method. But they are quite different from what i have already implemented, so this could take some time.
sleavey 6y ago
I've used this, and it works nicely: https://github.com/numericalalgorithmsgroup/pybobyqa https://github.com/numericalalgorithmsgroup/pybobyqa. I'd be happy if it were added to your project, then I could just use yours and have access to a bunch of alternatives with the same API.

bloaf 6y ago

Came here to ask about the Powell method. I find Powell is typically what engineers come up with if asked to optimize something "by hand" without a mathematical model of the system they are optimizing. https://en.wikipedia.org/wiki/Powell%27s_method https://en.wikipedia.org/wiki/Powell%27s_method