3 ms·

For anyone looking for a free optimization tool in python, sypy.opimize is easy to use. Eg if I have a complicated function revenue([A,B,C,D]) I can define ob

by oli5679 10y ago

For anyone looking for a free optimization tool in python, sypy.opimize is easy to use. Eg if I have a complicated function revenue([A,B,C,D]) I can define obj([A,B,C,D]) = -1* revenue([A,B,C,D]) and use: >>>import numpy as np >>>from scipy.optimize import minimize >>>X0 = np.array([1.5, 0.7, 1.2, 100]) >>>options={'xtol': 1e-4, 'disp': True}) >>>X* = minimize(obj, x0, method='nelder-mead', options) http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html http://docs.scipy.org/doc/scipy/reference/tutorial/optimize....

ogrisel 10y ago
scipy.optimize can only do single-criterion optimization (scalar-valued objective function instead of vector-valued objective functions).
- radarsat1 10y ago
  > The tradeoffs between these two criteria can either be managed by some supervisory decision maker (the driver of the car in this example) or by merging the multiple criteria into some single criteria and phrasing the problem as a standard optimization problem. You always have to reduce the problem to a scalar if you want a single answer.
  - ogrisel 10y ago
    You might want to discover the full Pareto frontier to gain some insights on the structure of the trade-off. I think the best methods to explore the Pareto frontier are based on the concepts of evolutionary computation like NSGA-II and SPEA-2: https://en.wikipedia.org/wiki/Multi-objective_optimization#A_posteriori_methods https://en.wikipedia.org/wiki/Multi-objective_optimization#A...
- oli5679 10y ago
  True, you need to assign weights to a vector of outputs as discussed in this article. After that, scipy.optimize is easy to use though.
Zephyr314 10y ago
The methods in scipy.optimize are great if your function is serial, cheap, deterministic, and convex. Many ML models and real world problems don't fit into this context though. We have a Jupyter notebook showing how SigOpt compares to several scipy.optimize methods as well as standard methods like grid/random on a simple non-convex problem here [1]. These results only get more striking as the dimensionality increases. [1]: https://github.com/sigopt/sigopt-examples/blob/master/ipython-notebook-example/SigOpt_Introduction.ipynb https://github.com/sigopt/sigopt-examples/blob/master/ipytho...
- oli5679 10y ago
  Many of these scipy methods can cope with concave, noisy functions. There's a bit of skill/alchemy with selecting starting values and tolerance parameters. Interesting to see that your approach has superior performance in that case, but in many cases these free tools with no need for an api would be sufficient.

by oli5679 10y ago

ogrisel 10y ago

scipy.optimize can only do single-criterion optimization (scalar-valued objective function instead of vector-valued objective functions).

radarsat1 10y ago
> The tradeoffs between these two criteria can either be managed by some supervisory decision maker (the driver of the car in this example) or by merging the multiple criteria into some single criteria and phrasing the problem as a standard optimization problem. You always have to reduce the problem to a scalar if you want a single answer.
- ogrisel 10y ago
  You might want to discover the full Pareto frontier to gain some insights on the structure of the trade-off. I think the best methods to explore the Pareto frontier are based on the concepts of evolutionary computation like NSGA-II and SPEA-2: https://en.wikipedia.org/wiki/Multi-objective_optimization#A_posteriori_methods https://en.wikipedia.org/wiki/Multi-objective_optimization#A...
oli5679 10y ago
True, you need to assign weights to a vector of outputs as discussed in this article. After that, scipy.optimize is easy to use though.

Zephyr314 10y ago

The methods in scipy.optimize are great if your function is serial, cheap, deterministic, and convex. Many ML models and real world problems don't fit into this context though. We have a Jupyter notebook showing how SigOpt compares to several scipy.optimize methods as well as standard methods like grid/random on a simple non-convex problem here [1]. These results only get more striking as the dimensionality increases. [1]: https://github.com/sigopt/sigopt-examples/blob/master/ipython-notebook-example/SigOpt_Introduction.ipynb https://github.com/sigopt/sigopt-examples/blob/master/ipytho...

oli5679 10y ago
Many of these scipy methods can cope with concave, noisy functions. There's a bit of skill/alchemy with selecting starting values and tolerance parameters. Interesting to see that your approach has superior performance in that case, but in many cases these free tools with no need for an api would be sufficient.