3 ms·
I agree with you, most of those methods are local and optimization is a form of search typically used for continuous and usually differentiable spaces. One can
by Dn_Ab 16y ago
I agree with you, most of those methods are local and optimization is a form of search typically used for continuous and usually differentiable spaces. One can also say search is a form of optimization.
But Based on my readings genetic algorithms are more known to be susceptible to local minima, while simulated annealing in theory is a global method.
---
As for genetic algorithms - for optimization I prefer Differential Evolution. For Exploration maybe genetic algorithms are more fruitfull although I think Genetic programming and Learning Classifier Systems be slept on. Speaking of slept on - another under appreciated but much more used method is stochastic gradient descent. Global optimums might be overrated.
- SeanLuke 16y ago> But Based on my readings genetic algorithms are more known to be susceptible to local minima, while simulated annealing in theory is a global method. Both simulated annealing and genetic algorithms are global methods, and both can get caught in local minima. The same goes for pretty much every other global stochastic optimization method out there. Simulated annealing got this "special" reputation among engineers back in the '80s because the Metropolis algorithm had a formal proof of global guarantees -- run it long enough and it's guaranteed to find the global optimum. But nowadays most every algorithm in this category has similar guarantees. It's relatively easy to make such a guarantee as it turns out. I like differential evolution too, though it's pretty exploitative.