3 ms·
> Totally obsurd. Gradient descent will 'almost' always get stuck in a local optima Well... just as evolutionary algorithms are essentially elaborate versions
by SeanLuke 12y ago
> Totally obsurd. Gradient descent will 'almost' always get stuck in a local optima
Well... just as evolutionary algorithms are essentially elaborate versions of hill-climbing which enable global search, likewise there are many well-established forms of gradient descent which are global searchers as well.
I think the key distinguishing feature is that gradient descent makes strong assumptions about the nature of the space being searched; and metaheuristics make much more general assumptions.
- GordyMD 12y agoGood point. This is key, as you say. Meta-heuristics have the flexibility of providing good performance on all Combinatorial Optimisation Problems. But meta-heuristics, can also be given domain knowledge. and/or be combined with a a heuristic. E.g. Ant Colony Optimisation techniques for Travelling Salesman Problems that combines ACO with Local Search.