4 ms·
Genetic algorithms are (and were already back in 2000) a pretty decent and -more importantly- generic solution to the problem of global optimization (as opposed
by ur-whale 8y ago
Genetic algorithms are (and were already back in 2000) a pretty decent and -more importantly- generic solution to the problem of global optimization (as opposed to local optimization) when the problem to optimize has some sort of (maybe not-so-smooth) structure.
Many of the recent "AI" development very often boil down to finding a local extremum using some sort of ski down the slope optimization program (aka "training"). These techniques very rarely tackles global optimization, or when they do, bundle it up under the moniker "hyper-parameter tuning".
A good example of something that falls under "global optimization" and isn't often tackled in deep learning would be finding the correct deep net architecture for a given problem.
The problem doesn't lend itself very well to local optimization, but might yield to GA-type optimization.
- foxes 8y agoHow are genetic algorithms guaranteed in any way to give you a global optimum? I thought it might just work better when your search space is non smooth.
- siekmanj 8y agoThey're not, as far as I know. In fact, that was one of the big selling points of reinforcement learning - that it tends to reach a better minimum than GA.
- AareyBaba 8y agoAgreed. However, there is no reason to believe that a GA can give a better result than just randomly picking points in search space and doing local optimization around those points. GA's are evo-babble with much hand waving about 'correlation' during mutation or crossover rubbish. The 'curse of dimensionality' in global optimization doesn't go away by using GA's nor can you talk about their effectiveness without characterizing the landscape you are searching (the no free lunch theorem).
- machinelearning 8y agohttps://arxiv.org/pdf/1709.06030.pdf https://arxiv.org/pdf/1709.06030.pdf