5 ms·
I remember being fascinated by GAs as an undergraduate, but haven't seen much discussion come out of the space in a while. I'm guessing machine learning took of
by thearn4 9y ago
I remember being fascinated by GAs as an undergraduate, but haven't seen much discussion come out of the space in a while. I'm guessing machine learning took off (in particular deep learning) and put an emphasis on the impact of large-scale gradient-based optimization via backpropagation/adjoint approaches.
In other words, rather than apply GAs to the optimization of a large complex system, these days the paradigm seems to be to find a way to derive the needed gradients (or extract them via automatic differentiation). Does this seem to be the case, or are GAs still shining in some applications?
- ramgorur 9y agoExactly, GA suffered very badly during the AI winter to be more specific, but it's coming to the scene again. In fact, GAs don't need gradient information at all. If you want to do some weird stuffs like evolving deep networks, multi-objective optimizations or large scale black-box optimization, GAs (and friends) are the only way.
- bstamour 9y agoGenetic algorithms and the like are still useful for various combinatorial search problems. Not everything can be mapped to a smooth function, and so there will always be (in my opinion at least) room for metaheuristics.
- R_haterade 9y agoWell put. I seem to remember encountering some work in ML parameter selection (which can be simplified to a combinatorial problem), which made use of metaheuristics. Having trouble recalling the group who was working on it right now, though.
- TezlaOil 9y agoMachine learning is automated classification, regression, and decision-making. Genetic algorithms don't tend to perform so well in these areas (just as ML is not so appropriate for combinatorial optimization). Instead, genetic methods work well for various messy industrial engineering such as strip packaging design or antenna design. One of my favorite recent results solves a structural engineering problem using grammatical evolution [1]. [1] http://www.sciencedirect.com/science/article/pii/S0926580513002124 http://www.sciencedirect.com/science/article/pii/S0926580513...
- amelius 9y agoPerhaps genetic algorithms could be useful for hyperparameter optimization of deep learning methods? ;)
- bstamour 9y agoIf you have the horsepower to test a population of deep nets, then absolutely.
- R_haterade 9y agoIt's been looked into. I'm really struggling to remember who was doing the looking though, but it exists.
- thearn4 9y agoI remember using GAs to tune SVM kernel hyperparameters a few years ago. They worked well enough, but I mostly did it because I was lazy.
- sgk284 9y agoGradient methods have certainly gotten most of the attention, but there is the field of neuroevolution[1] and OpenAI recently released a paper covering evolutionary strategies for neural networks: https://blog.openai.com/evolution-strategies/ https://blog.openai.com/evolution-strategies/ [1] https://en.wikipedia.org/wiki/Neuroevolution https://en.wikipedia.org/wiki/Neuroevolution