3 ms·

GAs are in this weird place where much of their lunch (by which I mean tasks such as classification and regression), has been eaten by more principled methods f

GAs are in this weird place where much of their lunch (by which I mean tasks such as classification and regression), has been eaten by more principled methods from Machine Learning. They've also been pushed out of most combinational optimisation problems by Constraint Programming methods and by other more principled meta-heuristics such as Large Neighbourhood Search. In my opinion part of the problem is that so much of the literature around GAs involves what I like to think of as voodoo optimisation. You take a simple to implement randomised algorithm, apply it to some poorly studied but high-dimensional problem and then poke things as appropriate until you eventually find some feasible solution. Maybe. If I'm being honest, the only time I'd consider GAs is if I had very little insight into the problem at hand and even then only after I've run out of other things to try.

radarsat1 9y ago
I think there are lots of applications for gradient-free methods, but it's never been clear to me what the advantage is of the "genetic" aspect as compared to other randomized search strategies. However, the ability to dynamically grow a program tree is interesting and somewhat difficult to model in an ANN-style framework. But in gradient free methods, there are lots of interesting ideas like particle swarm optimisation and simulated annealing, or even just random search. It's not really clear to me what the "gene swapping" approach brings to the table, except that it probably encourages exploring the area in the middle between some good minima that are potentially far apart in the search space. But you could formulate this same idea without appealing to evolutionary metaphors.
- TezlaOil 9y ago
  Gene swapping makes perfect sense when the success of a given solution is due to the various contributions of the solution's components (this is closely related to the Credit Assignment Problem). For example, a 'random' tour of the traveling salesman might have a nice sensible path between Paris and Moscow, while another solution has a good path between San Francisco and Mexico City. Chromosomal crossover (the fundamental operation of genetic algorithms) can create "offspring" tours that contain both of these components. This is not at all analogous to exploring the middle between two local optima - the solution space often has no meaningful geometric structure, so being located "between" two other solutions may not make sense at all.

radarsat1 9y ago

I think there are lots of applications for gradient-free methods, but it's never been clear to me what the advantage is of the "genetic" aspect as compared to other randomized search strategies. However, the ability to dynamically grow a program tree is interesting and somewhat difficult to model in an ANN-style framework. But in gradient free methods, there are lots of interesting ideas like particle swarm optimisation and simulated annealing, or even just random search. It's not really clear to me what the "gene swapping" approach brings to the table, except that it probably encourages exploring the area in the middle between some good minima that are potentially far apart in the search space. But you could formulate this same idea without appealing to evolutionary metaphors.

TezlaOil 9y ago
Gene swapping makes perfect sense when the success of a given solution is due to the various contributions of the solution's components (this is closely related to the Credit Assignment Problem). For example, a 'random' tour of the traveling salesman might have a nice sensible path between Paris and Moscow, while another solution has a good path between San Francisco and Mexico City. Chromosomal crossover (the fundamental operation of genetic algorithms) can create "offspring" tours that contain both of these components. This is not at all analogous to exploring the middle between two local optima - the solution space often has no meaningful geometric structure, so being located "between" two other solutions may not make sense at all.