3 ms·

I think there are lots of applications for gradient-free methods, but it's never been clear to me what the advantage is of the "genetic" aspect as compared to o

I think there are lots of applications for gradient-free methods, but it's never been clear to me what the advantage is of the "genetic" aspect as compared to other randomized search strategies. However, the ability to dynamically grow a program tree is interesting and somewhat difficult to model in an ANN-style framework. But in gradient free methods, there are lots of interesting ideas like particle swarm optimisation and simulated annealing, or even just random search. It's not really clear to me what the "gene swapping" approach brings to the table, except that it probably encourages exploring the area in the middle between some good minima that are potentially far apart in the search space. But you could formulate this same idea without appealing to evolutionary metaphors.

TezlaOil 9y ago
Gene swapping makes perfect sense when the success of a given solution is due to the various contributions of the solution's components (this is closely related to the Credit Assignment Problem). For example, a 'random' tour of the traveling salesman might have a nice sensible path between Paris and Moscow, while another solution has a good path between San Francisco and Mexico City. Chromosomal crossover (the fundamental operation of genetic algorithms) can create "offspring" tours that contain both of these components. This is not at all analogous to exploring the middle between two local optima - the solution space often has no meaningful geometric structure, so being located "between" two other solutions may not make sense at all.