3 ms·
GP is a gigantic field, comprising a huge number of approaches to many different problems, while the paper you cite only offers a brief critique of a small subs
by pmoriarty 7y ago
GP is a gigantic field, comprising a huge number of approaches to many different problems, while the paper you cite only offers a brief critique of a small subset of GP as applied only to symbolic regression. So it can not fairly be taken as a critique of GP as a whole.
Even when limited to symbolic regression alone, the objections they cite: 1 - that "state-of-the-art GP implementations often fail to return the original formula from which the input data was generated", and 2 - "the results returned are inconsistent and difficult to reproduce. A user who is not an expert in GP will not likely trust an algorithm which cannot reliably reproduce the same results with each invocation..." are not irrefutable arguments.
On point 1, not everyone is interested in "returning the original formula from which the input data was generated". For one, for many problems the original data was not generated with any formula, and any solution would suffice as long as it meets whatever requirements the user may have. Take the evolving of radio antennas with GP -- something that was done to a human-competitive level decades ago with GP. There the user wants a working antenna with certain properties, they are not interested in "returning the original formula from which the input data was generated". Or maybe the users are interested in evolving a team of soccer-playing robots (which was also done with GP). Once again, they're not interested in "returing the original formula from which the input data was generated". So for these users this critique would be irrelevant.
On point 2, yes, there is a random number generated involved with GP, and if the user is interested in evolving the exact same solution on a different run of the GP without reusing the same PRNG seed, they'd have a problem. But I'm not sure why they'd want to do that instead of just using the best solution(s) on new data. If they just used the best solutions, then they should get the same result every time when the solutions are used on the same data (unless the solutions themselves involved randomness, which is not usually the case, and would be independent of GP anyway).
- verdverm 7y agoThe main point of the paper is that you should account for the structure of the solution space, and take advantage of that. For SR, one example is a+b = b+a. For physical objects, like antenna, that would be symmetries along axes. PGE can be generalized to any problem which can represent their solution space as a trie representation and a grammar for local modifications or movements in the search space. With regard to PRNG, GP can only produce statistically significant results if run a minimum number of times, because it is a probabilistic algorithm. My stats professor said that was at least 29 times. If you fix the random seed, you lose the power of the GP algo. It's hard to trust results from GP research as the vast majority is not reproducible. There is one guy who publishes the world best results to Springer anytime someone out performs his results. Totally bogus work. The GP field needs to adopt the publishing and open code / data that we find in the DL/RL community.
- nestorD 7y agoGP can take the structure of the space into account (you just need to take it into account into your mutation and crossover functions) and algorithms are typically compared on enough runs to provide statistically significant results (29 runs might be far from enough if you have high variance...) but you don't need reproducibility when solving an optimisation problem in the real world, you just want to get a good result in a reasonable time. The fact that you can use GP to optimize about anything (both parameters and structures) makes it a good first approach (where it fails is that it tends to be slow and have way too much parameters for a casual user). (I did think that your paper is interesting in that it invites us to think about non-GP based approach to symbolic regression)
- randomsearch 7y agoMuch of the GP community already open sources code and data. I wouldn’t say the ML community is better.