3 ms·

The optimizations are great to apply to an end result, but the Genetic Programming process probably won't benefit from the expressions being optimized 'along th

by mdda 14y ago

The optimizations are great to apply to an end result, but the Genetic Programming process probably won't benefit from the expressions being optimized 'along the way' because some of the 'junk' subexpressions allow for more robustness in the face of the genetic operators. Putting it another way, your highly optimized version (4) is likely to be much more brittle to small 'defects' than (1).

DannyBee 14y ago
The last point can't be overstated. Optimization is highly likely to overfit the training data. For example, given training data 000001 000002 000003 000004 The "optimized" version would be \d+[1-4] If the goal was simply to find the smallest regex that matches all the training data, that's trivial to code, NP-hard to execute. But the goal isn't to find the smallest regexp, it's to find a "good" regexp.
- khafra 14y ago
  Actually, the last point can be overstated, because the optimizations he applied created a functionally equivalent state machine to the non-optimized version. If it overfitted, the original GA-generated algorithm overfitted as well.
- tlrobinson 14y ago
  My "optimizations" were trivial, in that they don't change the behavior of the regex at all, they just simplify it. As khafra points out, they are equivalent. I also wasn't trying to say those simplifications should be applied between iterations of the algorithm, only at the end.
  - mdda 14y ago
    Understood - and I was just making the observation that the expressions produced during the process were junky in a way that's characteristic of the way they are evolving : Mixing together two honed expressions would produce much more extreme deviations in 'children' than the offspring of two of these hairy expressions. Not only are the expressions tending to get more accurate with each generation, but their degree of 'evolvability' is also being selected for.

by mdda 14y ago

DannyBee 14y ago

The last point can't be overstated. Optimization is highly likely to overfit the training data. For example, given training data 000001 000002 000003 000004 The "optimized" version would be \d+[1-4] If the goal was simply to find the smallest regex that matches all the training data, that's trivial to code, NP-hard to execute. But the goal isn't to find the smallest regexp, it's to find a "good" regexp.

khafra 14y ago
Actually, the last point can be overstated, because the optimizations he applied created a functionally equivalent state machine to the non-optimized version. If it overfitted, the original GA-generated algorithm overfitted as well.
tlrobinson 14y ago
My "optimizations" were trivial, in that they don't change the behavior of the regex at all, they just simplify it. As khafra points out, they are equivalent. I also wasn't trying to say those simplifications should be applied between iterations of the algorithm, only at the end.
- mdda 14y ago
  Understood - and I was just making the observation that the expressions produced during the process were junky in a way that's characteristic of the way they are evolving : Mixing together two honed expressions would produce much more extreme deviations in 'children' than the offspring of two of these hairy expressions. Not only are the expressions tending to get more accurate with each generation, but their degree of 'evolvability' is also being selected for.