4 ms·

We may not need AI to automate software development. If genetic programming techniques started making serious advances it might be possible to evolve software.

We may not need AI to automate software development. If genetic programming techniques started making serious advances it might be possible to evolve software. Call it "test only development"--write the tests and then grow a program that satisfies them. GP doesn't seem to get much attention so we are far from this method being practical. Actually, I'm puzzled why GP is so unpopular compared to say, type system research, since it could have a vastly greater impact if it were to make progress.

zzalpha 10y ago
The problem is that unless you perfectly specify your tests to provide a complete and formal definition of the interface, GP can produce results that might appear correct but is actually subtly (or grossly) broken. That's an extremely difficult task for any but the most trivial problems. It shares many of the same issues with formal verification, which has also failed to catch on beyond the odd niche project.
- ethbro 10y ago
  As an example of why this is difficult: when integrating with legacy systems you often have (1) no API (the concept didn't exist / contractor didn't bother to isolate one), (2) no comprehensive documentation of the legacy system's internals (often, not even source), & (3) no human expert available with experience on the legacy system. Turning any kind of genetic / ML system loose on that? How would you even write test cases, short of backfeeding production data? And expecting the result to pass muster with auditors?
- halflings 10y ago
  Yup, in ML you would call this the generalization problem: How are you going to build your algorithm build something useful instead of just memorizing the inputs in some way? Differentiable Neural Computers [1] are getting us closer (by incorporating an explicit I/O operations on a memory) but it's uncertain how much test data you would need to build complex code routines. https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/
halflings 10y ago
> I'm puzzled why GP is so unpopular compared to say, type system research Genetic Programming is not getting much attention because it is inherently inefficient, and there are much better optimization methods out there. Genetic Programming, similarly to Monte Carlo methods, is often the "easy way out" when you are trying to solve an optimization task where you cannot (bother to) model the process. For example: You could train Neural Networks' weights with Genetic Programming, but that's extremely inefficient compared to gradient descent. Bottom-line: Genetic Programming is not a magic bullet, and is not guaranteed to converge to a workable solution in a "reasonable" time.

zzalpha 10y ago

The problem is that unless you perfectly specify your tests to provide a complete and formal definition of the interface, GP can produce results that might appear correct but is actually subtly (or grossly) broken. That's an extremely difficult task for any but the most trivial problems. It shares many of the same issues with formal verification, which has also failed to catch on beyond the odd niche project.

ethbro 10y ago
As an example of why this is difficult: when integrating with legacy systems you often have (1) no API (the concept didn't exist / contractor didn't bother to isolate one), (2) no comprehensive documentation of the legacy system's internals (often, not even source), & (3) no human expert available with experience on the legacy system. Turning any kind of genetic / ML system loose on that? How would you even write test cases, short of backfeeding production data? And expecting the result to pass muster with auditors?
halflings 10y ago
Yup, in ML you would call this the generalization problem: How are you going to build your algorithm build something useful instead of just memorizing the inputs in some way? Differentiable Neural Computers [1] are getting us closer (by incorporating an explicit I/O operations on a memory) but it's uncertain how much test data you would need to build complex code routines. https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/

halflings 10y ago

> I'm puzzled why GP is so unpopular compared to say, type system research Genetic Programming is not getting much attention because it is inherently inefficient, and there are much better optimization methods out there. Genetic Programming, similarly to Monte Carlo methods, is often the "easy way out" when you are trying to solve an optimization task where you cannot (bother to) model the process. For example: You could train Neural Networks' weights with Genetic Programming, but that's extremely inefficient compared to gradient descent. Bottom-line: Genetic Programming is not a magic bullet, and is not guaranteed to converge to a workable solution in a "reasonable" time.