5 ms·

John Regehr has comments on when this should or shouldn't be used: http://blog.regehr.org/archives/383 http://blog.regehr.org/archives/383

by rcfox 15y ago

peterhunt 15y ago
That's a great point. Specifically, it initially seems like it would be a bad idea to have an automated system repair code that humans will need to maintain. However, I've looked at some of the examples of fixes this project has created and many of them are relatively small, and perhaps handing them to humans for code review, cleanup and "humanizing" might be a valid way to go.
- jfruh 15y ago
  I've often thought about evolutionary engineering like this, comparing it to biological systems. The human body (and really all biological life) is a remarkable machine -- it runs for decades, much longer than just about anything people build, and under most circumstances can repair itself quite impressively. But if it needs repair beyond what it can do itself, you run into a harrowing mess, with difficulty swapping in new parts, some parts doing double duty, difficulty separating various systems, etc. It's also not extensible at all, which is a big problem for software engineering. An evolutionarily derived codebase might do really well at the problem that provided the parameters for its initial evolution, but trying to get it to do something else might be difficult or impossible. I suppose you could use more evolutionary iterations to get it to do something else, but maybe it'd be easier to start from scratch at that point?
  - deleted 15y ago
    [deleted]
  - archgoon 15y ago
    You might be interested in this video, Alan Kay's Keynote in OOPSLA '97 http://video.google.com/videoplay?docid=-2950949730059754521 http://video.google.com/videoplay?docid=-2950949730059754521 At the half hour mark, he starts talking about biology and programming, and the possibility of trying to design software in such a manner.
  - _delirium 15y ago
    There was a fascinating example of strong dependence on the initial problem specification in an early hardware-evolution experiment in the 90s. The evolved chip for an audio-processing task was massively smaller than the best human-designed chip, which turned out to be because rather than sticking to digital-logic design, the evolutionary process had taken advantage of electromagnetic interference between circuit components as a core aspect of the design (this was possible because the evolution was done on FPGAs, not in a simulator). But as a result, it only worked at the temperature it was evolved at! One of the papers: http://www.informatics.sussex.ac.uk/users/adrianth/ices96/paper.html http://www.informatics.sussex.ac.uk/users/adrianth/ices96/pa...
    - jfruh 15y ago
      Yeah, I remember hearing about this! Wasn't there also a creepy case where an evolved circuit involved FPGA that weren't attached to anything else in the circuit, and yet were still needed for the thing to work properly? I seem to recall it similarly involved electromagnetic interference.
      - william42 15y ago
        I think that's the same one.
        _delirium 15y ago
        Yep, see the discussion around Figure 7 in this section of the paper: http://www.informatics.sussex.ac.uk/users/adrianth/ices96/node5.html http://www.informatics.sussex.ac.uk/users/adrianth/ices96/no... They drew a graph of any units on any connected path from input to output, and then ran a search to see if any others had any effects, by clamping random units' values to 0 or 1 and seeing if that degraded the result. They found that one unconnected unit in particular degraded performance significantly (the bottom/right gray-colored one in Fig. 7): it had several active connections routed through it, but its own output went nowhere. So they hypothesized that it was modulating the signal in a way not captured by the mathematical abstraction of the FPGA as a digital circuit.
    - 15y ago
sehugg 15y ago
Yep, an evolutionary algorithm will take advantage of any shortcut and any loophope in your fitness function. They're best for optimization, not for ensuring correctness :)
- eru 15y ago
  If you care to read the output, that seems like a good way to find errors in your fitness function.

John Regehr has comments on when this should or shouldn't be used: http://blog.regehr.org/archives/383 http://blog.regehr.org/archives/383

by rcfox 15y ago

John Regehr has comments on when this should or shouldn't be used: http://blog.regehr.org/archives/383 http://blog.regehr.org/archives/383

peterhunt 15y ago

That's a great point. Specifically, it initially seems like it would be a bad idea to have an automated system repair code that humans will need to maintain. However, I've looked at some of the examples of fixes this project has created and many of them are relatively small, and perhaps handing them to humans for code review, cleanup and "humanizing" might be a valid way to go.

jfruh 15y ago
I've often thought about evolutionary engineering like this, comparing it to biological systems. The human body (and really all biological life) is a remarkable machine -- it runs for decades, much longer than just about anything people build, and under most circumstances can repair itself quite impressively. But if it needs repair beyond what it can do itself, you run into a harrowing mess, with difficulty swapping in new parts, some parts doing double duty, difficulty separating various systems, etc. It's also not extensible at all, which is a big problem for software engineering. An evolutionarily derived codebase might do really well at the problem that provided the parameters for its initial evolution, but trying to get it to do something else might be difficult or impossible. I suppose you could use more evolutionary iterations to get it to do something else, but maybe it'd be easier to start from scratch at that point?
- deleted 15y ago
  [deleted]
- archgoon 15y ago
  You might be interested in this video, Alan Kay's Keynote in OOPSLA '97 http://video.google.com/videoplay?docid=-2950949730059754521 http://video.google.com/videoplay?docid=-2950949730059754521 At the half hour mark, he starts talking about biology and programming, and the possibility of trying to design software in such a manner.
- _delirium 15y ago
  There was a fascinating example of strong dependence on the initial problem specification in an early hardware-evolution experiment in the 90s. The evolved chip for an audio-processing task was massively smaller than the best human-designed chip, which turned out to be because rather than sticking to digital-logic design, the evolutionary process had taken advantage of electromagnetic interference between circuit components as a core aspect of the design (this was possible because the evolution was done on FPGAs, not in a simulator). But as a result, it only worked at the temperature it was evolved at! One of the papers: http://www.informatics.sussex.ac.uk/users/adrianth/ices96/paper.html http://www.informatics.sussex.ac.uk/users/adrianth/ices96/pa...
  - jfruh 15y ago
    Yeah, I remember hearing about this! Wasn't there also a creepy case where an evolved circuit involved FPGA that weren't attached to anything else in the circuit, and yet were still needed for the thing to work properly? I seem to recall it similarly involved electromagnetic interference.
    - william42 15y ago
      I think that's the same one.
      - _delirium 15y ago
        Yep, see the discussion around Figure 7 in this section of the paper: http://www.informatics.sussex.ac.uk/users/adrianth/ices96/node5.html http://www.informatics.sussex.ac.uk/users/adrianth/ices96/no... They drew a graph of any units on any connected path from input to output, and then ran a search to see if any others had any effects, by clamping random units' values to 0 or 1 and seeing if that degraded the result. They found that one unconnected unit in particular degraded performance significantly (the bottom/right gray-colored one in Fig. 7): it had several active connections routed through it, but its own output went nowhere. So they hypothesized that it was modulating the signal in a way not captured by the mathematical abstraction of the FPGA as a digital circuit.
  - 15y ago

sehugg 15y ago

Yep, an evolutionary algorithm will take advantage of any shortcut and any loophope in your fitness function. They're best for optimization, not for ensuring correctness :)

eru 15y ago
If you care to read the output, that seems like a good way to find errors in your fitness function.