5 ms·

tldr: malware evolves into superintelligence. I'm afraid evolution doesn't work like that. Here's a good book (some math): https://www.amazon.com/Evolutionary

tldr: malware evolves into superintelligence. I'm afraid evolution doesn't work like that. Here's a good book (some math): https://www.amazon.com/Evolutionary-Dynamics-Exploring-Equations-Life/dp/0674023382 https://www.amazon.com/Evolutionary-Dynamics-Exploring-Equat... The long and the short of it is that software can't evolve because too much of its genome space is lethal, and the non-lethal parts are non-contiguous, so the quasispecies decoheres -- if it survives at all -- instead of climbing to the top of the fitness landscape.

ridgeguy 10y ago
I'd also suggest a book by Axelrod, The Evolution of Cooperation. [1] Not software explicitly, but patterns of behavior, which is pretty close. [1] https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation
- gopalv 10y ago
  The "Tit for Tat" in the Axelrod experiment winning out is probably the clearest example of a global-winning strategy that's still susceptible to a single-hit psychopath (worse, multiple hits by a sociopath). The core idea is that lots of nice strategies working with each other can beat the ingenious evil ones, is a really positive spin on the usual Prisoner's Dilemma's "defect first" model.
phreeza 10y ago
This is true for classical machine instructions, but what if future software looks more like a huge neural network with a small driver attached?
- pnathan 10y ago
  Modern artificial neural networks aren't designed like that; they are pattern recognizer/generator systems. Further, it is understood that Turing machines and neural networks are computationally equivalent. So it doesn't advance the question to reframe it into a different computational substrate.
pnathan 10y ago
I read this article as effectively a singularitian belief statement. Like the Singularity claims, it doesn't make sense with any existing technology stack from a hardware-code-and-bits perspective: no way to get there from here. It's a belief system or a visionary statement, not an engineering reasoning chain.
whack 10y ago
If I'm understanding you correctly, you're assuming that "software evolution" refers to random mutations in the software's core source code? If so, I might be inclined to agree with you on that narrow point, but software/AI programs can still evolve in other major ways. Assuming an AI that operates on the basis of Neural Networks for example, it could evolve its number of layers, the size of each layer, its sensitivity to new information, and other such parameters. It's not clear if the concept of predator-prey makes any sense in the context of software programs, but invader-defender and virus-host relationships could certainly exist in a advanced software ecosystem, similar to in nature. The premise of the essay seems a little too far-fetched for me to seriously worry about it, but it does seem like a great premise for a sci-fi novel.
- mnemonicsloth 10y ago
  If you insert a random nucleotide into a DNA sequence, the protein you get is usually very similar to the old one. But if you insert a random branch instruction into the object code of a working program, the results are somewhat different. The problem with software evolution is that the adjacent points in program representation space -- whether it's source space or binary , neural network parameters or whatever -- are not viable individuals that can participate in an evolving population. They code for junk -- stillborn offspring. This isn't terribly surprising. The apparatus that powers living systems and makes DNA sequence space so nice took almost a billion years to evolve. Building a such a system in silico, one that is both that flexible and that forgiving, is not likely to happen anytime soon. But you're right. I'd totally read the sci-fi novel.
  - eldondev 10y ago
    > But if you insert a random branch instruction into the object code of a working program, the results are somewhat different. Maybe, maybe not. I've observed lots of dead code in my time developing, where inserting random instructions would have no impact whatsoever. Inserting random instructions into a give program may have negative side effects, it may not. But (without having read your reference, though I might, or the linked article here), most of the source and object code that exists is highly specified and functionally dense. These are the ways we code when we want something very optimized and not broadly adaptive. Essentially you could say that almost all of the code we are writing is rain man style code. Good at some few sets of things in specific conditions. I think it is possible that there will exist (although perhaps not written in the same way) code where less of the genome is lethal, and there is more "wiggle room" for code to expand and change as a system. The stillborn offspring we might encounter if todays systems were "genetically evolved" are the equivalent of disseminated haploid genetic material. I think the conditions for conception (if you'll ride with me on my beaten metaphor) are not so far off as you may think, even if they are very different in terms of compilation/process execution/parallelism. While much of the code we write still mirrors the serial logic we often apply in the sciences, it is neither a foregone conclusion that software will continue exclusively in that way, nor a lack of capabilities of modern infrastructure to achieve parallelism comparable to some biological systems.

ridgeguy 10y ago

I'd also suggest a book by Axelrod, The Evolution of Cooperation. [1] Not software explicitly, but patterns of behavior, which is pretty close. [1] https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation https://en.wikipedia.org/wiki/The_Evolution_of_Cooperation

gopalv 10y ago
The "Tit for Tat" in the Axelrod experiment winning out is probably the clearest example of a global-winning strategy that's still susceptible to a single-hit psychopath (worse, multiple hits by a sociopath). The core idea is that lots of nice strategies working with each other can beat the ingenious evil ones, is a really positive spin on the usual Prisoner's Dilemma's "defect first" model.

phreeza 10y ago

This is true for classical machine instructions, but what if future software looks more like a huge neural network with a small driver attached?

pnathan 10y ago
Modern artificial neural networks aren't designed like that; they are pattern recognizer/generator systems. Further, it is understood that Turing machines and neural networks are computationally equivalent. So it doesn't advance the question to reframe it into a different computational substrate.

pnathan 10y ago

I read this article as effectively a singularitian belief statement. Like the Singularity claims, it doesn't make sense with any existing technology stack from a hardware-code-and-bits perspective: no way to get there from here. It's a belief system or a visionary statement, not an engineering reasoning chain.

whack 10y ago

If I'm understanding you correctly, you're assuming that "software evolution" refers to random mutations in the software's core source code? If so, I might be inclined to agree with you on that narrow point, but software/AI programs can still evolve in other major ways. Assuming an AI that operates on the basis of Neural Networks for example, it could evolve its number of layers, the size of each layer, its sensitivity to new information, and other such parameters. It's not clear if the concept of predator-prey makes any sense in the context of software programs, but invader-defender and virus-host relationships could certainly exist in a advanced software ecosystem, similar to in nature. The premise of the essay seems a little too far-fetched for me to seriously worry about it, but it does seem like a great premise for a sci-fi novel.

mnemonicsloth 10y ago
If you insert a random nucleotide into a DNA sequence, the protein you get is usually very similar to the old one. But if you insert a random branch instruction into the object code of a working program, the results are somewhat different. The problem with software evolution is that the adjacent points in program representation space -- whether it's source space or binary , neural network parameters or whatever -- are not viable individuals that can participate in an evolving population. They code for junk -- stillborn offspring. This isn't terribly surprising. The apparatus that powers living systems and makes DNA sequence space so nice took almost a billion years to evolve. Building a such a system in silico, one that is both that flexible and that forgiving, is not likely to happen anytime soon. But you're right. I'd totally read the sci-fi novel.
- eldondev 10y ago
  > But if you insert a random branch instruction into the object code of a working program, the results are somewhat different. Maybe, maybe not. I've observed lots of dead code in my time developing, where inserting random instructions would have no impact whatsoever. Inserting random instructions into a give program may have negative side effects, it may not. But (without having read your reference, though I might, or the linked article here), most of the source and object code that exists is highly specified and functionally dense. These are the ways we code when we want something very optimized and not broadly adaptive. Essentially you could say that almost all of the code we are writing is rain man style code. Good at some few sets of things in specific conditions. I think it is possible that there will exist (although perhaps not written in the same way) code where less of the genome is lethal, and there is more "wiggle room" for code to expand and change as a system. The stillborn offspring we might encounter if todays systems were "genetically evolved" are the equivalent of disseminated haploid genetic material. I think the conditions for conception (if you'll ride with me on my beaten metaphor) are not so far off as you may think, even if they are very different in terms of compilation/process execution/parallelism. While much of the code we write still mirrors the serial logic we often apply in the sciences, it is neither a foregone conclusion that software will continue exclusively in that way, nor a lack of capabilities of modern infrastructure to achieve parallelism comparable to some biological systems.