11 ms·

"Our AI is nowhere near capable enough to create more AI." Are you sure? Lots of machine-learning/optimization processes already work much like evolution. How

by gojomo 12y ago

"Our AI is nowhere near capable enough to create more AI." Are you sure? Lots of machine-learning/optimization processes already work much like evolution. How many people thought an email worm could bring down thousands of machines around the world in 1988? http://en.wikipedia.org/wiki/Morris_worm http://en.wikipedia.org/wiki/Morris_worm If one (or several) of the robots depicted in the below video wants someone dead, the "stair step" defense is not going to be enough: https://www.youtube.com/watch?v=QVdQM47Av20 https://www.youtube.com/watch?v=QVdQM47Av20

chubot 12y ago
Well, I don't want to dismiss the possibility too quickly, since there appear to be researchers taking it seriously: http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior... (section 3.4, Control, although the general skepticism is noted) But I will say that there is a difference between feedback and unconstrained evolution. Machine learning mechanisms are feedback loops. But that doesn't mean they will break out of their own paradigm without human intervention. Malware is actually a good example. The Morris worm is a static piece of program text, invented by a human. It took advantage of a homogeneous and static environment to spread widely (lots of computers running the same program with the same vulnerability.) There's a huge difference between spreading in this environment and adapting / reproducing so as to spread in novel environments. Think about Stuxnet. AFAICT, this was a completely novel solution to a problem that had never been seen before: jumping an air gap and destroying a piece of hardware via software. What do you think is more fruitful approach to designing something like Stuxnet? 1) Assemble the best and brightest minds from multiple domains, painstakingly build simulation environments based on the most plausible intelligence, test custom malware laboriously in those environments, etc. 2) Start with some existing malware (not unlike the Morris worm), and design a meta-algorithm to evolve it into something capable of destroying a nuclear plant from afar? #2 is science fiction, as far as I can tell. The level of ingenuity required for a task like this is just qualitatively outside the domain of computers. This is closer to what would be required for us to lose control: for AI to be designing AI, rather than humans designing AI. And keep in mind that AI designing AI is harder than AI designing malware. I have seen the demos from Deep Mind where it learns to play video games without any knowledge of the rules. I don't claim to fully understand these techniques. (I have however worked in a research lab on robotics, and seen the vast gulf between what the media portrays and reality.) But I would be interested if anyone knowledgeable in those techniques sees a realistic path from deep learning to something like #2.

by gojomo 12y ago

chubot 12y ago

Well, I don't want to dismiss the possibility too quickly, since there appear to be researchers taking it seriously: http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior... (section 3.4, Control, although the general skepticism is noted) But I will say that there is a difference between feedback and unconstrained evolution. Machine learning mechanisms are feedback loops. But that doesn't mean they will break out of their own paradigm without human intervention. Malware is actually a good example. The Morris worm is a static piece of program text, invented by a human. It took advantage of a homogeneous and static environment to spread widely (lots of computers running the same program with the same vulnerability.) There's a huge difference between spreading in this environment and adapting / reproducing so as to spread in novel environments. Think about Stuxnet. AFAICT, this was a completely novel solution to a problem that had never been seen before: jumping an air gap and destroying a piece of hardware via software. What do you think is more fruitful approach to designing something like Stuxnet? 1) Assemble the best and brightest minds from multiple domains, painstakingly build simulation environments based on the most plausible intelligence, test custom malware laboriously in those environments, etc. 2) Start with some existing malware (not unlike the Morris worm), and design a meta-algorithm to evolve it into something capable of destroying a nuclear plant from afar? #2 is science fiction, as far as I can tell. The level of ingenuity required for a task like this is just qualitatively outside the domain of computers. This is closer to what would be required for us to lose control: for AI to be designing AI, rather than humans designing AI. And keep in mind that AI designing AI is harder than AI designing malware. I have seen the demos from Deep Mind where it learns to play video games without any knowledge of the rules. I don't claim to fully understand these techniques. (I have however worked in a research lab on robotics, and seen the vast gulf between what the media portrays and reality.) But I would be interested if anyone knowledgeable in those techniques sees a realistic path from deep learning to something like #2.