3 ms·

I've been thinking about what I visualize as "plant" intelligence, in response to Elon Musk's statements about killer AI. It seems like Andrew Ng is saying the

by chubot 12y ago

I've been thinking about what I visualize as "plant" intelligence, in response to Elon Musk's statements about killer AI. It seems like Andrew Ng is saying the same thing. The metaphor is that you could have "sedentary" superintelligences that solve a relatively narrow problem. They take in huge amounts of data, and spit out ingenious answers to well-defined problems. Things like traffic-routing, energy allocation, or photo/video recognition, etc. They are tools for humans. By what mechanism do these superintelligences involve into organisms motivated to extinguish humans? Reproduction, which leads to independent evolution. That process is inherently unpredictable. But right now we're firmly in the stage of humans creating AI. Our AI is nowhere near capable enough to create more AI. And I would argue that there is no possibility of this happening by "accident"; you would have to specifically engineer AI to able to create more AI. I agree with Ng in that it's more of a distraction now than anything else. The jobs issue is a lot more pertinent. It reminds me of this book I got at Google 8 or 9 years ago. This guy asked a robotic professors how they would avoid being killed by a robot. And the professor said: "Climb up one step" (on a staircase).

gojomo 12y ago
"Our AI is nowhere near capable enough to create more AI." Are you sure? Lots of machine-learning/optimization processes already work much like evolution. How many people thought an email worm could bring down thousands of machines around the world in 1988? http://en.wikipedia.org/wiki/Morris_worm http://en.wikipedia.org/wiki/Morris_worm If one (or several) of the robots depicted in the below video wants someone dead, the "stair step" defense is not going to be enough: https://www.youtube.com/watch?v=QVdQM47Av20 https://www.youtube.com/watch?v=QVdQM47Av20
- chubot 12y ago
  Well, I don't want to dismiss the possibility too quickly, since there appear to be researchers taking it seriously: http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior... (section 3.4, Control, although the general skepticism is noted) But I will say that there is a difference between feedback and unconstrained evolution. Machine learning mechanisms are feedback loops. But that doesn't mean they will break out of their own paradigm without human intervention. Malware is actually a good example. The Morris worm is a static piece of program text, invented by a human. It took advantage of a homogeneous and static environment to spread widely (lots of computers running the same program with the same vulnerability.) There's a huge difference between spreading in this environment and adapting / reproducing so as to spread in novel environments. Think about Stuxnet. AFAICT, this was a completely novel solution to a problem that had never been seen before: jumping an air gap and destroying a piece of hardware via software. What do you think is more fruitful approach to designing something like Stuxnet? 1) Assemble the best and brightest minds from multiple domains, painstakingly build simulation environments based on the most plausible intelligence, test custom malware laboriously in those environments, etc. 2) Start with some existing malware (not unlike the Morris worm), and design a meta-algorithm to evolve it into something capable of destroying a nuclear plant from afar? #2 is science fiction, as far as I can tell. The level of ingenuity required for a task like this is just qualitatively outside the domain of computers. This is closer to what would be required for us to lose control: for AI to be designing AI, rather than humans designing AI. And keep in mind that AI designing AI is harder than AI designing malware. I have seen the demos from Deep Mind where it learns to play video games without any knowledge of the rules. I don't claim to fully understand these techniques. (I have however worked in a research lab on robotics, and seen the vast gulf between what the media portrays and reality.) But I would be interested if anyone knowledgeable in those techniques sees a realistic path from deep learning to something like #2.

by chubot 12y ago

gojomo 12y ago

"Our AI is nowhere near capable enough to create more AI." Are you sure? Lots of machine-learning/optimization processes already work much like evolution. How many people thought an email worm could bring down thousands of machines around the world in 1988? http://en.wikipedia.org/wiki/Morris_worm http://en.wikipedia.org/wiki/Morris_worm If one (or several) of the robots depicted in the below video wants someone dead, the "stair step" defense is not going to be enough: https://www.youtube.com/watch?v=QVdQM47Av20 https://www.youtube.com/watch?v=QVdQM47Av20

chubot 12y ago
Well, I don't want to dismiss the possibility too quickly, since there appear to be researchers taking it seriously: http://futureoflife.org/static/data/documents/research_priorities.pdf http://futureoflife.org/static/data/documents/research_prior... (section 3.4, Control, although the general skepticism is noted) But I will say that there is a difference between feedback and unconstrained evolution. Machine learning mechanisms are feedback loops. But that doesn't mean they will break out of their own paradigm without human intervention. Malware is actually a good example. The Morris worm is a static piece of program text, invented by a human. It took advantage of a homogeneous and static environment to spread widely (lots of computers running the same program with the same vulnerability.) There's a huge difference between spreading in this environment and adapting / reproducing so as to spread in novel environments. Think about Stuxnet. AFAICT, this was a completely novel solution to a problem that had never been seen before: jumping an air gap and destroying a piece of hardware via software. What do you think is more fruitful approach to designing something like Stuxnet? 1) Assemble the best and brightest minds from multiple domains, painstakingly build simulation environments based on the most plausible intelligence, test custom malware laboriously in those environments, etc. 2) Start with some existing malware (not unlike the Morris worm), and design a meta-algorithm to evolve it into something capable of destroying a nuclear plant from afar? #2 is science fiction, as far as I can tell. The level of ingenuity required for a task like this is just qualitatively outside the domain of computers. This is closer to what would be required for us to lose control: for AI to be designing AI, rather than humans designing AI. And keep in mind that AI designing AI is harder than AI designing malware. I have seen the demos from Deep Mind where it learns to play video games without any knowledge of the rules. I don't claim to fully understand these techniques. (I have however worked in a research lab on robotics, and seen the vast gulf between what the media portrays and reality.) But I would be interested if anyone knowledgeable in those techniques sees a realistic path from deep learning to something like #2.