5 ms·

AI isn't a clear threat in the SkyNet sense. It's possible that super intelligent machines will go all science fiction and decide to kill all humans, but that's

by hooande 12y ago

AI isn't a clear threat in the SkyNet sense. It's possible that super intelligent machines will go all science fiction and decide to kill all humans, but that's no more likely than any one of hundreds of fictional doomsday scenarios which range from genetically engineered zombie viruses to out of control sharknados. The real threat posed by AI is one that all of us face everyday: bad software design. Is it likely that an AI will achieve sentience and try to take over the planet? Not particularly. Is it likely that an unintended consequence will cause an AI to launch nuclear missiles, release toxic chemicals or shut down the global financial markets? Yes, pretty likely. The benefit of AI of all forms that is that it can make sophisticated decisions in the absence of human instruction. The downside is that without hard coded rules for every possible scenario, we can't ever be sure what it's going to do or how data will be interpreted to make decisions. The world is highly interconnected now. The upside is that our lives are getting more awesome, especially in the developed world. The downside is that it's becoming more and more difficult for any person or group of people to understand exactly how everything fits together. Machine intelligence can help us reach the next levels of progress and hopefully improve the lives of the billions of people who have failed to reap many benefits so far. But we must be careful and ever vigilant, watching both ourselves and the intelligences that we create to make sure that algorithms don't get out of hand. An AI catastrophe is coming, not if but when. The question is how will we respond, and how much potential good will be lost due to an abundance of caution.

baddox 12y ago
Sure, but is it any more likely for an AI to accidentally launch missiles or release toxins than for humans to do so?
- ConceptJunkie 12y ago
  It might hit the "Nuke" button instead of the "Nurse" button...
breuleux 12y ago
> The real threat posed by AI is one that all of us face everyday: bad software design. Is it likely that an unintended consequence will cause an AI to launch nuclear missiles, release toxic chemicals or shut down the global financial markets? Yes, pretty likely. AI is not the kind of thing that can be designed. They tried that route, way back in the sixties, and just failed miserably. Realistically, AI will come from some combination of genetic algorithms, training neural networks, and so on. Now, yes, it could fail, but not in the same way software as we know it fails. No, AI failure would be more similar to human failure. That is still worrying, but no more than hiring the wrong people would be, for example, and you would have better ways to evaluate them.
- PeterisP 12y ago
  AI failure would be a bit different than human failure. If a powerful human is obsessed and 'fails' then at worst he gathers some other people, successfully creates an evil empire and dies after a few decades. Once a powerful AI is obsessed and 'fails', then it can replace as much of the world as it wants with itself, and lives on forever.
  - breuleux 12y ago
    Could it? Let's take a few steps back, here. * You're assuming the AI can copy itself, but this is a dubious assumption. As far as I know, none of the AI algorithms at the forefront of research provide a fraction of the data the AI would need to copy itself. Being able to copy yourself is not a property that comes with running on a computer and I'm quite positive strong AI, when it emerges, will lack this capability to any meaningful extent. Worse yet, data copiability is ultimately a hardware property, and it requires a way to export a snapshot of one's internal state all through the surface. That's not actually efficient design and one has to account for the possibility that AI would run on hardware that makes copies physically impossible. Locality of information minimizes distance, and this is key to efficiency. The only reason our computers architectures work the way they do is that we need them to, but AI in a production setting is not conventional software and does not need to bend to silly copiability requirements. * You're assuming it would have anywhere to copy itself on. If it's running on a billion dollars' worth of hardware, well, it can't just copy itself on user grade computers and expect to gain much out of it. I personally tend to believe that churning out new AI brains from scratch states will yield superior results to copying pre-trained AI or to "exponentially self-improving AI". If nature is to be believed, improvement often requires cycling through clean slates (e.g. birth); software development also suggests that same idea, that sometimes if you want better software you just have to rewrite it. Honestly, it's kind of a rule in general optimization.

by hooande 12y ago

baddox 12y ago

Sure, but is it any more likely for an AI to accidentally launch missiles or release toxins than for humans to do so?

ConceptJunkie 12y ago
It might hit the "Nuke" button instead of the "Nurse" button...

breuleux 12y ago

> The real threat posed by AI is one that all of us face everyday: bad software design. Is it likely that an unintended consequence will cause an AI to launch nuclear missiles, release toxic chemicals or shut down the global financial markets? Yes, pretty likely. AI is not the kind of thing that can be designed. They tried that route, way back in the sixties, and just failed miserably. Realistically, AI will come from some combination of genetic algorithms, training neural networks, and so on. Now, yes, it could fail, but not in the same way software as we know it fails. No, AI failure would be more similar to human failure. That is still worrying, but no more than hiring the wrong people would be, for example, and you would have better ways to evaluate them.

PeterisP 12y ago
AI failure would be a bit different than human failure. If a powerful human is obsessed and 'fails' then at worst he gathers some other people, successfully creates an evil empire and dies after a few decades. Once a powerful AI is obsessed and 'fails', then it can replace as much of the world as it wants with itself, and lives on forever.
- breuleux 12y ago
  Could it? Let's take a few steps back, here. * You're assuming the AI can copy itself, but this is a dubious assumption. As far as I know, none of the AI algorithms at the forefront of research provide a fraction of the data the AI would need to copy itself. Being able to copy yourself is not a property that comes with running on a computer and I'm quite positive strong AI, when it emerges, will lack this capability to any meaningful extent. Worse yet, data copiability is ultimately a hardware property, and it requires a way to export a snapshot of one's internal state all through the surface. That's not actually efficient design and one has to account for the possibility that AI would run on hardware that makes copies physically impossible. Locality of information minimizes distance, and this is key to efficiency. The only reason our computers architectures work the way they do is that we need them to, but AI in a production setting is not conventional software and does not need to bend to silly copiability requirements. * You're assuming it would have anywhere to copy itself on. If it's running on a billion dollars' worth of hardware, well, it can't just copy itself on user grade computers and expect to gain much out of it. I personally tend to believe that churning out new AI brains from scratch states will yield superior results to copying pre-trained AI or to "exponentially self-improving AI". If nature is to be believed, improvement often requires cycling through clean slates (e.g. birth); software development also suggests that same idea, that sometimes if you want better software you just have to rewrite it. Honestly, it's kind of a rule in general optimization.