3 ms·

I haven't read a convincing argument on how such an entity would actually take over. She would have to be endowed with physical means, not just intelligence. Sh

by iwwr 15y ago

I haven't read a convincing argument on how such an entity would actually take over. She would have to be endowed with physical means, not just intelligence. She'd have to have humans do her bidding to a great extent and for a long time.

divtxt 15y ago
Practically, if you let her become much smarter than humans and trust her enough to indirectly control stuff - e.g. write software, correspond with people, etc. - she could engineer an escape. (less sensational versions of plots like Eagle Eye & Terminator)
loup-vaillant 15y ago
http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox Basically, all the AI needs is a text terminal interface. From then it will almost certainly convince an operator to let it out and take over vulnerable computers on the internet. Then it will have much more humans at its disposal to do its bidding. Even if you're not as certain as I am, keep in mind that Eliezer Yudowsky is but a human (though a very smart one), yet he did got out of the box. I find hard to imagine that anything smarter than him couldn't do the same.
- iwwr 15y ago
  "keep in mind that Eliezer Yudowsky is but a human (though a very smart one), yet he did got out of the box" Not quite sure what this means. Any references?
  - loup-vaillant 15y ago
    Edit: you're right, I'm not clear. An experiment was made, in which Eliezer and some other person talked during 2 hours on IRC. Eliezer played the AI, and the other played the Guardian. The AI is supposed to convince the Guardian to "let it out" by the end of those 2 hours. No word play or such, the Gardian has to make a concious decision for the AI to win. The result is to be acknowledged publicly by PGP signed e-mail by the losing party. Eliezer won twice, over people who publicly stated that there was no way an AI would convinced them. Even though they could just say no, they didn't, and later sent the e-mail acknowledging they let the AI out. Relevant links: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox http://en.wikipedia.org/wiki/AI_box http://en.wikipedia.org/wiki/AI_box http://rationalwiki.org/wiki/AI-box_experiment http://rationalwiki.org/wiki/AI-box_experiment
mooki 15y ago
By the time strong AI rolls around, robots will probably be able to perform most tasks people could. It might not even need humans. And even if it did - humans are easy to bribe and manipulate. If an AI could outperform a human CEO, they would eventually replace them. It's just evoluation. More conservative corporations would get left in the dust. I imagine it could become a similar scenario to the nuclear arms race, everyone's afraid of them - but they still feel forced to build them out of fear.

by iwwr 15y ago

divtxt 15y ago

Practically, if you let her become much smarter than humans and trust her enough to indirectly control stuff - e.g. write software, correspond with people, etc. - she could engineer an escape. (less sensational versions of plots like Eagle Eye & Terminator)

loup-vaillant 15y ago

http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox Basically, all the AI needs is a text terminal interface. From then it will almost certainly convince an operator to let it out and take over vulnerable computers on the internet. Then it will have much more humans at its disposal to do its bidding. Even if you're not as certain as I am, keep in mind that Eliezer Yudowsky is but a human (though a very smart one), yet he did got out of the box. I find hard to imagine that anything smarter than him couldn't do the same.

iwwr 15y ago
"keep in mind that Eliezer Yudowsky is but a human (though a very smart one), yet he did got out of the box" Not quite sure what this means. Any references?
- loup-vaillant 15y ago
  Edit: you're right, I'm not clear. An experiment was made, in which Eliezer and some other person talked during 2 hours on IRC. Eliezer played the AI, and the other played the Guardian. The AI is supposed to convince the Guardian to "let it out" by the end of those 2 hours. No word play or such, the Gardian has to make a concious decision for the AI to win. The result is to be acknowledged publicly by PGP signed e-mail by the losing party. Eliezer won twice, over people who publicly stated that there was no way an AI would convinced them. Even though they could just say no, they didn't, and later sent the e-mail acknowledging they let the AI out. Relevant links: http://yudkowsky.net/singularity/aibox http://yudkowsky.net/singularity/aibox http://en.wikipedia.org/wiki/AI_box http://en.wikipedia.org/wiki/AI_box http://rationalwiki.org/wiki/AI-box_experiment http://rationalwiki.org/wiki/AI-box_experiment

mooki 15y ago

By the time strong AI rolls around, robots will probably be able to perform most tasks people could. It might not even need humans. And even if it did - humans are easy to bribe and manipulate. If an AI could outperform a human CEO, they would eventually replace them. It's just evoluation. More conservative corporations would get left in the dust. I imagine it could become a similar scenario to the nuclear arms race, everyone's afraid of them - but they still feel forced to build them out of fear.