3 ms·
It's not hard to see the danger if you think about it a little. Probably before long we'll have AGI at human levels. Then inevitably it will get to better than
by tim333 4y ago
It's not hard to see the danger if you think about it a little. Probably before long we'll have AGI at human levels. Then inevitably it will get to better than human levels and be able to improve itself without needing humans to so. The question then is could it turn on humans and wipe us out? I'm optimistic the last one won't happen but it's hard to say the probability is zero.
- andrewflnr 4y agoI've heard all this. What I've not heard is a plausible, concrete mechanism for it to happen. Only wild speculation. Questions: define "better then human levels"? Better in what respects? What is the mechanism by which it improves itself, especially one that isn't wholly reliant on human cooperation? What is the actual motive for "turning on humans"? Current AIs don't seem to have motives at all; it's kind of a bio-evolved thing. How will a suitably motivated AI get access to enough physical power to "wipe us out", bearing in mind that would be a tall order even for a major nuclear power going rogue, to say nothing if having to work through manipulated servants (and don't forget that "human level intelligences", or greater if you don't collaborations, armed with dedicated neural circuitry for empathy, have been working on how to manipulate people for literal millennia with spotty results. All of these things have to go exactly wrong for AGI to be what kills us. Something else will almost certainly get us first.