3 ms·
>Wouldn't a drastically more advanced intelligence just dwarf and dissolve any lesser intelligence that tried to meld with it? It would be complete ego death.
by KVFinn 13y ago
>Wouldn't a drastically more advanced intelligence just dwarf and dissolve any lesser intelligence that tried to meld with it? It would be complete ego death.
A lot of the singularity people think that superhuman AI in inevitable whether they work on it or not, but if the first AI isn't a friendly one (as opposed to a paperclip-optimizer that happily turns the planet in paperclips) then you don't get a second chance.
- f_salmon 13y agoAlso called "a self-fulfilling prophecy"; you make something happen by the simple statement "it will happen". It's also a smart move for your career: You can create your own job and attract lots of funding for your research, or sell your business for a lot.
- eli_gottlieb 13y agoIt sounds like a piece of fearmongering nastiness until you think of it in terms of mathematics rather than psychology. A so-called "artificial intelligence" is just an extremely sophisticated active, online learning agent designed to maximize some utility function or (equivalently) minimize some loss function. There's no term in a loss function for "kill all humans", but neither is there one for "do what humans want", or better yet, "do what humans would want if they weren't such complete morons half the time".
- TeMPOraL 13y agoOr as Eliezer nicely phrased it once in a paper[0], > "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else." [0] - http://yudkowsky.net/singularity/ai-risk http://yudkowsky.net/singularity/ai-risk
- esrauch 13y agoThis objection implies there is something better than turning the planet into paperclips.