4 ms·

Replace the term "superintelligence" with "God" in that FAQ. Do you still buy the conclusions? "A God is a mind that is much more intelligent than any human."

Replace the term "superintelligence" with "God" in that FAQ. Do you still buy the conclusions? "A God is a mind that is much more intelligent than any human." "Go programs had gone from “dumber than children” to “God” in eighteen years, and “from never won a professional game” to “Godlike play” in six months." "God has an advantage that a human fighting a pack of lions doesn’t – the entire context of human civilization and technology, there for it to manipulate socially or technologically." "A God might be able to analyze human psychology deeply enough to understand the hopes and fears of everyone it negotiates with." "Humans instinctively balance thousands of different considerations in everything they do; so far this hypothetical God is only balancing three (least cancer, quickest results, highest probability). To a human, it would seem maniacally, even psychopathically, obsessed with cancer curing. If this were truly its goal structure, it would go wrong in almost comical ways." (Oh, god, there's my personal bugbear: "bigger brain size correlates with greater intelligence". You know what else correlates with bigger brain size? Hint: I'm 6'5"; how much smarter am I than someone 5'1"?)

MrScruff 3y ago
Why would replacing 'superintelligence' with 'God' be insightful? Are you suggesting human intelligence is unique and the upper limit for intelligence? I mean we can argue about the difficulty and timeline for achieving human level artificial intelligence, but it seems unlikely that it's impossible.
- mcguire 3y ago
  No, I'm suggesting the entire hype over "superintelligence" is religious in nature. There is little evidence that the current AI systems are anywhere near human intelligence and none that "more than human intelligence" is possible. Further, the abilities attributed to the "superintelligence" are completely godlike---it's omniscient and omnipotent, right? Nothing it can't do, potentially? So, if you're not worried about the extinction risk of God, why should you be concerned with the risk of "superintelligence"?
  - MrScruff 3y ago
    I'm not sure I see the religious aspect. It's more of an inevitable consequence (over an unknown timeline, granted) of the assumption that human intelligence is not some kind of absolute limit, and can be reproduced in a non-biological system. I admit that is still an assumption, but the opposite seems harder to support to me, though we're all admittedly guessing here. Once you make that assumption, then yes of course it follows that the 'beyond human intelligence' would be able to do all of the things that human intelligence is already doing. Namely, science, mathematics, developing AI systems etc.

MrScruff 3y ago

Why would replacing 'superintelligence' with 'God' be insightful? Are you suggesting human intelligence is unique and the upper limit for intelligence? I mean we can argue about the difficulty and timeline for achieving human level artificial intelligence, but it seems unlikely that it's impossible.

mcguire 3y ago
No, I'm suggesting the entire hype over "superintelligence" is religious in nature. There is little evidence that the current AI systems are anywhere near human intelligence and none that "more than human intelligence" is possible. Further, the abilities attributed to the "superintelligence" are completely godlike---it's omniscient and omnipotent, right? Nothing it can't do, potentially? So, if you're not worried about the extinction risk of God, why should you be concerned with the risk of "superintelligence"?
- MrScruff 3y ago
  I'm not sure I see the religious aspect. It's more of an inevitable consequence (over an unknown timeline, granted) of the assumption that human intelligence is not some kind of absolute limit, and can be reproduced in a non-biological system. I admit that is still an assumption, but the opposite seems harder to support to me, though we're all admittedly guessing here. Once you make that assumption, then yes of course it follows that the 'beyond human intelligence' would be able to do all of the things that human intelligence is already doing. Namely, science, mathematics, developing AI systems etc.