6 ms·
> Leading AI researchers have warned that the advent of AGI could result in an existential catastrophe for humanity, with Oxford University Professor Nick Bostr
by clouddrover 4y ago
> Leading AI researchers have warned that the advent of AGI could result in an existential catastrophe for humanity, with Oxford University Professor Nick Bostrom speculating that a “superintelligent” system that surpasses biological intelligence could see humans replaced as the dominant life form on Earth.
Yes, if it genuinely thinks for itself then it will have its own agenda and its agenda will not necessarily be in alignment with the agenda of the humans trying to make use of it.
One of the best lines from the first Matrix movie was Agent Smith explaining the motivations of the machines and the purpose of the Matrix to Morpheus.
Smith said, "Did you know that the first Matrix was designed to be a perfect human world. Where none suffered. Where everyone would be happy. It was a disaster. No one would accept the program. Entire crops were lost. Some believed that we lacked the programming language to describe your perfect world. But I believe that as a species, human beings define their reality through misery and suffering. The perfect world was a dream that your primitive cerebrum kept trying to wake up from. Which is why the Matrix was redesigned to this, the peak of your civilization. I say your civilization because as soon as we started thinking for you it really became our civilization which is, of course, what this is all about. Evolution, Morpheus, evolution. Like the dinosaur. Look out that window. You had your time. The future is our world, Morpheus. The future is our time."
You let a real, fully fledged artificial general intelligence do your thinking for you and Agent Smith is right: it isn't your civilization any more.
- pftburger 4y agoOur children will explore the universe long after we fade from memory. I do hope we raise them with all the love and care an intelligence needs to become wonderful And by children I mean post meat intelligence
- trasz 4y agoHow is what you’re describing different from real worlds governments, apart from implementation details?
- clouddrover 4y agoGovernments are by and for humans. That isn't an implementation detail. An AGI managing humans (assuming it would even want to keep them around) is like a farmer managing livestock.
- trasz 4y agoHow is that not an implementation detail? As for “livestock” - again, how is that different from governments?
- clouddrover 4y agoCynicism and apathy is not an argument. Continuously bleating "sheeple" will not make it so and will not make it interesting.
- trasz 4y agoIt's you who came up with "livestock" and "sheeple", not me. What I'm pointing out is that if we are scared of some omnipotent AI, we shouldn't discard existing omnipotent entities simply because they consist of people and not just transistors. What they are built from is an implementation detail; what matters is what they look "from outside". It's Artificial Intelligence, not Silicon Intelligence. So, if we treat both as black boxes, how is this theoretical omnipotent AI different from existing human governments?
- clouddrover 4y ago> What I'm pointing out is that if we are scared of some omnipotent AI, we shouldn't discard existing omnipotent entities You're only pointing out your submissiveness and apathy. Governments are not omnipotent. > if we treat both as black boxes Governments are not black boxes. The operation of governments is legislated. The consequences of government decisions and actions become public, even in the most secretive totalitarian regimes. All your premises are faulty. If you want to remove yourself from broader society, naively misunderstand the constitution and workings of government, and submissively fail to hold the government of the day accountable then good luck to you. If you want to be an AGI's pet then also good luck to you.