2 ms·
This article is not even wrong. Sentience, whatever that is, is not a requirement for superintelligence. Merely the ability to outsmart humans at particular tas
by lern_too_spel 4y ago
This article is not even wrong. Sentience, whatever that is, is not a requirement for superintelligence. Merely the ability to outsmart humans at particular tasks and have goals that are not aligned with our own is enough to be a potentially unstoppable danger. The author proposes that aligning the goals of a superintelligence with his will be as simple as setting a thermostat, not realizing that we already have an existence proof of superintelligences causing mass destruction.
Businesses put thousands of brains to work maximizing profit and causing climate change (leading to water security issues, mass migration, wars, and starvation), Bhopal disasters, and oil spills in Nigeria and the Gulf of Mexico, with legal teams making sure the profits are protected. Governments put millions of brains to work supporting a despot and leading to genocide, or trolling the Internet causing people to distrust a mostly scandal-free government and vote for a con-man (imagine a fully automated troll army). A single person is no match intellectually for these organizations, and their only hope is to band together to form other superintelligences to combat these.