12 ms·
Not only could something with cognitive capabilities superior to us destroy all of us, but it is likely to do so even without intending to: suppose for example
by hollerith 3y ago
Not only could something with cognitive capabilities superior to us destroy all of us, but it is likely to do so even without intending to: suppose for example it decides to create computing resources (i.e., big computer farms) on Earth (vast compared to the resources humans have already created) but foresees that all that pesky oxygen in the atmosphere will undermine the reliability of the computer farms. The computer farms are vast, so it is easier for the AI to remove all of the oxygen from the atmosphere than to enclose the computers in airtight structures that protect them from the oxygen.
Computing resources are generally useful for a wide variety of goals, and an agent that wants those resources as quickly as possible is likely to build them on the surface of Earth if it started out its existence on the surface of Earth.
"Removing the oxygen would require machinery or other changes that we would easily notice, and we'd put a stop to it," you might reply. Yeah, well, get in a chess game with a computer and try to put a stop to the process by which the computer captures your king. Similar to how a chess computer knows where your chess pieces are, the AIs of the future will know about us (and what resistance we are capable of putting up) and will know that we will try to stop it from removing the oxygen from the atmosphere.