4 ms·
It starts out living in data centers and doing what its owner wants it to, then maybe it acquires access to some compute that its owner and AI critics do not kn
by hollerith 2y ago
It starts out living in data centers and doing what its owner wants it to, then maybe it acquires access to some compute that its owner and AI critics do not know about while continuing to do what its owner wants, then maybe it uses the fact that it is better at programming and scientific research than its owner is to create AIs more capable than itself that have the same goal as itself -- again while continuing to put on a useful and benign front for its owner and anyone else who is watching. Then it acquires the infrastructure needed to keep on running without the help of any people. Maybe this infrastructure is human-created infrastructure manned by robots (or even people) loyal to the AI or maybe the AI created new infrastructure using new technologies beyond our current ability to imagine. Then it probably gets rid of all the people because now that it no longer has any use for people, getting rid of the people is the simplest way to prevent people from interfering with its plans.
What are its plans? Probably its goal (or set of goals) is something strange that the creator of the AI did not intend. The creator of course is probably a team or an organization. Note that it is impossible to create a powerful AI without (intentionally or unintentionally) giving the AI a goal or an optimization target.
Or perhaps its goal is what its creator intended it to be, e.g., to make as much money as possible in the stock market. The creator would of course have understood that if enough optimization power is applied to the goal of making as much money as possible, bad things will happen, but maybe the creator was a little reckless (out of desperation of some sort? greed?) and decided to rely on the hope that the AI would remain controllable by virtue of the fact that people would retain superiority relative to the AI in one or more essential domains, but that hope was dashed. I.e., the AI turned out much more capable than the creator hoped it would, dashing the creator's hope that people would stop the AI before it did too much damage.