4 ms·
Here's the terrifying thing; it probably wouldn't take a hundred years or our trust for this to happen. This is making some strong assumptions, but the first AI
by Houshalter 13y ago
Here's the terrifying thing; it probably wouldn't take a hundred years or our trust for this to happen. This is making some strong assumptions, but the first AI could probably improve itself. Making better AIs that make better AIs and so on. Very rapidly (possibly only hours, days, weeks) it could become much much smarter than humans. If that happens, than we don't know what's possible, but it could probably do a lot of things we consider impossible. Like hack computer networks trivially and spread itself through the entire internet. Solve the protein folding problem and design working nanotech.
Within a week it could pay/blackmail/manipulate some humans somewhere into developing some crude self-replicating robots or nanotech. Then almost immediately afterwards it consumes the entire Earth in a swarm of rapidly self-replicating nanobots.
...or something. How should I know what a mind literally millions of times more intelligent than me would do. It's like predicting the exact next move a chessmaster will make. I don't know, but I'm confident they'd beat me quickly.
The goal, of course, is to make an AI which won't do this. If the AI decides to terminate the human race, we've already failed making it friendly and it obviously doesn't share our goals and values. But what are our goals and values? I don't know if anyone can answer that. I'm not sure if there is a satisfactory answer.
- User8712 13y agoIt would be interesting to see exactly how intelligent and thought out an AI system could be at destroying us. Would it be barbaric? Would we end up with robots clubbing people and dropping bombs? Would it engineer some type of infectious disease that takes us out? Would it develop a fake shot for immunity, so those people locked away hiding from the disease take the shot, and then all die months later? Does it manipulate us for years by pretending to be millions of different fake users online, and slowly push an agenda that's pro AI and robots, while downvoting everyone against it? Do those people with power, that are against AI start dying of mysterious causes? Even if it decides to be our friend and to help our species, someone will of course fork that AI and give it a negative personality and goals. Then you have the evil AI trying to hack the friendly AI that exists in our homes, and it's a battle of the robots. Of course, whether or not AI is even possible, no one knows. If it is, I think we'll achieve it, and we'll open up a remarkable can of worms.
- loup-vaillant 13y ago> Even if it decides to be our friend and to help our species, someone will of course fork that AI and give it a negative personality and goals. No: the first AI won't let them. See, we're talking a rapidly improving super-intelligence. Whatever is contrary to its goals, it will squash like a bug. A mad scientist forking the code of the AI with a different goal structure is definitely contrary to the goals of that first super-intelligence, and will be shut down before it grows into a sizeable competitor. The result of intelligence explosion is a Singleton: the AI will be a perfectly efficient dictator. It may even shield us from the laws of physics until we graduate to adulthood.