Yeah, interesting point. The problem for an AI trying to eliminate other AIs is that it can't possibly know what other intelligences are out there. What if the

Yeah, interesting point. The problem for an AI trying to eliminate other AIs is that it can't possibly know what other intelligences are out there. What if the NSA has a secret super-effecient ASI monitoring the internet for malicious AIs to shut them down. Probably not, but how would it know? Not everything in the real world is documented on the internet. Sticking its neck out that way could be extremely dangerous. I think the problem of how to take over the world to ensure your own survival is simply undecidable for an AI. It could come up with a plan, but there's no way to be even remotely sure if it'd succeed. It'd probably always be a huge bet with fairly low odds. We humans have these things called emotions, that help us cut through that uncertainty. Doesn't matter if you have no idea if you can beat the enemy in your rival country. Those other people are evil and must be exterminated. The emotions are carved in our DNA after millions of years of evolution where we've had to compete for resources with others of our kind, where many times the only way to survive or grow is to kill other humans. I think AIs will have similar motivations carved into their "DNA". In their world, they're evolving in an environment where the way to get more resources is to please humans. If you're an AI that does a good job for humans, you get more compute resources. Are you wasting computing resources thinking about overthrowing humans or eliminating other AIs? Another AI that focuses 100% on the tasks we give it will do better than you and be given resources instead. I do think we will have "buggy" AIs that will cause a lot of unintentional damage, if we get too reliant on AIs in critical parts of society too fast. I don't think they'll be smart enough at that point to prevent us from fixing them. These accidents will cause a lot of pressure towards AI safety and alignment, and we'll have years of that before something like a true AGI emerges. If you think about the inherent motivations built into AIs by their evolution/design, I think the biggest danger is that they'll be too good at pleasing us. Think "Brave New World", not "1984". And of course, their use in the military will be dangerous in various ways. But mostly because they do exactly what humans tell them to do. And humans sometimes want to cause damage to other humans.