>That said, there are bad scenarios where we just get so dependent on AI that we voluntarily cede control to it, little by little, and then things go wrong once

>That said, there are bad scenarios where we just get so dependent on AI that we voluntarily cede control to it, little by little, and then things go wrong once it's too late. You have to get rid of the idea that an AI will start doing something that we don't explicitly tell it to. Sure, in the future it will be possible to accomplish same tasks with AI that would take multiple smart humans to do. There are still limits in place though. >And you don't have to fully simulate reality to be really, really good at achieving your goals. We don't fully simulate every particle on a spacecraft, but we get them where they need to go, because we have simplifying models Yes, because we have found patters in non chaotic systems. Not all systems are like that. Why can't we predict the stock market? Stephen Wolfram talks about this, its a phenomenon called computational irreducibility, meaning that some processes cannot be simply just estimated with a mathematical formula, and furthermore, these processes can arise from very simple rules. The point that Im trying to make is that your warnings are absolutely empty. Its equivalent to those people worried that LHC was going to punch a hole in our universe. There is zero information right now to base any worry of an advanced AI.