> But in the end, it would just be a translation app. It wouldn’t want to take over the world. It wouldn’t even “want” to become better at translating than it

> But in the end, it would just be a translation app. It wouldn’t want to take over the world. It wouldn’t even “want” to become better at translating than it was already. It would just translate stuff really well. > It could have media services that can write books or generate movies to fit your personal tastes. I'm not an expert in this topic, but wouldn't you say that the ability to create compelling narratives (even more so than 'mechanical' translation between languages), pretty much relies on your ability to empathise at some level with people who want to take over the world or the people who want to stop them? How would AI come up with the plot for one of the most profitable franchises in recent history - "Avengers: Infinity War" movies? It would have to be programmed with an understanding for Thanos' perspective, and the fundamental will of most everyone around him to be terrified of that and disagree - and even then it's not a good movie franchise without the romance and familial dynamics between so many characters. If an AI can already understand all that (and know that it has to understand all that) - well you've created a pretty smart human already - and the OP's argument about differentiating these powers doesn't seem to hold...