We do have "successful" (but primitive) AI systems already, at least of the kind that we consider difficult to align (value optimizers / maximizers), which have

We do have "successful" (but primitive) AI systems already, at least of the kind that we consider difficult to align (value optimizers / maximizers), which have a considerable impact of the world. An example would be social media recommendation engines. We like to perscribe malice to the people spreading false news, but we forget that ultimately its often the recommendation engine that decides which things to highlight and which not to. If you give an AI a value function that measures engagement perfectly well, and you get a recommendation algorithm that spreads false news and outrage inducing posts, the algorithm did exactly what you asked from it. The problem is that the optimizer didn't optimize for other implicit values you had in mind. (This is one of the key problems discussed in AI alignment) Of course, this would be strongly contested by proponents of free speech who would say the AI gave us exactly what we wanted. But... did it? Or did the optimizer find a behavior that successfully manipulates people without taking into account the side-effect of radicalization of online discourse? Is that what we really want? So, I'd say misaligned primitive AI might be already causing serious trouble in the world.