> That entity need not be a single AI. It could be a system like a large corporation This is not a new problem, then. Let's tackle the issue of corporations,

> That entity need not be a single AI. It could be a system like a large corporation This is not a new problem, then. Let's tackle the issue of corporations, rather than chase an AI boogeyman that doesn't fundamentally change anything. Look at oil companies, for example. They have humans in the loop at every level, and yet those humans do little to prevent profit incentives from leading them to destroy the planet. A broken reward function is a broken reward function, AI-assisted or not.