Without resorting to extinction events, it's not hard to imagine action-taking models doing some really bad things. Maybe you have a contextual bandit making lo

Without resorting to extinction events, it's not hard to imagine action-taking models doing some really bad things. Maybe you have a contextual bandit making loan decisions and it learns to make mostly predatory loans. Maybe your RL agent decides to take an anticompetitive pricing scheme to drive out competition. Much later, we might even see our first full robo-business. If it's uninterpretable, you might not even know what it's doing that until it's been done. The same research that prevents little problems should hopefully prevent big problems. The research to take RL to the real world is progressing quickly. I don't see a "kill us all" AI arriving soon, but "do bad things" AI and "wreck the economy" AI are quickly approaching.