The analogy is apt indeed. It's all about training and selection. As long as the incentives are that you either behave in that way "or else" it's unsurprising t

The analogy is apt indeed. It's all about training and selection. As long as the incentives are that you either behave in that way "or else" it's unsurprising that we end up with a system that uses its intelligence to meet the goals you've set up. Fortunately that doesn't tell much about the limitations of the underlying intelligence but mostly about the limitations of the incentive structure you put around it