I recommend Robert Miles's YouTube channel, and his "Intro to AI Safety, Remastered" is a good place to start: https://www.youtube.com/watch?v=pYXy-A4siMw https

I recommend Robert Miles's YouTube channel, and his "Intro to AI Safety, Remastered" is a good place to start: https://www.youtube.com/watch?v=pYXy-A4siMw https://www.youtube.com/watch?v=pYXy-A4siMw I find Robert Miles worryingly plausible when he says (about 12:40 into the video) "if you have a sufficiently powerful agent and you manage to come up with a really good objective function, which covers the top 20 things that humans value, the 21st thing that humans value is probably gone forever"