probably because its "utility function" (if it has one to begin with) is not aligned with human utility. a major part of MIRI's research program is set out on t

probably because its "utility function" (if it has one to begin with) is not aligned with human utility. a major part of MIRI's research program is set out on the question of this "alignment problem". Yudkowsky has an amazing talk on this issue. I don't recommend many talks but this one is very interesting and introductory: https://www.youtube.com/watch?v=EUjc1WuyPT8 https://www.youtube.com/watch?v=EUjc1WuyPT8