I don't know whether I have "like-minded views", but I think the risk is there, and it is not fantastical to me because the AGI doesn't need to "do" anything ou

I don't know whether I have "like-minded views", but I think the risk is there, and it is not fantastical to me because the AGI doesn't need to "do" anything outside of humans assisting it -- no killer robots or nanobots. It just needs to persuade humans to kill for it, or gather resources for it, and so on. And because it's an AGI, it has superhuman abilities for persuasion. The intuition for GPT-4 being on the way towards superhuman intelligence or persuasion abilities is that the only significant difference between it and GPT-3 appears to be the amount of compute power applied to training. The previous US President was much less than superhuman, and he still managed to seriously threaten democracy in the US, and by extension cause a minor existential risk to civilization. Imagine what a more intelligent agent with enough attention to be personally persuasive instead of generically persuasive could achieve for itself, given a goal. (If you think that LLMs can't express goals or preferences, read some of the Bing Sydney transcripts.)