It's not just you. I can understand it, in a sense -- humans aren't really well equipped emotionally to think about this sort of risk, even if they're working

It's not just you. I can understand it, in a sense -- humans aren't really well equipped emotionally to think about this sort of risk, even if they're working on it. We're really good at rationalizing things. You might think, "AGI is going to be good, that's why I want to keep working on it", when your actual chain of reasoning is more like "Hmm, I'm not a bad person, and neither are my friends. They all want to work on AGI, so it can't be the case that AGI is going to turn out badly, because that would make us all bad people, wouldn't it? So AGI must be fine." Or you might just not think about it much at all, you just keep showing up because it's really interesting work and you like your co-workers. If you think about it, that would make you uncomfortable, so you avoid thinking about it. I apologize if I've offended anyone with this. I'm saying this because I have experience with this sort of rationalization myself and things I've done in the past, and I don't think I'm less self-aware than the average person. If you work on this stuff, please take some time to think through and grapple with the way the future looks. Please question your assumptions. What sort of future are you imagining? Why do you think that machine super-intelligence will want to help humans? What sorts of failure scenarios are there, and how will we prevent them? How will the world look when there are a multitude of intelligences that are far beyond us?