I agree that it's hard to take seriously because it sounds like science-fiction. I mean, right now that's what it is. And I'm sure many people who read HN have

I agree that it's hard to take seriously because it sounds like science-fiction. I mean, right now that's what it is. And I'm sure many people who read HN have managed to learn how to catch themselves working on an interesting problem instead of one that needs to be fixed. And this sure sounds interesting. But the problem with AI is that it's unlike anything we've ever faced before and that once it becomes clear that it can actually overshadow environmental and political problems, it'll already be too late. Imagine the Manhattan project people finding out that there's a high probability of a single nuclear bomb actually taking out the entire planet. But that's not all... suppose the entire world already depends on nuclear power, the basic tools to build that bomb already exist everywhere and the first test might be around 1-3 decades in the future. I'm all for AI researchers allocating some of their time to preventing that sort of thing. If that's even possible.