There are two, mostly disjoint groups warning about AI. There are the people worried about comparatively mundane effects from comparatively mundane systems: job

There are two, mostly disjoint groups warning about AI. There are the people worried about comparatively mundane effects from comparatively mundane systems: job loss, spam, disinformation, maybe an occasional unfair loan-application rejection. These concerns don't have nothing going for them, but in all but the worst-case-scenario versions, these just aren't bad enough to make AI not be worth it. Then there's the people looking ahead, foreseeing a future where superintelligent AIs are more powerful than humanity, and worried that most possible variations of those superintelligences are incentivized to destroy us. I think this open letter puts much too much emphasis on the petty stuff, I think because they're trying to appeal to people who are allergic to anything that requires extrapolating more than a little bit into the future. But buying more time for alignment research, before we tackle superintelligence, does meaningfully improve humanity's odds of survival, so I hope this happens anyways.