I could be misrepresenting Maciej here (idlewords, please correct me if so), but my understanding of that talk was that "superintelligence is not worth worrying

I could be misrepresenting Maciej here (idlewords, please correct me if so), but my understanding of that talk was that "superintelligence is not worth worrying about" was only part of it. A lot of it was more about the deleterious knock-on effects that seem endemic to AI worriers, developers, SV bigwigs, and the rationalist community. A lot of the shitty behavior of these groups, that Maciej has been hammering on for some time now in previous talks, is closely correlated to belief in AI X-risk. He seemed to be claiming that this correspondence is not just correlative but causative. That may or may not be true, but even if not we should focus on improving it all the same. To wit: 1. Stop treating every problem in the world as an engineering problem to be solved with clever software, and realize that people and social problems are more complex than that. This is one of the motivating factors behind wanting AI (so we can turn social problems into programming problems) and leads to megalomania on our part and deep resentment on everyone else's part. 2. Similarly, stop thinking us developers are the noble heroes who are going to save the world from the ignorance of the masses. This is both dead wrong (we are in a rapacious capitalist industry that mostly wants to eliminate anything that stands in its way, whether or not the impediment has good reason) and again is building resentment toward us. 3. Start worrying more about the extremely serious problems that are being caused by data retention and machine learning right now, like the permanent loss of privacy, increasing wealth consolidation in the hands of a few capitalists, and the "laundering" of human bias through supposedly-but-not-actually-neutral algorithms. 4. Realize that whatever neurons in our brains are lit up by religious belief don't necessarily require a "religion" in the traditional sense of the word, and that a lot of this eschatological transhumanist and AI X-risk stuff seems to slot into the same pathways, especially for very smart engineers. That doesn't make them wrong necessarily, but it does mean their claims should be analyzed with the same bias toward extreme skepticism that most of us would automatically use with religious claims. I suspect Maciej would be much less distressed by AI worry if the AI worriers also concerned themselves more with 1 through 4. At that point it would be at worst a harmless distraction, and maybe even beneficial if AI X-risk is in fact a legitimate concern. That, I think, would be best for everyone.