This is more or less exactly where I land on this. The extinction risk argument is somewhat plausible in the long term[0], but it doesn't seem that we are espe

This is more or less exactly where I land on this. The extinction risk argument is somewhat plausible in the long term[0], but it doesn't seem that we are especially close to developing that class of problem. Meanwhile, there are very real and immediate risks to this type of technology -- in various forms of giving some humans more power over other humans. The attacks proposed by the extinction risk people actively make those immediate risks worse. The only way we've found to prevent some from leveraging almost any technology to control others is to democratize said technology -- so that everyone has access. Limiting access will just accrue more power to those with access, even if they need to break laws to get it. [0]- Say, the lower bound of the 95% confidence interval is 50-100 years out. People who I know to be sensible disagree with me on this estimate, but it's where I honestly end up.