Assuming that we could develop much-smarter-than-human-AI, I would support a pause for exactly that reason: the Culture may be the best-case scenario, and the h

Assuming that we could develop much-smarter-than-human-AI, I would support a pause for exactly that reason: the Culture may be the best-case scenario, and the humans in the Culture are basically pets. And a lot of possible outcomes might be worse than the Culture. I am deeply baffled by the people who claim (1) we can somehow build something much smarter than us, and (2) this would not pose any worrying risks. That has the same energy as parents who say, "Of course my teenagers will always follow the long list of rules I gave them."