I think the issue is that such contemplations are taking a very large leap forward over a moat that we don't know yet how to cross. The moat being, assuming we

I think the issue is that such contemplations are taking a very large leap forward over a moat that we don't know yet how to cross. The moat being, assuming we could create a "safe" ASI, the transition from where we are now moving forward to that point without essentially destroying ourselves. Even the primitive AI that we have now is looking like it will bring about potential severe issues with verifiable reality and trust. That is likely an untenable existence for stable social order.