The OP's point is: 1. The "superintelligence" crowd are capable of making very convincing arguments which he simply cannot refute on their own terms (I guess h

The OP's point is: 1. The "superintelligence" crowd are capable of making very convincing arguments which he simply cannot refute on their own terms (I guess he'd say your example is one of them) 2. Since he can't refute them, if he engages with them, he will probably end up accepting them 3. However other groups exist which a) present equally-convincing (to the OP) arguments; and b) believe in a future which contradicts that espoused by the superintelligence crowd 4. With multiple sets of equally-convincing arguments leading to contradictory outcomes, it is not rational to accept any one set. This is not a conclusion which sits well with tech people, because they like to assume they are capable of engaging with each set of arguments and deciding on the most likely set. OP's point is that because each group is really good at making these arguments, it's easy to get this decision-making process wrong. I think that's a pretty great point, but it requires accepting the fact that you may be less than perfect at picking apart 'superingelligence' arguments and premises.