Burden of proof on neutral matters depends on whoever makes the claim. But AI and the potential for doom isn't a neutral matter. The question is what should be

Burden of proof on neutral matters depends on whoever makes the claim. But AI and the potential for doom isn't a neutral matter. The question is what should be one's default belief until proven otherwise. It is a matter of determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. But when it comes to AGI, extreme caution is the only sensible default position. The history of humanity is replete with examples of the slightly more technologically advanced group decimating their competition. The default position should be that uneven advantage is extremely dangerous to those disadvantaged. This idea that an intelligence significantly greater than our own is benign just doesn't pass the smell test.