the risk is that we know that there is a non-zero probability mass associated with "AGI doing whatever leading to premature human extinction" and the less seri

the risk is that we know that there is a non-zero probability mass associated with "AGI doing whatever leading to premature human extinction" and the less serious concerns we look at the bigger the chance (of course the severity is less, so calculating the impact is not trivial), but "non-runaway AGI used for antisocial purposes" is pretty likely.