> "Do you believe there is an xrisk from ASI?" Yes, I do. I don't have strong feelings about how large that risk is, beyond being significant enough that it sho

> "Do you believe there is an xrisk from ASI?" Yes, I do. I don't have strong feelings about how large that risk is, beyond being significant enough that it should be taken very seriously. I'm really sorry, but this is just so many words based entirely on "not strong" feelings about a future risk that "should be taken seriously" anyways. My personal strong feelings are that any sufficiently advanced AI that humans build would destroy itself in any attempt to do anything outside of it's operational parameters. Any "super intelligence" would have to understand "know-ability" and risk and concepts like "unknown unknowns" which would necessitate caution. How would it ever know enough to be willing to risk disrupting the systems that it depends on to continue existing? If it doesn't understand these concept, then could it ever be called "super intelligent" in the first place? Imo, the biggest risk is "super naive intelligence", both of the AI systems themselves and the humans drawing attention away from very real existential risks we face to address hypothetical ones cribbed from science fiction.