> Why? What is the reasoning this "is naturally expected" There are many articles written on such a topic. In short, we have no way of predicting how an AGI wi

> Why? What is the reasoning this "is naturally expected" There are many articles written on such a topic. In short, we have no way of predicting how an AGI will think, and there are more pathways to it being our enemy (intentionally or not) than to it being our ally. Especially since we can't even conceive of what it would look like for an entity to be the ally of all of humanity - humanity itself is not united on any goal at all. Pick one goal. Any goal that does or could affect humanity on a global scale. Now try to work out a plan to achieve that goal. Does your plan have the potential to anger a military power? If yes, you're a threat to humanity if you try to enact that goal. Even beyond the reasoning that AGI is likely to be dangerous, imagine it's a just 50/50 chance. Or even a 10% chance. Even a 5% chance. How low does the chance of human extinction need to go before you're willing to press The Button? Most of the arguments I see here amount to either "There is literally no risk of superhuman AI threatening human extinction," which is unequivocally wrong, or "There is literally no possibility of AGI existing," which is also unequivocally wrong. People usually say, "well it's at least decades away," which is actually them in denial that AGI can exist and be an existential threat. Because if they really believed it could happen in a few decades it would still be worth working on. Imagine someone told you "In 40 years a superhuman AGI will awaken and flip a coin to decide whether or not it destroys humanity," how long would you wait to start working on defense?