> We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. Seems like a distinction without a difference to me. Whether

> We're not even talking about AGI or an entity that is "smart" somehow. It's a bloody algorithm. Seems like a distinction without a difference to me. Whether or not the machine has some sort of "soul" is a question for philosophers or theologians, it has little bearing on the practical capabilities of the machine. > AGI isn't likely to destroy humanity. Can you give us some idea for the order of unlikely you are supposing? 1 in 10? 1 in 100? For your consideration: > During the next three months scientists in secret conference discussed the dangers of fusion but without agreement. Again Compton took the lead in the final decision. If, after calculation, he said, it were proved that the chances were more than approximately three in one million that the earth would be vaporized by the atomic explosion, he would not proceed with the project. Calculations proved the figures slightly less - and the project continued. http://large.stanford.edu/courses/2015/ph241/chung1/ http://large.stanford.edu/courses/2015/ph241/chung1/ Four in a million certainly "isn't likely" but Arthur Compton was apparently willing to halt the Manhattan Project if the likelihood of an atomic bomb triggering a fusion reaction in earth's atmosphere was merely that likely. Or to put it another way: If I load a revolver with a single bullet, spin the cylinder then point it at you, you are "not likely" to die with a 1 in 6 chance of the loaded chamber aligning with the barrel when I pull the trigger. Is Russian Roulette a game you'd like to play? Remember, it "isn't likely" that you're going to die.