I already described the difference between fear mongering and pushing for caution, so don't conflate those terms. It's incoherent to not engage the arguments--t

I already described the difference between fear mongering and pushing for caution, so don't conflate those terms. It's incoherent to not engage the arguments--take the control problem into consideration. Are you familiar with the reasoning for the expectancy that an AGI will be built that is smarter than humans at any reasoning task? Are you willing to go along with that? If so, then what follows is that the AGI improves the design of its own software and hardware better than any human engineer. At this point, there is a recursive improvement process termed an "intelligence explosion". From that point forward, we have no idea how to ensure that the goals of the AGI remain in alignment with human values. And the AGI can out-maneuver any attempt to turn it off. It's not guaranteed that it will want to avoid being turned off; but you could easily see how avoiding being turned off would improve the probability that it achieves its optimization parameters. Where do you check out? Edit: Nick Bostrom is the thought leader on this subject and he spells it out in this talk: https://goo.gl/DzNnk2 https://goo.gl/DzNnk2. His book, Superintelligence, is really good. Edit 2: And here is the audio for a Sam Harris talk also on this subject: https://goo.gl/6wdG43 https://goo.gl/6wdG43