> The Argument From Gilligan's Island > A recurring flaw in AI alarmism is that it treats intelligence as a property of individual minds, rather than recognizi

> The Argument From Gilligan's Island > A recurring flaw in AI alarmism is that it treats intelligence as a property of individual minds, rather than recognizing that this capacity is distributed across our civilization and culture. This is a stronger argument in the post and we arguably did not know whether it is true when the post was made. AlphaZero destroyed the illusion in 2017. Thousands of years of human Go and Chess knowledge, played and accumulated in communities of many million players, were surpassed within hours of training. AlphaZero is limited in its domain of application but there is no guarantee that all future AIs would be similarly limited. > The Argument From Stephen Hawking's Cat ... > But ultimately, if the cat doesn't want to get in the carrier, there's nothing Hawking can do about it despite his overpowering advantage in intelligence. > Even if he devoted his career to feline motivation and behavior, rather than theoretical physics, he still couldn't talk the cat into it. An average human can set up a cat trap, and lure the cat into it. They don’t even need to do it themselves; asking or paying someone else to do it is simple enough. Humans don’t often fall into a similarly “obvious” trap but we fall for more subtle traps all the time: popularity, money, attraction, etc. A disembodied AI can use social engineering or financial incentives to get other humans to lure their targets into such a trap. > The Argument From My Roommate... > It's perfectly possible an AI won't do much of anything, except use its powers of hyperpersuasion to get us to bring it brownies. Yes, an AI can decide to lie around and just try to get brownies but we know that some kinds of AGI are potentially very powerful for achieving real-world tasks and there are groups who try hard to develop them. This is like saying an ancient bacteria we found from ten-thousand-year-old ice melted by global warming, or a group of advanced aliens who are arriving in three decades, could be totally harmless; just rest and don’t pay much attention to them. Most arguments I have read in the post are weak, so I’ll stop here.