Wooly Definitions: The only definition that matters is ability to make correct predictions about the future. All other things called "intelligence" are a conseq

Wooly Definitions: The only definition that matters is ability to make correct predictions about the future. All other things called "intelligence" are a consequence of this. This definition is formalized with AIXI, which is uncomputable, but computable approximations exist. Hawking's Cat, Einstein's Cat: Scams require some intelligence from the victim. The victim needs to mistakenly believe they're doing something smart. Cats are too stupid to scam. Unlike humans they can't talk and they fail the mirror test, suggesting no self awareness. Human behavior when confronted with a deceptive superintelligence is not going to be the same as cat behavior when confronted with a deceptive human. Emus: The name "Great Emu War" is a joke. The humans had limited resources available, because killing emus wasn't important enough to justify more. If we really wanted to kill all the emus we could do it. We've made plenty of other animals extinct. The motivation for an AI is set by its reward function, which can be as high as necessary. Slavic Pessimism: This argument suggest that building nuclear weapons is impossible. Complex Motivations: This isn't obvious nonsense, but consider that all intelligences we've seen so far are the result of evolution, which tends to produce a more complexity than needed. A leg is more complicated than a wheel. A non-evolved intelligence would not necessarily have complex motivations. Actual AI: Compare with the argument from actual primitive nuclear reactors, which get mildly warm and never explode. My Roommate: As a human, you roommate had a human reward function. Unsurprisingly, he acted like a human. Why should a non-human reward function result in human-like behavior? Brain Surgery: Brain surgery would be a lot easier if you could take backups of brains, and duplicate them, and you had no concern about killing the patient. Childhood: If this turns out to be needed, why can't increasing intelligence result in increased ability to simulate an environment suitable for raising superintelligences? Gilligan's Island: There's no reason to assume an AI would be isolated. It could have or gain access to the internet and most human knowledge, and the mind architecture could contain many independent sub-minds. Grandiosity: This depends on assigning ethical value to hypothetical humans, which isn't obviously correct. Outside Argument, Megalomania, Comic Book Ethics, String Theory For Programmers: Ad-hominem. Transhuman Voodoo, Data Hunger, AI Cosplay: Why should something be false because it's deplorable? And why should something be false because it encourages deplorable behavior? Religion 2.0: Ted Chiang talked about the definition of "magic" in an interview (https://medium.com/learning-for-life/stories-of-ted-chiangs-... https://medium.com/learning-for-life/stories-of-ted-chiangs-...). >Another way to think about these two depictions is to ask whether the universe of the story recognizes the existence of persons. I think magic is an indication that the universe recognizes certain people as individuals, as having special properties as an individual, whereas a story in which turning lead into gold is an industrial process is describing a completely impersonal universe. All religions require some element of magic. Even Buddhism, which is arguably the least magical of all religions, treats consciousness as magic. AI requires no magic, therefore it is not a religion. Simulation Fever: Simulated universes do not have to be magical by Chiang's definition. A universe could be simulated by something that pays no attention to individuals within the simulation, eg. something that lets a large number of universes run their course then examines them statistically. Possibility of this increases the possibility of living in a non-magical universe despite possibility of living in a simulation. Incentivizing Crazy: This isn't an argument, it's a description of a field. Perhaps the author meant it to be an ad-hominem: "the idea is false because crazy people believe it".