Exactly! A distinction that doesn't make a difference is a very poor distinction. "If I take my Tesla car and beat it up with a hammer, it's my right to do it

Exactly! A distinction that doesn't make a difference is a very poor distinction. "If I take my Tesla car and beat it up with a hammer, it's my right to do it. My neighbor might think that I am crazy, but it's my property. It's just a machine and I can do with it what I want. But if I beat up my dog, the police will come and arrest me. What is the difference? The dog can suffer, the dog is a conscious being; it has some rights. The Tesla is not a conscious being. But if machines at some point become conscious, then there will be ethical, legal, and political consequences. So, it matters a great deal whether or not a machine is conscious." At various times, in certain circles, it has been argued that animals are not 'sensate' (or 'conscious' or various other terms); if you poke a dog with a needle, it acts like it is hurt. But it isn't really in pain. Instead it is 'pretending', acting like it is in pain to avoid further damage or to gain sympathy. Mostly, this kind of idea has been discarded. Mostly. Koch is making the same kind of argument. If you had a computer based system that looked and acted in all ways like a human being, or an animal, you could cheerfully "beat it up with a hammer" because it's not a 'conscious' being. Even if it looks and acts like one, it simply cannot be because you aren't looking "at the behavior of the machine, but at the actual substrate that has causal power" which is a von Neumann architecture, and "the causal power of Von Neumann-chips is minute" and therefore "any AI that runs on such a chip, however intelligent it might behave, will still not be conscious like a human brain." To me, this is a philosophically uninteresting theory. (And I fear the day that they start applying their consciousness -meter to random people: "The theory has given rise to the construction of a consciousness-meter that is being tested in various clinics in the U.S. and in Europe. The idea is to detect whether seriously brain-injured patients are conscious, or whether truly no one is home.") The idea that yes, you could build a machine that looks and acts intelligent, conscious, causal, or whatever word you like, but it still would not be real intelligence, or whatever. It always struck me as very akin to the "we don't understand it, so therefore it's magic" school.