9 ms·

> Any AI that runs on such a chip, however intelligent it might behave, will still not be conscious like a human brain.... No. It doesn't matter whether the Von

> Any AI that runs on such a chip, however intelligent it might behave, will still not be conscious like a human brain.... No. It doesn't matter whether the Von Neumann machine is running a weather simulation, playing poker, or simulating the human brain; its integrated information is minute. Consciousness is not about computation; it's a causal power associated with the physics of the system. There are serious problems with this. Koch will have to explain how a simulation of a brain can reproduce to perfect detail all the possible behaviors of a brain without having an equivalent integrated information. Presumably integrated information is a property of the organization of a process, and as such it should have consequences for its set of possible behaviors. So if the von Neumann system has constrained integrated information, its behavior should be constrained as well in some externally identifiable way. But by assumption the simulation could be arbitrarily good. How does Koch break this tension? The other glaring issue is the fact that consciousness under this view has no explanatory power for the system's behavior. If a non-conscious system can reproduce exactly the behavior of a conscious system, then there is nothing informative to the behavior from the property of conscious; it is entirely incidental to a conscious system's behavior. It doesn't explain why a conscious subject winces with pain after touching a hot stove, nor why it learns to never touch a hot stove bare-handed again. That's a pill too big to swallow.

KhoomeiK 6y ago
Yeah, I didn't really understand the "physics" requirement as well. Sure if you're measuring the integrated information of the hardware architecture it'll be quite low, but why isn't the integrated information of the abstracted neural simulation software (which is presumably significantly higher) relevant?
- stubish 6y ago
  Especially when the "physics" is the same. My brain is built from the same "physics" as the computer chip. We know complex systems are built on simple systems. Koch is claiming that highly connected systems cannot be built from less connected systems?
- mannykannot 6y ago
  Koch does not even seem to be consistent with hardware. A synapse is sufficiently complex, yet a transistor is not? He seems to be offering a peculiar panpsychism in which almost everything, except transistors, has the potential for consciousness. It seems to be a remarkably tendentious view.
  - mannykannot 6y ago
    ...and thinking about this some more, one does not, of course, need transistors to make a Turing-equivalent machine - one could, in principle, use neurons and synapses... or people. Koch's position, at least as he has presented it in this interview, seems utterly incoherent.
    - hackinthebochs 6y ago
      Agreed. "Integrated information" is intrinsically substance independent. Yet for some reason he doesn't grasp the obvious fact that a Turing machine running the right program would exhibit the same level of integrated information. His argument seems to require an extra ingredient to exclude programs running on computers but it goes unstated. I have yet to see something that resolves the obvious incoherence.
adrianmonk 6y ago
If anything, isn't the simulator more conscious? It's integrating all the information that the other one does. But it's also translating between its compute model and another compute model, which is additional integration of information. The simulator is "thinking about" whatever the simulated brain is thinking about and is also "thinking about" how it's doing the simulation.
doctoboggan 6y ago
I like his idea that consciousness requires a high degree of interconnectivity. I also disagree with his assertion that standard computers can't achieve consciousness. While a von Neumann machine doesn't typically have the high degree of physical interconnectivity that Koch and Tononi say is required, it has the potential through programming to have a high degree of virtual interconnectivity. I don't see why there is a need to differentiate between physical and virtual interconnectivity. To be fair to the authors, I have only read this article and the IEEE Spectrum article linked from the original. Maybe they go into more depth in their book.
mcguire 6y ago
Exactly! A distinction that doesn't make a difference is a very poor distinction. "If I take my Tesla car and beat it up with a hammer, it's my right to do it. My neighbor might think that I am crazy, but it's my property. It's just a machine and I can do with it what I want. But if I beat up my dog, the police will come and arrest me. What is the difference? The dog can suffer, the dog is a conscious being; it has some rights. The Tesla is not a conscious being. But if machines at some point become conscious, then there will be ethical, legal, and political consequences. So, it matters a great deal whether or not a machine is conscious." At various times, in certain circles, it has been argued that animals are not 'sensate' (or 'conscious' or various other terms); if you poke a dog with a needle, it acts like it is hurt. But it isn't really in pain. Instead it is 'pretending', acting like it is in pain to avoid further damage or to gain sympathy. Mostly, this kind of idea has been discarded. Mostly. Koch is making the same kind of argument. If you had a computer based system that looked and acted in all ways like a human being, or an animal, you could cheerfully "beat it up with a hammer" because it's not a 'conscious' being. Even if it looks and acts like one, it simply cannot be because you aren't looking "at the behavior of the machine, but at the actual substrate that has causal power" which is a von Neumann architecture, and "the causal power of Von Neumann-chips is minute" and therefore "any AI that runs on such a chip, however intelligent it might behave, will still not be conscious like a human brain." To me, this is a philosophically uninteresting theory. (And I fear the day that they start applying their consciousness -meter to random people: "The theory has given rise to the construction of a consciousness-meter that is being tested in various clinics in the U.S. and in Europe. The idea is to detect whether seriously brain-injured patients are conscious, or whether truly no one is home.") The idea that yes, you could build a machine that looks and acts intelligent, conscious, causal, or whatever word you like, but it still would not be real intelligence, or whatever. It always struck me as very akin to the "we don't understand it, so therefore it's magic" school.

KhoomeiK 6y ago

Yeah, I didn't really understand the "physics" requirement as well. Sure if you're measuring the integrated information of the hardware architecture it'll be quite low, but why isn't the integrated information of the abstracted neural simulation software (which is presumably significantly higher) relevant?

stubish 6y ago
Especially when the "physics" is the same. My brain is built from the same "physics" as the computer chip. We know complex systems are built on simple systems. Koch is claiming that highly connected systems cannot be built from less connected systems?
mannykannot 6y ago
Koch does not even seem to be consistent with hardware. A synapse is sufficiently complex, yet a transistor is not? He seems to be offering a peculiar panpsychism in which almost everything, except transistors, has the potential for consciousness. It seems to be a remarkably tendentious view.
- mannykannot 6y ago
  ...and thinking about this some more, one does not, of course, need transistors to make a Turing-equivalent machine - one could, in principle, use neurons and synapses... or people. Koch's position, at least as he has presented it in this interview, seems utterly incoherent.
  - hackinthebochs 6y ago
    Agreed. "Integrated information" is intrinsically substance independent. Yet for some reason he doesn't grasp the obvious fact that a Turing machine running the right program would exhibit the same level of integrated information. His argument seems to require an extra ingredient to exclude programs running on computers but it goes unstated. I have yet to see something that resolves the obvious incoherence.

adrianmonk 6y ago

If anything, isn't the simulator more conscious? It's integrating all the information that the other one does. But it's also translating between its compute model and another compute model, which is additional integration of information. The simulator is "thinking about" whatever the simulated brain is thinking about and is also "thinking about" how it's doing the simulation.

doctoboggan 6y ago

I like his idea that consciousness requires a high degree of interconnectivity. I also disagree with his assertion that standard computers can't achieve consciousness. While a von Neumann machine doesn't typically have the high degree of physical interconnectivity that Koch and Tononi say is required, it has the potential through programming to have a high degree of virtual interconnectivity. I don't see why there is a need to differentiate between physical and virtual interconnectivity. To be fair to the authors, I have only read this article and the IEEE Spectrum article linked from the original. Maybe they go into more depth in their book.

mcguire 6y ago

Exactly! A distinction that doesn't make a difference is a very poor distinction. "If I take my Tesla car and beat it up with a hammer, it's my right to do it. My neighbor might think that I am crazy, but it's my property. It's just a machine and I can do with it what I want. But if I beat up my dog, the police will come and arrest me. What is the difference? The dog can suffer, the dog is a conscious being; it has some rights. The Tesla is not a conscious being. But if machines at some point become conscious, then there will be ethical, legal, and political consequences. So, it matters a great deal whether or not a machine is conscious." At various times, in certain circles, it has been argued that animals are not 'sensate' (or 'conscious' or various other terms); if you poke a dog with a needle, it acts like it is hurt. But it isn't really in pain. Instead it is 'pretending', acting like it is in pain to avoid further damage or to gain sympathy. Mostly, this kind of idea has been discarded. Mostly. Koch is making the same kind of argument. If you had a computer based system that looked and acted in all ways like a human being, or an animal, you could cheerfully "beat it up with a hammer" because it's not a 'conscious' being. Even if it looks and acts like one, it simply cannot be because you aren't looking "at the behavior of the machine, but at the actual substrate that has causal power" which is a von Neumann architecture, and "the causal power of Von Neumann-chips is minute" and therefore "any AI that runs on such a chip, however intelligent it might behave, will still not be conscious like a human brain." To me, this is a philosophically uninteresting theory. (And I fear the day that they start applying their consciousness -meter to random people: "The theory has given rise to the construction of a consciousness-meter that is being tested in various clinics in the U.S. and in Europe. The idea is to detect whether seriously brain-injured patients are conscious, or whether truly no one is home.") The idea that yes, you could build a machine that looks and acts intelligent, conscious, causal, or whatever word you like, but it still would not be real intelligence, or whatever. It always struck me as very akin to the "we don't understand it, so therefore it's magic" school.