6 ms·

In Koch’s picture, then, the Turing Test is irrelevant to diagnosing inner life. What’s more, it implies that the transhumanist dream of downloading one’s mind

In Koch’s picture, then, the Turing Test is irrelevant to diagnosing inner life. What’s more, it implies that the transhumanist dream of downloading one’s mind into an (immortal) computer circuit is a fantasy. At best, such circuits would simulate the inputs and outputs of a brain while having absolutely no experience at all. It will be “nothing but clever programming… fake consciousness—pretending by imitating people at the biophysical level.” For that, he thinks, is all AI can be. These systems might beat us at chess and Go, and deceive us into thinking they are alive. But those will always be hollow victories, for the machine will never enjoy them. This is really damning humans with faint praise. "Machines may eventually do every job better than we can, and be immortal, but I promise that humans will remain superior in some completely undetectable way."

deleted 7y ago
[deleted]
htk 7y ago
Koch believes “that consciousness is a fundamental, elementary property of living matter.” Consciousness is “magic” then. Even if we build machines to mimic a real brain, “it’ll be like a golem stumbling about,” he writes: adept at the Turing Test perhaps, but a zombie. This guy created a whole moat of unfalsifiability around his views.
- vanusa 7y ago
  Consciousness is “magic” then. That doesn't follow at all.
  - clSTophEjUdRanu 7y ago
    They're in the camp that since you can't measure subjective experience its magic. Well, they're here reading this, since they cant measure their experience they must not exist. /s
- ben_w 7y ago
  Your second quote sounds like a fairly straightforward description of P-zombies to me: https://en.m.wikipedia.org/wiki/Philosophical_zombie https://en.m.wikipedia.org/wiki/Philosophical_zombie However, to add to the criticism you have of this article about this book, your two quotes taken together appear to be contradictory: if consciousness is a fundamental element of living matter, then given that we can make new living matter, why should there be any reason we can’t make a conscious artificial machine?
  - edflsafoiewq 7y ago
    P-zombies are magical nonsense.
    - goatlover 7y ago
      Magic has nothing to do with it. P-zombies are a philosophical thought experiment against physicalism. The point of the argument is to demonstrate that a purely physical description leaves out conscious experience. Nagel made the same point regarding bat sonar consciousness and science’s view from nowhere.
- goatlover 7y ago
  No more magic than anything else fundamental. I don’t know whether consciousness is fundamental, but magical has nothing to do with being fundamental, unless existence itself is magical.
rosybox 7y ago
How would Koch see a mechanism where neurons or other cells in a human brain are sequentially replaced over time with synthetic components that simulate their function, is the consciousness lost along the way? Is there consciousness as long as there is at least one biological cell left?
- mgolawala 7y ago
  I wonder if an intelligence higher than ours were to figure out exactly how our brain worked. Every neuron, every synapse, every hormone and neurotransmitter.. How memories were made, stored and retrieved. Would we appear to them to merely be "simulating" consciousness through the use of these mechanisms.
- thedirt0115 7y ago
  You should write a book, "The Cyborg of Theseus" :)
fdsfdsgad 7y ago
I am a "panpsychist," or whatever, and I don't really see the silver lining in any of this, either. I really don't care if DeepMind or Stockfish are enjoying how much they can drag me through the mud, because I already don't. It's a huge non sequitur to respond to such things with "it doesn't matter, it's just fake thought." And it is one the most embarassing things to see people going down that lane, alongside that other road, of "but can machines make art?"
lachlan-sneff 7y ago
I've never understood why people feel that way. As far as I can tell, there is no evidence that consciousness is not able to be emulated by a machine.
- fdsfdsgad 7y ago
  Emulation is fine, is whether machines are able to achieve "consciousness" that is at stake. I don't know if anyone feels strongly on the matter either, at least on the academic level. But it's the same as with other odd philosophical position: people run into problems with other alternatives, go like "why not..." and suddenly they have an odd belief.
  - lachlan-sneff 7y ago
    Personally, I don't see the difference.
    - jonkho 7y ago
      The difference is that a physical process makes specific physical state changes to the physical world while an emulation does not. A computer simulation of an ice cube melting does not and cannot make state changes to the physical world as a physical ice cube can. For example, if you wanted to chill your beverage, dropping an ice-cubed sized computer running the algorithmic emulation of an of an ice cube melting will not chill your drink no matter how perfect your emulation is, while a physical ice cube can perform that physical state change of your drink. Is consciousness a physical phenomenon? If it's not physical, then what is it? Is it an algorithm? Well, for one, there is no precedent. Nothing else in the material world is purely an algorithm...except maybe a computer running an algorithm...
      - perl4ever 7y ago
        "If it's not physical, then what is it? Is it an algorithm? Well, for one, there is no precedent. Nothing else in the material world is purely an algorithm...except maybe a computer running an algorithm..." I don't see the distinction you're making. I think I want to say "ok, so nothing is purely an algorithm". A computer relies on the rest of the universe just like your brain and I guess it's therefore a "physical phenomenon"? In order to draw a distinction, perhaps one could say that a "pure algorithm" is something that can run on "many" substrates, so the difference between a "pure algorithm" and a "physical phenomenon" is about the variance in the substrates maybe? But everything is physical. Statements about abstractions are statements about patterns of physical things.
stared 7y ago
In this case, Koch really needs to read Daniel Dennett's "Consciousness Explained". Even though it does not explains consciousness, it takes it seriously to dispel myths and magical thinking about consciousness.
- JackFr 7y ago
  He makes the point that there’s no magic, but the tell is that he’s not able to explain it. His religion is that it’s explainable.
Gatsky 7y ago
Is there some proof that we can never detect consciousness?
unishark 7y ago
I think the author is being a bit fast and loose with Koch's quotes. Koch is not postulating that computers can never be conscious, he's generally saying current approaches to making computer systems are not based on feedback hence can't be conscious because IIT requires feedback. This paraphrase "Koch believes the theory establishes that machines built along the lines of our current silicon-chip technology can never become conscious, no matter what awesome degree of processing power they possess." would seem to come from this statement: "Whether or not a network has this property of influencing itself depends on its architecture. If information is merely “fed forward” to convert inputs to outputs, as in digital computers, then IIT insists it can only generate zombie intelligence." However all IIT (and presumably therefore Koch) requires we do to solve this is use a different architecture (i.e. some kind of self-modifying feedback approach in the computer). "In Koch’s words, consciousness is “a system’s ability to be acted upon by its own state in the past and to influence its own future.”"

deleted 7y ago

[deleted]

htk 7y ago

Koch believes “that consciousness is a fundamental, elementary property of living matter.” Consciousness is “magic” then. Even if we build machines to mimic a real brain, “it’ll be like a golem stumbling about,” he writes: adept at the Turing Test perhaps, but a zombie. This guy created a whole moat of unfalsifiability around his views.

vanusa 7y ago
Consciousness is “magic” then. That doesn't follow at all.
- clSTophEjUdRanu 7y ago
  They're in the camp that since you can't measure subjective experience its magic. Well, they're here reading this, since they cant measure their experience they must not exist. /s
ben_w 7y ago
Your second quote sounds like a fairly straightforward description of P-zombies to me: https://en.m.wikipedia.org/wiki/Philosophical_zombie https://en.m.wikipedia.org/wiki/Philosophical_zombie However, to add to the criticism you have of this article about this book, your two quotes taken together appear to be contradictory: if consciousness is a fundamental element of living matter, then given that we can make new living matter, why should there be any reason we can’t make a conscious artificial machine?
- edflsafoiewq 7y ago
  P-zombies are magical nonsense.
  - goatlover 7y ago
    Magic has nothing to do with it. P-zombies are a philosophical thought experiment against physicalism. The point of the argument is to demonstrate that a purely physical description leaves out conscious experience. Nagel made the same point regarding bat sonar consciousness and science’s view from nowhere.
goatlover 7y ago
No more magic than anything else fundamental. I don’t know whether consciousness is fundamental, but magical has nothing to do with being fundamental, unless existence itself is magical.

rosybox 7y ago

How would Koch see a mechanism where neurons or other cells in a human brain are sequentially replaced over time with synthetic components that simulate their function, is the consciousness lost along the way? Is there consciousness as long as there is at least one biological cell left?

mgolawala 7y ago
I wonder if an intelligence higher than ours were to figure out exactly how our brain worked. Every neuron, every synapse, every hormone and neurotransmitter.. How memories were made, stored and retrieved. Would we appear to them to merely be "simulating" consciousness through the use of these mechanisms.
thedirt0115 7y ago
You should write a book, "The Cyborg of Theseus" :)

fdsfdsgad 7y ago

I am a "panpsychist," or whatever, and I don't really see the silver lining in any of this, either. I really don't care if DeepMind or Stockfish are enjoying how much they can drag me through the mud, because I already don't. It's a huge non sequitur to respond to such things with "it doesn't matter, it's just fake thought." And it is one the most embarassing things to see people going down that lane, alongside that other road, of "but can machines make art?"

lachlan-sneff 7y ago

I've never understood why people feel that way. As far as I can tell, there is no evidence that consciousness is not able to be emulated by a machine.

fdsfdsgad 7y ago
Emulation is fine, is whether machines are able to achieve "consciousness" that is at stake. I don't know if anyone feels strongly on the matter either, at least on the academic level. But it's the same as with other odd philosophical position: people run into problems with other alternatives, go like "why not..." and suddenly they have an odd belief.
- lachlan-sneff 7y ago
  Personally, I don't see the difference.
  - jonkho 7y ago
    The difference is that a physical process makes specific physical state changes to the physical world while an emulation does not. A computer simulation of an ice cube melting does not and cannot make state changes to the physical world as a physical ice cube can. For example, if you wanted to chill your beverage, dropping an ice-cubed sized computer running the algorithmic emulation of an of an ice cube melting will not chill your drink no matter how perfect your emulation is, while a physical ice cube can perform that physical state change of your drink. Is consciousness a physical phenomenon? If it's not physical, then what is it? Is it an algorithm? Well, for one, there is no precedent. Nothing else in the material world is purely an algorithm...except maybe a computer running an algorithm...
    - perl4ever 7y ago
      "If it's not physical, then what is it? Is it an algorithm? Well, for one, there is no precedent. Nothing else in the material world is purely an algorithm...except maybe a computer running an algorithm..." I don't see the distinction you're making. I think I want to say "ok, so nothing is purely an algorithm". A computer relies on the rest of the universe just like your brain and I guess it's therefore a "physical phenomenon"? In order to draw a distinction, perhaps one could say that a "pure algorithm" is something that can run on "many" substrates, so the difference between a "pure algorithm" and a "physical phenomenon" is about the variance in the substrates maybe? But everything is physical. Statements about abstractions are statements about patterns of physical things.

stared 7y ago

In this case, Koch really needs to read Daniel Dennett's "Consciousness Explained". Even though it does not explains consciousness, it takes it seriously to dispel myths and magical thinking about consciousness.

JackFr 7y ago
He makes the point that there’s no magic, but the tell is that he’s not able to explain it. His religion is that it’s explainable.

Gatsky 7y ago

Is there some proof that we can never detect consciousness?

unishark 7y ago

I think the author is being a bit fast and loose with Koch's quotes. Koch is not postulating that computers can never be conscious, he's generally saying current approaches to making computer systems are not based on feedback hence can't be conscious because IIT requires feedback. This paraphrase "Koch believes the theory establishes that machines built along the lines of our current silicon-chip technology can never become conscious, no matter what awesome degree of processing power they possess." would seem to come from this statement: "Whether or not a network has this property of influencing itself depends on its architecture. If information is merely “fed forward” to convert inputs to outputs, as in digital computers, then IIT insists it can only generate zombie intelligence." However all IIT (and presumably therefore Koch) requires we do to solve this is use a different architecture (i.e. some kind of self-modifying feedback approach in the computer). "In Koch’s words, consciousness is “a system’s ability to be acted upon by its own state in the past and to influence its own future.”"