7 ms·

> Look, clearly we’re machines governed by the laws of physics. We’re computers made of meat, as Marvin Minsky put it. This really seem inescapable to me. Th

> Look, clearly we’re machines governed by the laws of physics. We’re computers made of meat, as Marvin Minsky put it. This really seem inescapable to me. The remaining arguments all seem like smokescreens. Being able to copy computer programs? Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all. Even if there is some quantum gravitational aspect of human consciousness, the idea that you'd be able to distinguish between that and a reasonable computational approximation of it seems far-fetched at best. That said, I feel that we are extremely far away from producing a computer-based consciousness. Even human consciousness needs a couple of years of full-speed processing to reach the point at which it could even vaguely be considered conscious by adult standards. Most likely 5-10 years after we give up on the notion of creating an AI we'll discover that we created one 15 years ago but just haven't yet been able to recognize it as an emergent property of a hugely complex system.

Animats 10y ago
"Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all." That's a feature of the Xen hypervisor.[1] You can even migrate a process from one machine to another without stopping it. [1] http://wiki.prgmr.com/mediawiki/index.php/Chapter_9:_Xen_Migration http://wiki.prgmr.com/mediawiki/index.php/Chapter_9:_Xen_Mig...
- Zikes 10y ago
  This puts a fun spin on the teleportation/cloning thought experiment.
  - ArkyBeagle 10y ago
    What do you think all the shimmery stuff on the Star Trek TOS transporter was?
- andrewla 10y ago
  > distributed And even the Xen example is rough; the new copy of the system will be mostly functionally identical, but any reliance on the state of external things (like remote storage or IO or the state of unfair nondeterministic locks) will rapidly cause state to diverge. Which only begs the question about how high-fidelity a copy has to be to be a "copy", even in the case of a hypothetical consciousness copy, and leads into the even more subtle question of the time evolution of a person and whether the me of now and the me of twenty seconds ago can be said to represent the same singular "consciousness" at all.
- scott_s 10y ago
  You linked to the migration of a single operating system kernel. That is very different from reproducing the running state of an entire distributed system. Migrating a single kernel is not easy, but straight-forward: you have access to all of the relevant pieces (in memory process state, and state on disks) in one place. With a distributed system, that is not so, since the the running state of a distributed system will include information in-flight, some of which is not virtualized. It is possible to migrate the state of a distributed system by designing a protocol such that all of the components stop sending information around, so that you can be sure you did not miss anything. But I think that violates the spirit of the phrase "running state", since you essentially stopped the system. It's also not necessarily so that post-migration, the system will act the same.
  - deegles 10y ago
    So, basically the same reasons why I'd never step into a teleporter :)
  - yarou 10y ago
    Why would you need to copy the entire running state of the distributed system?
  - teraflop 10y ago
    You might be interested in the Chandy-Lamport snapshot algorithm, which can do what you describe without stopping the system's execution.
    - scott_s 10y ago
      I'm aware of such things, and in fact I work on a system that supports something very similar. (Paper to be published by my colleagues in the industry track of VLDB.) But that is a logical snapshot. Through a clever protocol, and assuming some characteristics of your transport, you can save a state across your system that is logically consistent. That is, if you restored to this logical state, the distributed application can resume from that point, and maintain integrity. You may, however, lose information from your system that was computed after you took your snapshot. The state is logically consistent, but unlikely to be representative of any actual point in time.
joe_the_user 10y ago
That said, I feel that we are extremely far away from producing a computer-based consciousness Views of distance to the production of "conscious" machines are likely all over the place because there aren't really any clear ideas about what consciousness is and why it's important. * Is it one state of mind that represents another state of mind? We trivially can construct computers with states which reflect each other. * Is it being able to act without external control, ie, apparently randomly? Computers can do that better than people right now. And despite the fogginess and disagreement about what consciousness is, it's highly valued and thought necessary by many people looking into this stuff. If you think about that for a second, it's a reason to be even more skeptical about it.
- andyh2 10y ago
  I believe the broadest consensus defines consciousness as having a subjective experience. What's unclear, and some deem impossible, is how we can externally detect consciousness. A machine could act and respond in a manner indistinguishable from humans but still be unconscious.
  - drabiega 10y ago
    Not just a machine. I agree with definition of consciousness, but it's implication is that we can only ever be sure of our own present consciousness. The consciousness of other people and even our past selves can't be verified.
  - andrewla 10y ago
    Ugh... replacing one vague and undefined word with three vague and undefined words (I'm including "having" because that's a hard one to nail down in context). function i_am_conscious(x) { return x + Math.random(); } This function has a subjective experience. To avoid any discussion of consciousness becoming a discussion of the meaning of the word "consciousness" we have to accept that consciousness (like everything else in the real world) is not a boolean property, but rather a subjective "I know it when I see it" sort of thing. Or in Bayesian terms, it's a weighted evaluation of evidence against a vast array of mostly unknown priors. That's why I think it will sneak up on us; there will be systems that some very small group of people (mostly crazy) think are conscious, then systems that a slightly larger group of people (maybe more crazy, maybe less) think are conscious, until one day You wake up and realize that You are one of the crazy people. But directed attempts to satisfy some definition of "consciousness" in a system will just result in a counterexample to that definition.
petra 10y ago
There are already some AI systems with basic forms of consciousness at [1]. I wonder to what human mental age they are equal to ? [1]http://www.scientificamerican.com/article/automaton-robots-become-self-aware/ http://www.scientificamerican.com/article/automaton-robots-b...
- Bromskloss 10y ago
  Really? They count that as consciousness?
- dekhn 10y ago
  I wouldn't call that a basic form of consciousness.
Koshkin 10y ago
To me, what seems to be almost inescapable is the thought that a "thinking" computer will, eventually, have many, many orders of magnitude greater difference in thinking power between it and a human than a human has between itself and, say, an insect.

Animats 10y ago

"Reproducing the running state of even a moderately complex distributed system is effectively impossible, so this argument bothers me not at all." That's a feature of the Xen hypervisor.[1] You can even migrate a process from one machine to another without stopping it. [1] http://wiki.prgmr.com/mediawiki/index.php/Chapter_9:_Xen_Migration http://wiki.prgmr.com/mediawiki/index.php/Chapter_9:_Xen_Mig...

Zikes 10y ago
This puts a fun spin on the teleportation/cloning thought experiment.
- ArkyBeagle 10y ago
  What do you think all the shimmery stuff on the Star Trek TOS transporter was?
andrewla 10y ago
> distributed And even the Xen example is rough; the new copy of the system will be mostly functionally identical, but any reliance on the state of external things (like remote storage or IO or the state of unfair nondeterministic locks) will rapidly cause state to diverge. Which only begs the question about how high-fidelity a copy has to be to be a "copy", even in the case of a hypothetical consciousness copy, and leads into the even more subtle question of the time evolution of a person and whether the me of now and the me of twenty seconds ago can be said to represent the same singular "consciousness" at all.
scott_s 10y ago
You linked to the migration of a single operating system kernel. That is very different from reproducing the running state of an entire distributed system. Migrating a single kernel is not easy, but straight-forward: you have access to all of the relevant pieces (in memory process state, and state on disks) in one place. With a distributed system, that is not so, since the the running state of a distributed system will include information in-flight, some of which is not virtualized. It is possible to migrate the state of a distributed system by designing a protocol such that all of the components stop sending information around, so that you can be sure you did not miss anything. But I think that violates the spirit of the phrase "running state", since you essentially stopped the system. It's also not necessarily so that post-migration, the system will act the same.
- deegles 10y ago
  So, basically the same reasons why I'd never step into a teleporter :)
- yarou 10y ago
  Why would you need to copy the entire running state of the distributed system?
- teraflop 10y ago
  You might be interested in the Chandy-Lamport snapshot algorithm, which can do what you describe without stopping the system's execution.
  - scott_s 10y ago
    I'm aware of such things, and in fact I work on a system that supports something very similar. (Paper to be published by my colleagues in the industry track of VLDB.) But that is a logical snapshot. Through a clever protocol, and assuming some characteristics of your transport, you can save a state across your system that is logically consistent. That is, if you restored to this logical state, the distributed application can resume from that point, and maintain integrity. You may, however, lose information from your system that was computed after you took your snapshot. The state is logically consistent, but unlikely to be representative of any actual point in time.

joe_the_user 10y ago

That said, I feel that we are extremely far away from producing a computer-based consciousness Views of distance to the production of "conscious" machines are likely all over the place because there aren't really any clear ideas about what consciousness is and why it's important. * Is it one state of mind that represents another state of mind? We trivially can construct computers with states which reflect each other. * Is it being able to act without external control, ie, apparently randomly? Computers can do that better than people right now. And despite the fogginess and disagreement about what consciousness is, it's highly valued and thought necessary by many people looking into this stuff. If you think about that for a second, it's a reason to be even more skeptical about it.

andyh2 10y ago
I believe the broadest consensus defines consciousness as having a subjective experience. What's unclear, and some deem impossible, is how we can externally detect consciousness. A machine could act and respond in a manner indistinguishable from humans but still be unconscious.
- drabiega 10y ago
  Not just a machine. I agree with definition of consciousness, but it's implication is that we can only ever be sure of our own present consciousness. The consciousness of other people and even our past selves can't be verified.
- andrewla 10y ago
  Ugh... replacing one vague and undefined word with three vague and undefined words (I'm including "having" because that's a hard one to nail down in context). function i_am_conscious(x) { return x + Math.random(); } This function has a subjective experience. To avoid any discussion of consciousness becoming a discussion of the meaning of the word "consciousness" we have to accept that consciousness (like everything else in the real world) is not a boolean property, but rather a subjective "I know it when I see it" sort of thing. Or in Bayesian terms, it's a weighted evaluation of evidence against a vast array of mostly unknown priors. That's why I think it will sneak up on us; there will be systems that some very small group of people (mostly crazy) think are conscious, then systems that a slightly larger group of people (maybe more crazy, maybe less) think are conscious, until one day You wake up and realize that You are one of the crazy people. But directed attempts to satisfy some definition of "consciousness" in a system will just result in a counterexample to that definition.

petra 10y ago

There are already some AI systems with basic forms of consciousness at [1]. I wonder to what human mental age they are equal to ? [1]http://www.scientificamerican.com/article/automaton-robots-become-self-aware/ http://www.scientificamerican.com/article/automaton-robots-b...

Bromskloss 10y ago
Really? They count that as consciousness?
dekhn 10y ago
I wouldn't call that a basic form of consciousness.

Koshkin 10y ago

To me, what seems to be almost inescapable is the thought that a "thinking" computer will, eventually, have many, many orders of magnitude greater difference in thinking power between it and a human than a human has between itself and, say, an insect.