4 ms·

This is a good place as any to ask this. Do hackernews readers think that computers will eventually have a "mind" and "consciousness" ? Or will it just be simul

This is a good place as any to ask this. Do hackernews readers think that computers will eventually have a "mind" and "consciousness" ? Or will it just be simulated? As described in the chinese room thought experiment: https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

derrickdirge 11y ago
I struggle to draw distinctions between a simulated consciousness and a "real" one.
- bitcointicker 11y ago
  So you don't believe there is any distinction between being "alive" and running software? Providing of course the software is complex enough to display human traits such as thought, emotions, creativity and so forth? Just to be clear, I hold no solid viewpoint on this, I'm just interested as to what other folk here think.
  - kazinator 11y ago
    "Alive" doesn't require human traits. Bacteria and plants are alive. I'm with you though: consciousness and sense of "self" are just matters of complexity of the brain.
  - closed 11y ago
    I think making that distinction would be similar to saying that a computer is not running software because it was built inside minecraft. Although, it's possible that creating something that does what humans do requires the same materials humans are made of (but seems unlikely, or at least I think other materials will allow a close approximation).
  - njharman 11y ago
    "alive", life has deferent meaning (and defined requirements e.g self-reproduction) than "consciousness", "sentience" or "mind". Unless you belief in a mystical soul component (I do not), there is nothing else "there" but machine. A very complex, biological machine. But there is no reason to believe that that a sufficiently powerful software/hardware can not replicate the functionality of that machine exactly. Then what possible difference could there be between the two duplicate machines? But, we will have systems that exhibit a many aspects of "thought" and sentience long before we are able to code up homo sapien sapien.
    - bitcointicker 11y ago
      I'm not sure it's just about computational power. Currently we mainly deal with digital computers that process information in a very discrete way. This is a nice article with regards to the human brain being analogue or digital- https://www.quora.com/Is-the-human-brain-analog-or-digital https://www.quora.com/Is-the-human-brain-analog-or-digital The development of quantum computers could offer a huge leap forward with the development of AI.
  - sehugg 11y ago
    As soon as software can convince the majority of humanity that it's conscious, the question will be academic.
derivagral 11y ago
That depends on whether you think your own mind and consciousness are different (and how?) from a very advanced computer, as mentioned in your link.
- bitcointicker 11y ago
  Personally, I'm not sure, which is one of the reasons I posed the question. But I seem to be being down voted for even raising it...
dboreham 11y ago
(a) it doesn't matter because these are just names for things that we can't even define succinctly for humans and (b) they're the same thing (simulated or not simulated, what's the difference??)
- bitcointicker 11y ago
  Eventually I'm pretty sure this will be an important question. Hundreds, or thousands of years into the future. It will matter one day. Will these artificial life forms have the same rights as humans? If there is no difference between simulated or human conciousness, then the answer is yes, they should have the same rights. But will people accept this?
- ThrustVectoring 11y ago
  I've got a very clear model for what consciousness is. Let's start by looking at how brains work for something that is much easier to reason about - vision. Light hits the eye, and it activates a set of neurons that are wired up to the light receptors. There's further out neurons that are set up to catch certain higher-level patterns, like edge detection or objects headed directly toward the eye. Deeper and deeper these connections go, until you get something that you subjectively experience as "seeing". It's at this point that optical illusions work - flat images on paper that are designed to make the higher level visual pattern recognizers fire. At a very high level, how parts of the brain work is by activating based off the sensory data represented by the activation of other parts of the brain. Just like detecting edges in your visual field was important in the ancestral environment, so was answering questions like "what did you do on the hunt?" with a good story. Just like there's a portion of the brain the constructs edges from lower-level visual sensation, there's a portion of the brain that constructs narratives from lots of different kinds of sensations. Furthermore, this narrative module has access to remembered narratives (since consistent stories are better ones) as another form of sense data. That, in short, is what consciousness is - the brain making sensible stories out of the sense data it encounters, and making those stories available to the brain (including the story-making module, which is why you can make stories about the qualitative experience of story-making).
  - 11y ago
mesozoic 11y ago
It will of course be simulated. Without getting too metaphysical the question of whether there is anything beyond computation is at this point largely spiritual. There are also reasonable arguments that all of reality as we know it is one large simulation. http://www.simulation-argument.com/simulation.html http://www.simulation-argument.com/simulation.html
- erichocean 11y ago
  I'm not so sure that there's a fundamental difference between "simulating" consciousness and actually having consciousness.
ThrustVectoring 11y ago
1. What, exactly, do you mean by "consciousness"? 2. What is the difference between "having" consciousness and "simulating" it?
- bitcointicker 11y ago
  These seem plausible: https://en.wikipedia.org/wiki/Artificial_consciousness#Aspects_of_consciousness https://en.wikipedia.org/wiki/Artificial_consciousness#Aspec... Awareness, memory, leaning, anticipation and subjective experience. Empathy is a good example, will a computer ever "feel" empathy? Or will it be simulated empathy? Is there actually a distinction? I'm not sure.
tyingq 11y ago
I think it's a good question, and I'm surprised by the terse responses. The link you provided seems to indicate that people with reasonable credentials, either in Philosophy, or AI, struggle with it.
- bitcointicker 11y ago
  Indeed, it's perhaps one of the most fundamental questions in life, at least in my view. I only wish I could somehow peer into the distant future to see how things pan out. It's likely that eventually the boundaries between humans and AI will blur. I guess at some point there will be cognitive augmentation, but imagine the debate that will generate. Look how we view physical enhancement by drugs today in sport.
kazinator 11y ago
Will what be simulated, is the question. When we speak about consciousness, we are referring to something internal which we intuitively know about ourselves, but cannot directly observe in others. When we have a good artificial consciousness, the external behavior will be convincing. Then we can ask: is the visible behavior just some simulation (like a mindless script being followed) or is there an internal reality which is the same as our consciousness. Answers to the question will probably come from the convergence of two fronts: understanding better what the brain does, and correlating that with an understanding of what is going on in the machine. That is, maybe it will be shown that the kinds of states and state changes in the two are very similar (or can be mapped to each other in some way). In other words, we externalize the structures and states as much as possible and identify an equivalence. Then we can proclaim that what is going on in the machine isn't just some elaborate script; the states are actually human like: the external behavior is underpinned by apparently the same stuff that makes people tick. We will also simply be convinced on an emotional level, due to the machines leading complex lives, demonstrating traits such as shame, regret and self-loathing, as well as joy that looks genuine. There will be depressed AI's that require therapy (possibly from humans), and ones that choose to terminate themselves. People who are not computer scientists or philosophers will be convinced that the machines are conscious, by the ways in which their lives intertwine with those of the machines and the relationships they form; only a dwindling group of skeptics will remain, even long after there is no space on the field where the goal-posts can be moved any farther.
- bitcointicker 11y ago
  If this is indeed the way it does play out, and what you have said all sounds very plausible, it's going to raise all sorts of difficult questions for future generations. Who will decide how many of these AI's can be created? Will AI's ultimately be able to create their own offspring? We already have limited resources, this will only increase the demand :-) I see trouble ahead... as well huge opportunities for advancing our understanding of the Universe.
  - mlindner 11y ago
    Many of these questions have been explored in AI fiction for many decades. Even right down to the titles, like the book "Do Androids Dream of Electric Sheep" which became "Blade Runner" for the movie version. Or "I, Robot" by Isacc Asimov.
coffeeaddicted 11y ago
The chinese room thought experiment is simply annoying because it doesn't even get the questions right. It basically asks if the hardware becomes conscious/understanding when running software - instead of asking the real question - if the software can gain the understanding/consciousness. The man in the room is just a hardware component replacing a computer processor. But the only place to look for intelligence in this experiment should be in the rule-system, not in the processor executing those rules. As those rule systems decide if the communication makes sense or not. To make a real conversion the rule-system needs flexibility to handle dynamic input and thereby creating a dynamic flow between the inside and outside of the room, but that flow is about symbol manipulation, not about the symbol manipulator. I think no one in AI ever even argued that the hardware part of a computer would gain understanding. The only good part about this thought experiment is that it makes the hard question of consciousness - how can it arise from physical phenomena - somewhat more obvious. But it's still the same question as for normal human minds and neither an argument for software systems becoming conscious nor against it.
geofft 11y ago
If you're going to apply a word to a place where it previously hasn't been meaningful to apply the word, it's worth clarifying what meaning that word now has. For instance, is Android a Linux distro? Depends why you're using the phrase "Linux distro". For some applications of that phrase, yes; for some applications no. Saying that it is, or that it isn't, doesn't tell you anything more, and doesn't help you answer those questions; you still have to answer them, so you might as well answer them directly. For instance, if you're packaging some software for a bunch of Linux distros, you probably don't want to count Android (or at least, handle it very differently from Debian or Fedora). If you're tracking fixes to kernel CVEs, you probably do want to count Android. Or, more mathematically, is 0^0 = 0 or 1? Depends how you ended up with 0^0 in the first place: if through 0^y as y approaches 0, then 0, but if through x^0 as x approaches 0, then 1. Saying "It's 1" or "It's undefined" is satisfying to the part of the human brain that likes clean answers, but it doesn't inform us. Similarly, let's not ask "what does 'consciousness' really mean", but "where do we use the term 'consciousness'." Should AIs be able to drive a car unsupervised? Should a sufficiently advanced AI have civil rights, such as the right to life? Should an AI be entrusted with political office? Or with the vote, and if so, what representation is fair? Can I give an AI ownership of a company? For the religious among us, do AIs have the ability to have human-like morality? Those are all questions worth answering, but saying "Yes, this computer has a conscious mind" or "No, it doesn't" is not really going to help those questions get answered. The Chinese room experiment is sort of not useful in that, while "the ability to learn Chinese" is a thing we expect of consciousnesses, it's not really generalizable. Nothing else in our society really depends on people's ability to learn a language; those few things that do, generally don't require the learner to be a person (e.g., you want something translated and you give it to a company).
jtolmar 11y ago
Yes, it will be possible unless you specifically construct your definition of consciousness to make it impossible. However, having a mind is unnecessary (and inefficient) for the most intelligent possible machines. Imagine extremely advanced future branches of statistics, which have yet-to-be-invented but still entirely statistical methods for discovering state spaces and causal rules, all with formal proofs of optimality. Imagine that, even if executing these methods is impossible in physical hardware, future computer science is capable of proving optimal sampling methods for a given problem and level of available computing resources. A machine that executes these algorithms is smarter than us, but does not have a mind. Will we construct machines with minds despite that? Since "to see if we can" is usually sufficient motivation for humans, I expect yes. Biologically-inspired computing techniques will probably lead to computers with minds, and I can see these computers eventually becoming smarter than humans. Will we make such machines before we invent enough of future-statistics to make a mindless machine that's smarter than us? I don't know, that's an interesting question.
jhdevos 11y ago
Of course, another question is: will the AI's consider us humans to have the equivalent of their consciousness? Or will they think "These humans don't not really possess consciousness, they just exhibit a crude biological simulation of it". (BTW: This is not just meant as a joke, nor just as instigating fear of the singularity - turning a question on its head can sometimes lead to interesting insights).
mindcrime 11y ago
Do hackernews readers think that computers will eventually have a "mind" and "consciousness" ? Or will it just be simulated? Personally? Yeah, I do think computers will eventually achieve "consciousness". I don't think of consciousness as being anything magical or terribly special to the point that a computer couldn't have it. I generally think it's just a question of a sort of recursion where the brain is "hearing" it's own internal processing as it works. When we ask questions, run simulations (day-dream) and we "see" and "hear" those things, I suspect it's just some sort of feedback loop between internal modules of the brain. I'd go so far as to suggest that some computer programs may already be conscious, and we just don't have any good way to measure/test that fact.
criddell 11y ago
Searle's Chinese Room experiment always bothered me because it feels like he is too tightly constraining "understanding". Because an airplane doesn't flap it's wings to fly like a bird does, should we say it isn't really flying?
DonaldFisk 11y ago
Every time there's a "breakthrough" in AI, this question crops up. Consciousness is poorly understood and there's no good reason to think it's a product of intelligence. In this case, all that's happened is that a game played by people can now be played well by a computer program which is designed specifically to recognize patterns. (A go position is just a 19x19 array of trits.) Computer programs reached human-level performance decades ago at chequers/draughts, and later at chess and backgammon. Those programs did not lead to a general AI.

derrickdirge 11y ago

I struggle to draw distinctions between a simulated consciousness and a "real" one.

bitcointicker 11y ago
So you don't believe there is any distinction between being "alive" and running software? Providing of course the software is complex enough to display human traits such as thought, emotions, creativity and so forth? Just to be clear, I hold no solid viewpoint on this, I'm just interested as to what other folk here think.
- kazinator 11y ago
  "Alive" doesn't require human traits. Bacteria and plants are alive. I'm with you though: consciousness and sense of "self" are just matters of complexity of the brain.
- closed 11y ago
  I think making that distinction would be similar to saying that a computer is not running software because it was built inside minecraft. Although, it's possible that creating something that does what humans do requires the same materials humans are made of (but seems unlikely, or at least I think other materials will allow a close approximation).
- njharman 11y ago
  "alive", life has deferent meaning (and defined requirements e.g self-reproduction) than "consciousness", "sentience" or "mind". Unless you belief in a mystical soul component (I do not), there is nothing else "there" but machine. A very complex, biological machine. But there is no reason to believe that that a sufficiently powerful software/hardware can not replicate the functionality of that machine exactly. Then what possible difference could there be between the two duplicate machines? But, we will have systems that exhibit a many aspects of "thought" and sentience long before we are able to code up homo sapien sapien.
  - bitcointicker 11y ago
    I'm not sure it's just about computational power. Currently we mainly deal with digital computers that process information in a very discrete way. This is a nice article with regards to the human brain being analogue or digital- https://www.quora.com/Is-the-human-brain-analog-or-digital https://www.quora.com/Is-the-human-brain-analog-or-digital The development of quantum computers could offer a huge leap forward with the development of AI.
- sehugg 11y ago
  As soon as software can convince the majority of humanity that it's conscious, the question will be academic.

derivagral 11y ago

That depends on whether you think your own mind and consciousness are different (and how?) from a very advanced computer, as mentioned in your link.

bitcointicker 11y ago
Personally, I'm not sure, which is one of the reasons I posed the question. But I seem to be being down voted for even raising it...

dboreham 11y ago

(a) it doesn't matter because these are just names for things that we can't even define succinctly for humans and (b) they're the same thing (simulated or not simulated, what's the difference??)

bitcointicker 11y ago
Eventually I'm pretty sure this will be an important question. Hundreds, or thousands of years into the future. It will matter one day. Will these artificial life forms have the same rights as humans? If there is no difference between simulated or human conciousness, then the answer is yes, they should have the same rights. But will people accept this?
ThrustVectoring 11y ago
I've got a very clear model for what consciousness is. Let's start by looking at how brains work for something that is much easier to reason about - vision. Light hits the eye, and it activates a set of neurons that are wired up to the light receptors. There's further out neurons that are set up to catch certain higher-level patterns, like edge detection or objects headed directly toward the eye. Deeper and deeper these connections go, until you get something that you subjectively experience as "seeing". It's at this point that optical illusions work - flat images on paper that are designed to make the higher level visual pattern recognizers fire. At a very high level, how parts of the brain work is by activating based off the sensory data represented by the activation of other parts of the brain. Just like detecting edges in your visual field was important in the ancestral environment, so was answering questions like "what did you do on the hunt?" with a good story. Just like there's a portion of the brain the constructs edges from lower-level visual sensation, there's a portion of the brain that constructs narratives from lots of different kinds of sensations. Furthermore, this narrative module has access to remembered narratives (since consistent stories are better ones) as another form of sense data. That, in short, is what consciousness is - the brain making sensible stories out of the sense data it encounters, and making those stories available to the brain (including the story-making module, which is why you can make stories about the qualitative experience of story-making).
- 11y ago

mesozoic 11y ago

It will of course be simulated. Without getting too metaphysical the question of whether there is anything beyond computation is at this point largely spiritual. There are also reasonable arguments that all of reality as we know it is one large simulation. http://www.simulation-argument.com/simulation.html http://www.simulation-argument.com/simulation.html

erichocean 11y ago
I'm not so sure that there's a fundamental difference between "simulating" consciousness and actually having consciousness.

ThrustVectoring 11y ago

1. What, exactly, do you mean by "consciousness"? 2. What is the difference between "having" consciousness and "simulating" it?

bitcointicker 11y ago
These seem plausible: https://en.wikipedia.org/wiki/Artificial_consciousness#Aspects_of_consciousness https://en.wikipedia.org/wiki/Artificial_consciousness#Aspec... Awareness, memory, leaning, anticipation and subjective experience. Empathy is a good example, will a computer ever "feel" empathy? Or will it be simulated empathy? Is there actually a distinction? I'm not sure.

tyingq 11y ago

I think it's a good question, and I'm surprised by the terse responses. The link you provided seems to indicate that people with reasonable credentials, either in Philosophy, or AI, struggle with it.

bitcointicker 11y ago
Indeed, it's perhaps one of the most fundamental questions in life, at least in my view. I only wish I could somehow peer into the distant future to see how things pan out. It's likely that eventually the boundaries between humans and AI will blur. I guess at some point there will be cognitive augmentation, but imagine the debate that will generate. Look how we view physical enhancement by drugs today in sport.

kazinator 11y ago

Will what be simulated, is the question. When we speak about consciousness, we are referring to something internal which we intuitively know about ourselves, but cannot directly observe in others. When we have a good artificial consciousness, the external behavior will be convincing. Then we can ask: is the visible behavior just some simulation (like a mindless script being followed) or is there an internal reality which is the same as our consciousness. Answers to the question will probably come from the convergence of two fronts: understanding better what the brain does, and correlating that with an understanding of what is going on in the machine. That is, maybe it will be shown that the kinds of states and state changes in the two are very similar (or can be mapped to each other in some way). In other words, we externalize the structures and states as much as possible and identify an equivalence. Then we can proclaim that what is going on in the machine isn't just some elaborate script; the states are actually human like: the external behavior is underpinned by apparently the same stuff that makes people tick. We will also simply be convinced on an emotional level, due to the machines leading complex lives, demonstrating traits such as shame, regret and self-loathing, as well as joy that looks genuine. There will be depressed AI's that require therapy (possibly from humans), and ones that choose to terminate themselves. People who are not computer scientists or philosophers will be convinced that the machines are conscious, by the ways in which their lives intertwine with those of the machines and the relationships they form; only a dwindling group of skeptics will remain, even long after there is no space on the field where the goal-posts can be moved any farther.

bitcointicker 11y ago
If this is indeed the way it does play out, and what you have said all sounds very plausible, it's going to raise all sorts of difficult questions for future generations. Who will decide how many of these AI's can be created? Will AI's ultimately be able to create their own offspring? We already have limited resources, this will only increase the demand :-) I see trouble ahead... as well huge opportunities for advancing our understanding of the Universe.
- mlindner 11y ago
  Many of these questions have been explored in AI fiction for many decades. Even right down to the titles, like the book "Do Androids Dream of Electric Sheep" which became "Blade Runner" for the movie version. Or "I, Robot" by Isacc Asimov.

coffeeaddicted 11y ago

The chinese room thought experiment is simply annoying because it doesn't even get the questions right. It basically asks if the hardware becomes conscious/understanding when running software - instead of asking the real question - if the software can gain the understanding/consciousness. The man in the room is just a hardware component replacing a computer processor. But the only place to look for intelligence in this experiment should be in the rule-system, not in the processor executing those rules. As those rule systems decide if the communication makes sense or not. To make a real conversion the rule-system needs flexibility to handle dynamic input and thereby creating a dynamic flow between the inside and outside of the room, but that flow is about symbol manipulation, not about the symbol manipulator. I think no one in AI ever even argued that the hardware part of a computer would gain understanding. The only good part about this thought experiment is that it makes the hard question of consciousness - how can it arise from physical phenomena - somewhat more obvious. But it's still the same question as for normal human minds and neither an argument for software systems becoming conscious nor against it.

geofft 11y ago

If you're going to apply a word to a place where it previously hasn't been meaningful to apply the word, it's worth clarifying what meaning that word now has. For instance, is Android a Linux distro? Depends why you're using the phrase "Linux distro". For some applications of that phrase, yes; for some applications no. Saying that it is, or that it isn't, doesn't tell you anything more, and doesn't help you answer those questions; you still have to answer them, so you might as well answer them directly. For instance, if you're packaging some software for a bunch of Linux distros, you probably don't want to count Android (or at least, handle it very differently from Debian or Fedora). If you're tracking fixes to kernel CVEs, you probably do want to count Android. Or, more mathematically, is 0^0 = 0 or 1? Depends how you ended up with 0^0 in the first place: if through 0^y as y approaches 0, then 0, but if through x^0 as x approaches 0, then 1. Saying "It's 1" or "It's undefined" is satisfying to the part of the human brain that likes clean answers, but it doesn't inform us. Similarly, let's not ask "what does 'consciousness' really mean", but "where do we use the term 'consciousness'." Should AIs be able to drive a car unsupervised? Should a sufficiently advanced AI have civil rights, such as the right to life? Should an AI be entrusted with political office? Or with the vote, and if so, what representation is fair? Can I give an AI ownership of a company? For the religious among us, do AIs have the ability to have human-like morality? Those are all questions worth answering, but saying "Yes, this computer has a conscious mind" or "No, it doesn't" is not really going to help those questions get answered. The Chinese room experiment is sort of not useful in that, while "the ability to learn Chinese" is a thing we expect of consciousnesses, it's not really generalizable. Nothing else in our society really depends on people's ability to learn a language; those few things that do, generally don't require the learner to be a person (e.g., you want something translated and you give it to a company).

jtolmar 11y ago

Yes, it will be possible unless you specifically construct your definition of consciousness to make it impossible. However, having a mind is unnecessary (and inefficient) for the most intelligent possible machines. Imagine extremely advanced future branches of statistics, which have yet-to-be-invented but still entirely statistical methods for discovering state spaces and causal rules, all with formal proofs of optimality. Imagine that, even if executing these methods is impossible in physical hardware, future computer science is capable of proving optimal sampling methods for a given problem and level of available computing resources. A machine that executes these algorithms is smarter than us, but does not have a mind. Will we construct machines with minds despite that? Since "to see if we can" is usually sufficient motivation for humans, I expect yes. Biologically-inspired computing techniques will probably lead to computers with minds, and I can see these computers eventually becoming smarter than humans. Will we make such machines before we invent enough of future-statistics to make a mindless machine that's smarter than us? I don't know, that's an interesting question.

jhdevos 11y ago

Of course, another question is: will the AI's consider us humans to have the equivalent of their consciousness? Or will they think "These humans don't not really possess consciousness, they just exhibit a crude biological simulation of it". (BTW: This is not just meant as a joke, nor just as instigating fear of the singularity - turning a question on its head can sometimes lead to interesting insights).

mindcrime 11y ago

Do hackernews readers think that computers will eventually have a "mind" and "consciousness" ? Or will it just be simulated? Personally? Yeah, I do think computers will eventually achieve "consciousness". I don't think of consciousness as being anything magical or terribly special to the point that a computer couldn't have it. I generally think it's just a question of a sort of recursion where the brain is "hearing" it's own internal processing as it works. When we ask questions, run simulations (day-dream) and we "see" and "hear" those things, I suspect it's just some sort of feedback loop between internal modules of the brain. I'd go so far as to suggest that some computer programs may already be conscious, and we just don't have any good way to measure/test that fact.

criddell 11y ago

Searle's Chinese Room experiment always bothered me because it feels like he is too tightly constraining "understanding". Because an airplane doesn't flap it's wings to fly like a bird does, should we say it isn't really flying?

DonaldFisk 11y ago

Every time there's a "breakthrough" in AI, this question crops up. Consciousness is poorly understood and there's no good reason to think it's a product of intelligence. In this case, all that's happened is that a game played by people can now be played well by a computer program which is designed specifically to recognize patterns. (A go position is just a 19x19 array of trits.) Computer programs reached human-level performance decades ago at chequers/draughts, and later at chess and backgammon. Those programs did not lead to a general AI.