6 ms·

To build on your thought experiment: > So the question is, this simulated brain, which is behaving very similarly indeed to a biological brain: does it have co

by adwn 7y ago

To build on your thought experiment: > So the question is, this simulated brain, which is behaving very similarly indeed to a biological brain: does it have consciousness? Does it feel? If you answer "yes" to this question, what happens if you split the computations of that simulated brain over a million people equipped with pen and paper? Where is its consciousness happening now? I have yet to hear a satisfying answer from anyone who believes that an AGI will automatically be conscious. But then again, I have yet to hear a satisfying explanation of consciousness from anyone...

dbqpdb 7y ago
If you haven't already, I highly suggest you read Permutation City by Greg Egan, these kinds of questions are explored in great depth.
- rapnie 7y ago
  Yes, that is indeed a great read!
mannykannot 7y ago
Ah, Searle's Chinese Room, for which the widely accepted response is the system reply: the system as a whole, and not one element of it, is conscious. Consider performing the algorithm of Alpha Go Zero through a million people equipped with pen and paper. One can hardly argue against this being possible in principle, but what is playing the game? This is not proof that AGI will automatically be conscious, or even just that some sort of conscious machine intelligence is possible, it is merely intended to suggest that Searle's line of argument does not show its impossibility.
- DonaldFisk 7y ago
  There are two distinct meanings of consciousness being used here: my own experience of consciousness in me (it's always first person), and the conclusions people make about consciousness in other people or animals - that they're conscious because they're like them in important ways. The Chinese Room argument is confusing because it conflates intelligence (understanding Chinese) with consciousness (knowing what it's like to understand Chinese). So I simplified the problem, by replacing the ability to understand Chinese with the ability to see in colour: http://www.fmjlang.co.uk/blog/ChineseRoom.html http://www.fmjlang.co.uk/blog/ChineseRoom.html While the simplified system is able to distinguish colours, how does it have the first person experience of seeing in colour?
  - mannykannot 7y ago
    That is a good argument, though you may not agree with my reason for saying so. Clearly, the system as a whole can discriminate a wide range of colors, despite the fact that no single component, including the person who is executing the process, can do so. This is the essence of the system reply, and I think this thought experiment illustrates it well. There is a sense in which the system has an experience (necessarily first-'person'), but it is nothing like what we experience on seeing color. It is no more or less an experience than an ordinary color meter would have, and it lacks any awareness or concept of having an experience, no memory of it, no recognition of a similarity to previous experiences or of it being unlike any previous experience, no ability to imagine such experiences, no expectation of future experiences, and no emotional reaction to it ('my favorite color!') It is an experience only in the weakest sense possible.
  - Faark 7y ago
    I'm not sure I understand what your link has to do with consciousness. Yes, people experience the world differently. Someone with Tetrachromacy will see the world different than us, but that hardly invalidates our much less capable experience. Perception organs differ somewhat between individuals. I'd see little reason to make a distinction between those organic we are born with and those humans managed to invent. So yes, the guy in your room can experience colors by using the sheets (though in a different and much less capable way than most of us, thus him not wanting to equate this with our common understanding of experiencing color), and is color blind without. And I kind of get deaf when plugging my ears, though that experience is vastly different from someone who never had hearing to begin with. That doesn't answer what it is like "seeing red" and "feeling blue". I don't know what it's like for you. I can only look at your behavior, see it being similar to mine under the same circumstances and thus assume your internal state to be similar. Until your behavior tells me it being different... like someone blind running into a wall or someone approaching a chasm with no signs of fear. ps: had to google "feel blue", as it meaning "depressed" isn't really a thing in my native language. Translating (color based) idioms is actually kind of telling... it seems only possible directly if both cultures had shared aspects / experiences. So yeah, while conflating consciousness with intelligence doesn't make a lot of sense to me. But I'm not sure equating it with perception is getting us anywhere better (hope idiom didn't get lost in translation).
- naasking 7y ago
  > Consider performing the algorithm of Alpha Go Zero through a million people equipped with pen and paper. One can hardly argue against this being possible in principle, but what is playing the game? I prefer a simpler analogy: what step in a sorting algorithm actually does the sorting? The question is actually nonsense, because every step is necessary, and removing any single step will break the sorting property for some inputs. When you have all the relevant information, it's clear that the original question itself is ill-posed. We lack some of that information when it comes to.consciousness, hence why the Chinese Room is convincing to some.
yeellow 7y ago
My phone is black, but looking at individuals atoms where is the color? You are right that we don't have a good (any?) definition of consciousness therefore it is difficult to properly ask if Chinese room has it. After reading and talking with people it seems that consciousness is something one can only claim to have (like a subjective feeling), but can't really manifest (I can only assume you are conscious because I believe I am, and you behave similarly enough).
gglon 7y ago
> what happens if you split the computations of that simulated brain over a million people equipped with pen and paper? Where is its consciousness happening now? The thing with information is that it is not as localized as other physical quantities like spin, mass, or even energy. Conversely if the information were localized we would call it lack of thereof. Information between systems mean correlation between them. Same goes with information processing where changes in one system corresponds to changes in the other. It happens in between.
slacka 7y ago
million? There are over 100 trillion neural connections in the human brain. And look up artificial vs biological neuron. With sheer number and added complexity of biological neurons, there's likely not enough atoms in the Universe to provide enough ink and paper to run this simulation. Also, you run into the problem of time slices. One step in a simulation of my brain is not conscious. There must be some self-reference going on in there that give us this illusion over time that I doubt a pen and paper model could capture. I'm of the school of thought that a turing machine, let alone pen and paper, might not even be capable of this simulation. For example, it's possible that there are quantum effects going on in biological neurons that touring machines are not equipped to simulate.
- turk73 7y ago
  I was working on a response same as yours. Only thing I can offer is that maybe we can create consciousness without having to "simulate" all those neurons, as in, there is some more reasonable sampling that will give a reasonable approximation. I do think it is self-referential and that is how consciousness emerges. I disagree that there are quantum mechanical effects, but what do I know? I do kind of feel like I'm in more than one place at the same time right now. It's "Turing" not Touring, btw. And no, the brain is not a Turing machine.
- simiones 7y ago
  Quantum computers can be perfectly simulated by classical computers (but only inefficiently, as far as we know). So at best, you could claim that computation in the brain may involve quantum effects and therefore it would not be tractable to simulate it on a classical computer. But a quantum computer is still a Turing machine. Of course, we can not yet rule out the possibility that there exist more powerful models of computation than Turing machines. We just know quantum effects don't lead there.
hackinthebochs 7y ago
>Where is its consciousness happening now? Our brain organization can be described by its various information cascades. But if conscious is implementing the right kind of information cascade, then a brain simulated by a million people would be conscious. The "where" of conscious is the same question as the "where" of informational relationships. But this isn't a well-formed question. There is no one location where some sequence of bits and its proper interpretation is stored. It's distributed among the components involved in the information cascade. The "where" of consciousness is the same.
brazzy 7y ago
My take: consciousness is an emergent property of the system as a whole as it becomes sufficiently complex. Yes, that million people (you'd probably need more) working with pen and paper create, in combination, an additional new consciousness, albeit an absurdly slow-thinking one.
- appstateguy 7y ago
  What differs this scenario from individual neurons firing together in order to make the human mind? Can you keep taking that line of thinking down to the subatomic properties of the neurons themselves? I'm guessing this is the argument for panpsychism. The emergent property of human consciousness could be seen as the culmination of all the individual parts themselves have some form of primitive consciousness.
  - EForEndeavour 7y ago
    Isn't that argument roughly as useless as attributing the ability of a watch to show time down to each individual mechanical component? It's technically true, but not very useful to elucidate what actually makes the watch a watch. For example, it sounds kind of silly to say that a single watch gear has some form of primitive time-keeping ability, or has some form of primitive ability to tell me whether I'm running late. The universe is rife with examples of emergent complexity in which the whole is greater than the sum of its parts.
    - Nasrudith 7y ago
      Well it is still useful but not to you directly. The distinction being that the right intermediary could make use of it. If you had a physics model it could describe it perfectly but it would likely involve a very large group of matricies and linear algebra constraints. You technically could work through every equation given a large set but examining the watch yourself would likely be quicker, if not learning to be a sufficiently good watchmaker. However if you assemble a computer model to handle every component's interaction individuallt and a physics engine to do number crunching you could get a working mathematical model of the clock.
state_less 7y ago
To take it a step further, let's assume we simulate the whole universe. We figured out all the rules for interaction so that there is nothing left over or left out of the simulation. All the rules and state are encoded. Do you think beings in this second run of the universe are conscious? Or do you think that somehow something has been left out of the simulation?
- TuringTest 7y ago
  How could I tell if there's consciousness in that simulated universe, when I can't tell whether other beings in this original universe are conscious?
  - state_less 7y ago
    If you replicate the universe, then the properties would hold from the reference and simulation. It shouldn't matter if we use information vs. whatever the reference implementation is using. We do this sort of thing when we simulate known systems, like a computer running in software. So if there is conscious beings in the universe, it'll be in the simulation. You hold that you're conscious, so then I'd hope you'd agree the simulation has at least one conscious being in it. I don't know why some folks consciousness test only seems to work for themselves. I've been convinced by the simple argument by induction: I am an example of an animal. I have consciousness, so others likely do too.
    - TuringTest 7y ago
      Induction is not proof :-) I'm also convinced, but primarily by Turing's argument ad politeness: "Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks".[1] I haven't really found a compelling rational argument to fully reject solipsism. Though I agree that looking for yourself in a perfect simulation is a clever hack. :-) [1] https://books.google.com/books?id=7pFHBAAAQBAJ&lpg=PT51&ots=rCScTqp0va&dq=turing%20conscious%20politeness&pg=PT51#v=onepage&q=turing%20conscious%20politeness&f=false https://books.google.com/books?id=7pFHBAAAQBAJ&lpg=PT51&ots=...
codeulike 7y ago
I have yet to hear a satisfying answer from anyone ... Here's Dennett on this very subject, about 20 years ago. In Consciousness Explained he imagines what a Turing Test conversation between a human 'judge' and a 'Chinese Room' might look like: Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.” CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss. J: Fair enough but I told you the joke because I want you to explain it to me. CR: Boring! You should never explain jokes. J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”? CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough? Dennett: “The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all.” His point (if I need to re-explain it?) is that the Chinese Room argument is superficially compelling. But if you actually follow through and imagine the Chinese Room doing the things that the thought experiment defines it to be doing, then you get a picture of an extraordinarily complicated system, and the nuts and bolts of exactly what that system is running on fall away into irrelevance.
BurningFrog 7y ago
> I have yet to hear a satisfying explanation of consciousness from anyone... So it's not surprising if we struggle to answer "Can machines be X?", when X is undefined.
Veedrac 7y ago
Alice: We can agree the brain is conscious, yes? Bob: Of course. Alice: But what if you scaled up all the neurons in the brain a factor of 2? Would it still be conscious? Bob: Sure, if it's the same system running the same way. There's no reason to expect it to matter. Alice: Then if it was a factor 4 larger? Bob: Absolutely. Human brains are sized as they are as a matter of evolutionary coincidence. Bird neurons are much more compact than ours, and still manage much of the same work. Alice: Right, but if every neuron also contained chloroplasts? Bob: Which did what? There isn't much light there. Alice: Nothing, they're completely ancillary. Bob: Then yes, evidently it's still the same system... at what point does this become a hard question? Alice: Just hold on a few orders of magnitude...
- imoldtoo 7y ago
  This scaling argument only works if the consituent molecules, atoms and particles also scale proportionally. If you scale the system but keep one end tied down it isn't the same system.
  - FisDugthop 7y ago
    Presumably the grandparent is thinking of organisms like Pando [0]. [0] https://en.wikipedia.org/wiki/Pando_(tree) https://en.wikipedia.org/wiki/Pando_(tree)
    - imoldtoo 7y ago
      Yes, that makes more sense. ta
  - Veedrac 7y ago
    No, it holds much more strongly than that. The idea that consciousness can only arise in neurons shaped exactly like human neurons—and not, say, slightly larger neurons like exist in many other animals, or neurons with slightly different energy-carrying chemicals, or whatever—is implausible, because evolution isn't goal-directed. Evolution didn't find the one true physical layout that leads to conscious experience, because evolutionary strategies fundamentally can't solve those sorts of problems.

by adwn 7y ago

dbqpdb 7y ago

If you haven't already, I highly suggest you read Permutation City by Greg Egan, these kinds of questions are explored in great depth.

rapnie 7y ago
Yes, that is indeed a great read!

mannykannot 7y ago

Ah, Searle's Chinese Room, for which the widely accepted response is the system reply: the system as a whole, and not one element of it, is conscious. Consider performing the algorithm of Alpha Go Zero through a million people equipped with pen and paper. One can hardly argue against this being possible in principle, but what is playing the game? This is not proof that AGI will automatically be conscious, or even just that some sort of conscious machine intelligence is possible, it is merely intended to suggest that Searle's line of argument does not show its impossibility.

DonaldFisk 7y ago
There are two distinct meanings of consciousness being used here: my own experience of consciousness in me (it's always first person), and the conclusions people make about consciousness in other people or animals - that they're conscious because they're like them in important ways. The Chinese Room argument is confusing because it conflates intelligence (understanding Chinese) with consciousness (knowing what it's like to understand Chinese). So I simplified the problem, by replacing the ability to understand Chinese with the ability to see in colour: http://www.fmjlang.co.uk/blog/ChineseRoom.html http://www.fmjlang.co.uk/blog/ChineseRoom.html While the simplified system is able to distinguish colours, how does it have the first person experience of seeing in colour?
- mannykannot 7y ago
  That is a good argument, though you may not agree with my reason for saying so. Clearly, the system as a whole can discriminate a wide range of colors, despite the fact that no single component, including the person who is executing the process, can do so. This is the essence of the system reply, and I think this thought experiment illustrates it well. There is a sense in which the system has an experience (necessarily first-'person'), but it is nothing like what we experience on seeing color. It is no more or less an experience than an ordinary color meter would have, and it lacks any awareness or concept of having an experience, no memory of it, no recognition of a similarity to previous experiences or of it being unlike any previous experience, no ability to imagine such experiences, no expectation of future experiences, and no emotional reaction to it ('my favorite color!') It is an experience only in the weakest sense possible.
- Faark 7y ago
  I'm not sure I understand what your link has to do with consciousness. Yes, people experience the world differently. Someone with Tetrachromacy will see the world different than us, but that hardly invalidates our much less capable experience. Perception organs differ somewhat between individuals. I'd see little reason to make a distinction between those organic we are born with and those humans managed to invent. So yes, the guy in your room can experience colors by using the sheets (though in a different and much less capable way than most of us, thus him not wanting to equate this with our common understanding of experiencing color), and is color blind without. And I kind of get deaf when plugging my ears, though that experience is vastly different from someone who never had hearing to begin with. That doesn't answer what it is like "seeing red" and "feeling blue". I don't know what it's like for you. I can only look at your behavior, see it being similar to mine under the same circumstances and thus assume your internal state to be similar. Until your behavior tells me it being different... like someone blind running into a wall or someone approaching a chasm with no signs of fear. ps: had to google "feel blue", as it meaning "depressed" isn't really a thing in my native language. Translating (color based) idioms is actually kind of telling... it seems only possible directly if both cultures had shared aspects / experiences. So yeah, while conflating consciousness with intelligence doesn't make a lot of sense to me. But I'm not sure equating it with perception is getting us anywhere better (hope idiom didn't get lost in translation).
naasking 7y ago
> Consider performing the algorithm of Alpha Go Zero through a million people equipped with pen and paper. One can hardly argue against this being possible in principle, but what is playing the game? I prefer a simpler analogy: what step in a sorting algorithm actually does the sorting? The question is actually nonsense, because every step is necessary, and removing any single step will break the sorting property for some inputs. When you have all the relevant information, it's clear that the original question itself is ill-posed. We lack some of that information when it comes to.consciousness, hence why the Chinese Room is convincing to some.

yeellow 7y ago

My phone is black, but looking at individuals atoms where is the color? You are right that we don't have a good (any?) definition of consciousness therefore it is difficult to properly ask if Chinese room has it. After reading and talking with people it seems that consciousness is something one can only claim to have (like a subjective feeling), but can't really manifest (I can only assume you are conscious because I believe I am, and you behave similarly enough).

gglon 7y ago

> what happens if you split the computations of that simulated brain over a million people equipped with pen and paper? Where is its consciousness happening now? The thing with information is that it is not as localized as other physical quantities like spin, mass, or even energy. Conversely if the information were localized we would call it lack of thereof. Information between systems mean correlation between them. Same goes with information processing where changes in one system corresponds to changes in the other. It happens in between.

slacka 7y ago

million? There are over 100 trillion neural connections in the human brain. And look up artificial vs biological neuron. With sheer number and added complexity of biological neurons, there's likely not enough atoms in the Universe to provide enough ink and paper to run this simulation. Also, you run into the problem of time slices. One step in a simulation of my brain is not conscious. There must be some self-reference going on in there that give us this illusion over time that I doubt a pen and paper model could capture. I'm of the school of thought that a turing machine, let alone pen and paper, might not even be capable of this simulation. For example, it's possible that there are quantum effects going on in biological neurons that touring machines are not equipped to simulate.

turk73 7y ago
I was working on a response same as yours. Only thing I can offer is that maybe we can create consciousness without having to "simulate" all those neurons, as in, there is some more reasonable sampling that will give a reasonable approximation. I do think it is self-referential and that is how consciousness emerges. I disagree that there are quantum mechanical effects, but what do I know? I do kind of feel like I'm in more than one place at the same time right now. It's "Turing" not Touring, btw. And no, the brain is not a Turing machine.
simiones 7y ago
Quantum computers can be perfectly simulated by classical computers (but only inefficiently, as far as we know). So at best, you could claim that computation in the brain may involve quantum effects and therefore it would not be tractable to simulate it on a classical computer. But a quantum computer is still a Turing machine. Of course, we can not yet rule out the possibility that there exist more powerful models of computation than Turing machines. We just know quantum effects don't lead there.

hackinthebochs 7y ago

>Where is its consciousness happening now? Our brain organization can be described by its various information cascades. But if conscious is implementing the right kind of information cascade, then a brain simulated by a million people would be conscious. The "where" of conscious is the same question as the "where" of informational relationships. But this isn't a well-formed question. There is no one location where some sequence of bits and its proper interpretation is stored. It's distributed among the components involved in the information cascade. The "where" of consciousness is the same.

brazzy 7y ago

My take: consciousness is an emergent property of the system as a whole as it becomes sufficiently complex. Yes, that million people (you'd probably need more) working with pen and paper create, in combination, an additional new consciousness, albeit an absurdly slow-thinking one.

appstateguy 7y ago
What differs this scenario from individual neurons firing together in order to make the human mind? Can you keep taking that line of thinking down to the subatomic properties of the neurons themselves? I'm guessing this is the argument for panpsychism. The emergent property of human consciousness could be seen as the culmination of all the individual parts themselves have some form of primitive consciousness.
- EForEndeavour 7y ago
  Isn't that argument roughly as useless as attributing the ability of a watch to show time down to each individual mechanical component? It's technically true, but not very useful to elucidate what actually makes the watch a watch. For example, it sounds kind of silly to say that a single watch gear has some form of primitive time-keeping ability, or has some form of primitive ability to tell me whether I'm running late. The universe is rife with examples of emergent complexity in which the whole is greater than the sum of its parts.
  - Nasrudith 7y ago
    Well it is still useful but not to you directly. The distinction being that the right intermediary could make use of it. If you had a physics model it could describe it perfectly but it would likely involve a very large group of matricies and linear algebra constraints. You technically could work through every equation given a large set but examining the watch yourself would likely be quicker, if not learning to be a sufficiently good watchmaker. However if you assemble a computer model to handle every component's interaction individuallt and a physics engine to do number crunching you could get a working mathematical model of the clock.

state_less 7y ago

To take it a step further, let's assume we simulate the whole universe. We figured out all the rules for interaction so that there is nothing left over or left out of the simulation. All the rules and state are encoded. Do you think beings in this second run of the universe are conscious? Or do you think that somehow something has been left out of the simulation?

TuringTest 7y ago
How could I tell if there's consciousness in that simulated universe, when I can't tell whether other beings in this original universe are conscious?
- state_less 7y ago
  If you replicate the universe, then the properties would hold from the reference and simulation. It shouldn't matter if we use information vs. whatever the reference implementation is using. We do this sort of thing when we simulate known systems, like a computer running in software. So if there is conscious beings in the universe, it'll be in the simulation. You hold that you're conscious, so then I'd hope you'd agree the simulation has at least one conscious being in it. I don't know why some folks consciousness test only seems to work for themselves. I've been convinced by the simple argument by induction: I am an example of an animal. I have consciousness, so others likely do too.
  - TuringTest 7y ago
    Induction is not proof :-) I'm also convinced, but primarily by Turing's argument ad politeness: "Instead of arguing continually over this point it is usual to have the polite convention that everyone thinks".[1] I haven't really found a compelling rational argument to fully reject solipsism. Though I agree that looking for yourself in a perfect simulation is a clever hack. :-) [1] https://books.google.com/books?id=7pFHBAAAQBAJ&lpg=PT51&ots=rCScTqp0va&dq=turing%20conscious%20politeness&pg=PT51#v=onepage&q=turing%20conscious%20politeness&f=false https://books.google.com/books?id=7pFHBAAAQBAJ&lpg=PT51&ots=...

codeulike 7y ago

I have yet to hear a satisfying answer from anyone ... Here's Dennett on this very subject, about 20 years ago. In Consciousness Explained he imagines what a Turing Test conversation between a human 'judge' and a 'Chinese Room' might look like: Judge: Did you hear about the Irishman who found a magic lamp? When he rubbed it a genie appeared and granted him three wishes. “I’ll have a pint of Guiness!” the Irishman replied and immediately it appeared. The Irishman eagerly set to sipping and then gulping, but the level of Guiness in the glass was always magically restored. After a while the genie became impatient. “Well, what about your second wish?” he asked. Replied the Irishman between gulps, “Oh well, I guess I’ll have another one of these.” CHINESE ROOM: Very funny. No, I hadn’t heard it– but you know I find ethnic jokes in bad taste. I laughed in spite of myself, but really, I think you should find other topics for us to discuss. J: Fair enough but I told you the joke because I want you to explain it to me. CR: Boring! You should never explain jokes. J: Nevertheless, this is my test question. Can you explain to me how and why the joke “works”? CR: If you insist. You see, it depends on the assumption that the magically refilling glass will go on refilling forever, so the Irishman has all the stout he can ever drink. So he hardly has a reason for wanting a duplicate but he is so stupid (that’s the part I object to) or so besotted by the alcohol that he doesn’t recognize this, and so, unthinkingly endorsing his delight with his first wish come true, he asks for seconds. These background assumptions aren’t true, of course, but just part of the ambient lore of joke-telling, in which we suspend our disbelief in magic and so forth. By the way we could imagine a somewhat labored continuation in which the Irishman turned out to be “right” in his second wish after all, perhaps he’s planning to throw a big party and one glass won’t refill fast enough to satisfy all his thirsty guests (and it’s no use saving it up in advance– we all know how stale stout loses its taste). We tend not to think of such complications which is part of the explanation of why jokes work. Is that enough? Dennett: “The fact is that any program that could actually hold up its end in the conversation depicted would have to be an extraordinary supple, sophisticated, and multilayered system, brimming with “world knowledge” and meta-knowledge and meta-meta-knowledge about its own responses, the likely responses of its interlocutor, and much, much more…. Maybe the billions of actions of all those highly structured parts produce genuine understanding in the system after all.” His point (if I need to re-explain it?) is that the Chinese Room argument is superficially compelling. But if you actually follow through and imagine the Chinese Room doing the things that the thought experiment defines it to be doing, then you get a picture of an extraordinarily complicated system, and the nuts and bolts of exactly what that system is running on fall away into irrelevance.

BurningFrog 7y ago

> I have yet to hear a satisfying explanation of consciousness from anyone... So it's not surprising if we struggle to answer "Can machines be X?", when X is undefined.

Veedrac 7y ago

Alice: We can agree the brain is conscious, yes? Bob: Of course. Alice: But what if you scaled up all the neurons in the brain a factor of 2? Would it still be conscious? Bob: Sure, if it's the same system running the same way. There's no reason to expect it to matter. Alice: Then if it was a factor 4 larger? Bob: Absolutely. Human brains are sized as they are as a matter of evolutionary coincidence. Bird neurons are much more compact than ours, and still manage much of the same work. Alice: Right, but if every neuron also contained chloroplasts? Bob: Which did what? There isn't much light there. Alice: Nothing, they're completely ancillary. Bob: Then yes, evidently it's still the same system... at what point does this become a hard question? Alice: Just hold on a few orders of magnitude...

imoldtoo 7y ago
This scaling argument only works if the consituent molecules, atoms and particles also scale proportionally. If you scale the system but keep one end tied down it isn't the same system.
- FisDugthop 7y ago
  Presumably the grandparent is thinking of organisms like Pando [0]. [0] https://en.wikipedia.org/wiki/Pando_(tree) https://en.wikipedia.org/wiki/Pando_(tree)
  - imoldtoo 7y ago
    Yes, that makes more sense. ta
- Veedrac 7y ago
  No, it holds much more strongly than that. The idea that consciousness can only arise in neurons shaped exactly like human neurons—and not, say, slightly larger neurons like exist in many other animals, or neurons with slightly different energy-carrying chemicals, or whatever—is implausible, because evolution isn't goal-directed. Evolution didn't find the one true physical layout that leads to conscious experience, because evolutionary strategies fundamentally can't solve those sorts of problems.