5 ms·

That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thou

That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself. I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required.

torginus 1y ago
I'd appreciate if you tried to explain why instead of resorting to ad hominem. > I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation. What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness. I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. Which is an argument for the existence of the supernatural and deist thinking.
- CrossVR 1y ago
  > I'd appreciate if you tried to explain why instead of resorting to ad hominem. It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious. I'm not saying you personally believe that and therefore your arguments are invalid. > Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact. > I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions. Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem? (Also, I apologize for using two accounts, I'm not attempting to sock puppet)
  - torginus 1y ago
    What are his arguments then? >Searle argues that, without "understanding" (or "intentionality"), we cannot describe what the machine is doing as "thinking" and, since it does not think, it does not have a "mind" in the normal sense of the word. This is the only sentence that seems to be pointing to what constitutes the specialness of humans, and the terms of 'understanding' and 'intentionality' are in air quotes so who knows? This sounds like the archetypical no true scotsman fallacy. In mathematical analysis, if we conclude that the difference between 2 numbers is smaller than any arbitrary number we can pick, those 2 numbers must be the same. In engineering, we can reduce the claim to 'any difference large about to care about' Likewise if the difference between a real human brain and an arbitrarily sophisticated Chinese Room brain is arbitrarily small, they are the same. If our limited understanding of physics and engineering makes the practical difference not zero, this essentially becomes a bit of a somewhat magical 'superscience' argument claiming we can't simulate the real world to a good enough resolution that the meaningful differences between our 'consciousness simulator' and the thing itself disappear - which is an extraordinary claim.
    - CrossVR 1y ago
      > What are his arguments then? They're in the "Complete Argument" section of the article. > This sounds like the archetypical no true scotsman fallacy. I get what you're trying to say, but he is not arguing only a true Scotsman is capable of thought. He is arguing that our current machines lack the required "causal powers" for thought. Powers that he doesn't prescribe to only a true Scotsman, though maybe we should try adding bagpipes to our AI just to be sure...
      - torginus 1y ago
        Thanks, but that makes his arguments even less valid. He argues that computer programs only manipulate symbols and thus have no semantic understanding. But that's not true - many programs, like compilers that existed back when the argument was made, had semantic understanding of the code (in a limited way, but they did have some understanding about what the program did). LLMs in contrast have a very rich semantic understanding of the text they parse - their tensor representations encode a lot about each token, or you can just ask them about anything - they might not be human level at reading subtext, but they're not horrible either.
- dahart 1y ago
  > I cannot help but conclude that Searle argues that ‘understanding’ is only something that humans can do, which means… Regardless of whether Searle is right or wrong, you’ve jumped to conclusions and are misunderstanding his argument and making further assumptions based on your misunderstanding. Your argument is also ad-hominem by accusing people of believing things they don’t believe. Maybe it would be prudent to read some of the good critiques of Searle before trying to litigate it rapidly and sloppily on HN. The randomness stuff is very straw man, definitely not a good argument, best to drop it. Today’s LLMs are deterministic, not random. Pseudorandom sequences come in different varieties, but they model some properties of randomness, not all of them. The functioning of today’s neural networks, both training and inference, is exactly a book of static rules, despite their use of pseudorandom sequences. In case you missed it in the WP article, most of the field of cognitive science thinks Searle is wrong. However, they’re largely not critiquing him for using metaphysics, because that’s not his argument. He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. That much is certainly true. Whether there’s a difference in theory is unproven. But today currently there absolutely is a difference in practice, nobody has ever simulated the real world or a human brain using deterministic computation.
  - torginus 1y ago
    If scientific consensus is that he's wrong why is he being constantly brought up and defended - am I not right to call them out then? Nobody brings up that light travels through the aether, that diseases are caused by bad humors etc. - is it not right to call out people for stating theory that's believed to be false? >The randomness stuff is very straw man, And a direct response to what armada651 wrote: >I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. > He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. Once again the argument here changed from 'computers which only manipulate symbols cannot create consciousness' to 'we don't have the algorithm for consiousness yet'. He might have successfully argued against the expert systems of his time - and true, mechanistic attempts at language translation have largely failed - but that doesn't extend to modern LLMs (and pre LLM AI) or even statistical methods.

torginus 1y ago

I'd appreciate if you tried to explain why instead of resorting to ad hominem. > I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation. What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness. I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. Which is an argument for the existence of the supernatural and deist thinking.

CrossVR 1y ago
> I'd appreciate if you tried to explain why instead of resorting to ad hominem. It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious. I'm not saying you personally believe that and therefore your arguments are invalid. > Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact. > I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions. Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem? (Also, I apologize for using two accounts, I'm not attempting to sock puppet)
- torginus 1y ago
  What are his arguments then? >Searle argues that, without "understanding" (or "intentionality"), we cannot describe what the machine is doing as "thinking" and, since it does not think, it does not have a "mind" in the normal sense of the word. This is the only sentence that seems to be pointing to what constitutes the specialness of humans, and the terms of 'understanding' and 'intentionality' are in air quotes so who knows? This sounds like the archetypical no true scotsman fallacy. In mathematical analysis, if we conclude that the difference between 2 numbers is smaller than any arbitrary number we can pick, those 2 numbers must be the same. In engineering, we can reduce the claim to 'any difference large about to care about' Likewise if the difference between a real human brain and an arbitrarily sophisticated Chinese Room brain is arbitrarily small, they are the same. If our limited understanding of physics and engineering makes the practical difference not zero, this essentially becomes a bit of a somewhat magical 'superscience' argument claiming we can't simulate the real world to a good enough resolution that the meaningful differences between our 'consciousness simulator' and the thing itself disappear - which is an extraordinary claim.
  - CrossVR 1y ago
    > What are his arguments then? They're in the "Complete Argument" section of the article. > This sounds like the archetypical no true scotsman fallacy. I get what you're trying to say, but he is not arguing only a true Scotsman is capable of thought. He is arguing that our current machines lack the required "causal powers" for thought. Powers that he doesn't prescribe to only a true Scotsman, though maybe we should try adding bagpipes to our AI just to be sure...
    - torginus 1y ago
      Thanks, but that makes his arguments even less valid. He argues that computer programs only manipulate symbols and thus have no semantic understanding. But that's not true - many programs, like compilers that existed back when the argument was made, had semantic understanding of the code (in a limited way, but they did have some understanding about what the program did). LLMs in contrast have a very rich semantic understanding of the text they parse - their tensor representations encode a lot about each token, or you can just ask them about anything - they might not be human level at reading subtext, but they're not horrible either.
dahart 1y ago
> I cannot help but conclude that Searle argues that ‘understanding’ is only something that humans can do, which means… Regardless of whether Searle is right or wrong, you’ve jumped to conclusions and are misunderstanding his argument and making further assumptions based on your misunderstanding. Your argument is also ad-hominem by accusing people of believing things they don’t believe. Maybe it would be prudent to read some of the good critiques of Searle before trying to litigate it rapidly and sloppily on HN. The randomness stuff is very straw man, definitely not a good argument, best to drop it. Today’s LLMs are deterministic, not random. Pseudorandom sequences come in different varieties, but they model some properties of randomness, not all of them. The functioning of today’s neural networks, both training and inference, is exactly a book of static rules, despite their use of pseudorandom sequences. In case you missed it in the WP article, most of the field of cognitive science thinks Searle is wrong. However, they’re largely not critiquing him for using metaphysics, because that’s not his argument. He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. That much is certainly true. Whether there’s a difference in theory is unproven. But today currently there absolutely is a difference in practice, nobody has ever simulated the real world or a human brain using deterministic computation.
- torginus 1y ago
  If scientific consensus is that he's wrong why is he being constantly brought up and defended - am I not right to call them out then? Nobody brings up that light travels through the aether, that diseases are caused by bad humors etc. - is it not right to call out people for stating theory that's believed to be false? >The randomness stuff is very straw man, And a direct response to what armada651 wrote: >I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. > He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. Once again the argument here changed from 'computers which only manipulate symbols cannot create consciousness' to 'we don't have the algorithm for consiousness yet'. He might have successfully argued against the expert systems of his time - and true, mechanistic attempts at language translation have largely failed - but that doesn't extend to modern LLMs (and pre LLM AI) or even statistical methods.