4 ms·

I don't know if AGI will emerge from LLM, but I'm always reminded of the Chinese room thought experiment. With billions thrown at the idea it will certainly be

I don't know if AGI will emerge from LLM, but I'm always reminded of the Chinese room thought experiment. With billions thrown at the idea it will certainly be the ultimate answer as to whether true understanding can emerge from a large enough dictionary.

torginus 1y ago
Please stop refering to the Chinese Room - it's just magical/deist thinking in disguise. It postulates that humans have way of 'understanding' things that is impossible to replicate mechanically. The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. (That doesn't mean LLMs are or will be AGI, its just this argument is tautological and meaningless)
- simianparrot 1y ago
  It is still relevant because it hasn’t been disproven yet. So far all computer programs are Chinese Rooms, LLM’s included.
  - IanCal 1y ago
    If you’re talking about it being proven or disproven you’re misunderstanding the point of the thought experiment.
- armada651 1y ago
  That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself. I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required.
  - torginus 1y ago
    I'd appreciate if you tried to explain why instead of resorting to ad hominem. > I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation. What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness. I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. Which is an argument for the existence of the supernatural and deist thinking.
    - CrossVR 1y ago
      > I'd appreciate if you tried to explain why instead of resorting to ad hominem. It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious. I'm not saying you personally believe that and therefore your arguments are invalid. > Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact. > I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions. Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem? (Also, I apologize for using two accounts, I'm not attempting to sock puppet)
- welferkj 1y ago
  The human way of understanding things can be replicated mechanically, because it is mechanical in nature. The contents of your skull are an existence proof of AGI.
  - armada651 1y ago
    The contents of my skull are only a proof for AGI if your mechanical machine replicates all its processes. It's not a question about whether a machine can reproduce that, it's a question about whether we have given our current machines all the tools it needs to do that.
    - torginus 1y ago
      The theory of special relativity does not say 'you can't exceed the speed of light(unless you have a really big rocket)'. It presents a theoretical limit. Likewise the Chinese room doesn't state that consciousness is an intractable engineering problem, but an impossibility. But the way Searle formulates his argument, by not defining what consciousness is, he essentially gives himself enough wiggle room to be always right - he's essentially making the 'No True Scotsman' fallacy.
  - hiatus 1y ago
    The A stands for artificial.
- herculity275 1y ago
  "Please stop referring to this thought experiment because it has possible interpretations I don't personally agree with"
  - torginus 1y ago
    Please give me an interpretation that is both correct an meaningful (as in possible to disprove)
- globnomulous 1y ago
  > The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. That's one possibility. The other is that your pomposity and dismissiveness towards the entire field of philosophy speaks volumes on how little you know about either philosophical arguments in general or this philosophical argument in particular.
  - torginus 1y ago
    Another ad hominem, I'd like you to refute my claim that Searle's argument is essentially 100% magical thinking. And yes, if for example, medicine would be no worse at curing cancer than it is today, yet doctors asserted that crystal healing is a serious study, that would reflect badly on the field at large, despite most of it being sound.
    - dahart 1y ago
      Searle refutes your claim that there’s magical thinking. “Searle does not disagree with the notion that machines can have consciousness and understanding, because, as he writes, "we are precisely such machines". Searle holds that the brain is, in fact, a machine, but that the brain gives rise to consciousness and understanding using specific machinery.”
      - torginus 1y ago
        But the core of the original argument is that programs only manipulate symbols and consciousness can never arise just through symbol manipulation - which here then becomes 'we have not discovered the algorithms' for consciousness yet. It's just a contradiction.
        dahart 1y ago
        When you say something that contradicts his statements, it doesn’t mean he’s wrong, it most likely means you haven’t understood or interpreted his argument correctly. The Wikipedia page you linked to doesn’t use the word “algorithm”, so the source of the contradiction you imagine might be you. Searle says he thinks humans are biological machines, so your argument should withstand that hypothesis rather than dismiss it.

torginus 1y ago

Please stop refering to the Chinese Room - it's just magical/deist thinking in disguise. It postulates that humans have way of 'understanding' things that is impossible to replicate mechanically. The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. (That doesn't mean LLMs are or will be AGI, its just this argument is tautological and meaningless)

simianparrot 1y ago
It is still relevant because it hasn’t been disproven yet. So far all computer programs are Chinese Rooms, LLM’s included.
- IanCal 1y ago
  If you’re talking about it being proven or disproven you’re misunderstanding the point of the thought experiment.
armada651 1y ago
That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself. I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required.
- torginus 1y ago
  I'd appreciate if you tried to explain why instead of resorting to ad hominem. > I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data. Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation. What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness. I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. Which is an argument for the existence of the supernatural and deist thinking.
  - CrossVR 1y ago
    > I'd appreciate if you tried to explain why instead of resorting to ad hominem. It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious. I'm not saying you personally believe that and therefore your arguments are invalid. > Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing. The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact. > I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not. In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions. Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem? (Also, I apologize for using two accounts, I'm not attempting to sock puppet)
welferkj 1y ago
The human way of understanding things can be replicated mechanically, because it is mechanical in nature. The contents of your skull are an existence proof of AGI.
- armada651 1y ago
  The contents of my skull are only a proof for AGI if your mechanical machine replicates all its processes. It's not a question about whether a machine can reproduce that, it's a question about whether we have given our current machines all the tools it needs to do that.
  - torginus 1y ago
    The theory of special relativity does not say 'you can't exceed the speed of light(unless you have a really big rocket)'. It presents a theoretical limit. Likewise the Chinese room doesn't state that consciousness is an intractable engineering problem, but an impossibility. But the way Searle formulates his argument, by not defining what consciousness is, he essentially gives himself enough wiggle room to be always right - he's essentially making the 'No True Scotsman' fallacy.
- hiatus 1y ago
  The A stands for artificial.
herculity275 1y ago
"Please stop referring to this thought experiment because it has possible interpretations I don't personally agree with"
- torginus 1y ago
  Please give me an interpretation that is both correct an meaningful (as in possible to disprove)
globnomulous 1y ago
> The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there. That's one possibility. The other is that your pomposity and dismissiveness towards the entire field of philosophy speaks volumes on how little you know about either philosophical arguments in general or this philosophical argument in particular.
- torginus 1y ago
  Another ad hominem, I'd like you to refute my claim that Searle's argument is essentially 100% magical thinking. And yes, if for example, medicine would be no worse at curing cancer than it is today, yet doctors asserted that crystal healing is a serious study, that would reflect badly on the field at large, despite most of it being sound.
  - dahart 1y ago
    Searle refutes your claim that there’s magical thinking. “Searle does not disagree with the notion that machines can have consciousness and understanding, because, as he writes, "we are precisely such machines". Searle holds that the brain is, in fact, a machine, but that the brain gives rise to consciousness and understanding using specific machinery.”
    - torginus 1y ago
      But the core of the original argument is that programs only manipulate symbols and consciousness can never arise just through symbol manipulation - which here then becomes 'we have not discovered the algorithms' for consciousness yet. It's just a contradiction.
      - dahart 1y ago
        When you say something that contradicts his statements, it doesn’t mean he’s wrong, it most likely means you haven’t understood or interpreted his argument correctly. The Wikipedia page you linked to doesn’t use the word “algorithm”, so the source of the contradiction you imagine might be you. Searle says he thinks humans are biological machines, so your argument should withstand that hypothesis rather than dismiss it.