4 ms·

Reminds me of the Chinese room [1] argument: Does a computer really understand Chinese language if it can respond to Chinese inputs with Chinese outputs? [1] h

by gessha 2y ago

Reminds me of the Chinese room [1] argument: Does a computer really understand Chinese language if it can respond to Chinese inputs with Chinese outputs? [1] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

kubanczyk 2y ago
Isn't a large company more handy model than a "Chinese room"? Hordes of these in practice own our species. They come up with the decisions on carbon footprint, on whether to colonize Mars, on where to shift collective attention. Although these entities are only one of achievements of our culture, for now they dominate on global level. Big achievement. "Consciousness" not strictly needed. Whatever the definition of "consciousness", a big company is driven by a glorified stack of excels and definitely couldn't write Chinese (assuming none of employees does, after Searle). So, running a planet and overriding any individual human is not enough, let's focus on entertaining an individual human with an interesting conversation in Chinese? What do you even seek with that model? /s
h0l0cube 2y ago
> Searle's thought experiment begins with this hypothetical premise: suppose that artificial intelligence research has succeeded in constructing a computer that behaves as if it understands Chinese. It takes Chinese characters as input and, by following the instructions of a computer program, produces other Chinese characters, which it presents as output. Suppose, says Searle, that this computer performs its task so convincingly that it comfortably passes the Turing test: it convinces a human Chinese speaker that the program is itself a live Chinese speaker. To all of the questions that the person asks, it makes appropriate responses, such that any Chinese speaker would be convinced that they are talking to another Chinese-speaking human being. > The question Searle wants to answer is this: does the machine literally "understand" Chinese? Or is it merely simulating the ability to understand Chinese? Searle calls the first position "strong AI" and the latter "weak AI". (Emphasis added) If we were to make an analogy to contemporary machine learning, we're talking about the difference between an LLM (with context) and a Markov Chain. 'Understanding' requires novel reuse of recollections. Recollections require memory (i.e. context), and the novel reuse of those recollections require a world model with which to inference.
- TaylorAlexander 2y ago
  I am personally in favor of the use of the word 'understanding' with LLMs, but it should be noted that many people strongly disagree with that use of the term.
- Terr_ 2y ago
  You're saying that LLMs meet the stronger definition of "understanding?" I disagree: You're confusing necessary with sufficient. [0] Take the original analogy of a person with an if-then opaque phrase table, and make it a slightly fancier two-step process where they must also compute a score based on recent phrases or words to determine the proper output. So now the system has "memory" and gives better output... But was that sufficient to create understanding? Nope. [0] And perhaps not even necessary, if we consider an operator who does understand Chinese but their short-term memory has been damaged by an injury.
  - h0l0cube 2y ago
    I think the confusion comes here in believing that humans have some exceptional mechanic for understanding. Ultimately human intelligence is the product of neural networks. The only thing that's separates us from ML is both scale (in terms of compute and memory) and agency, the latter giving us the ability to train ourselves. We take inputs, we output behaviors (including language). The idea that we are, in all actuality, deterministic computers is really at the heart of the existential panic around AI.
    - pests 2y ago
      > Ultimately human intelligence is the product of neural networks Citation needed? Sounds like "human intelligence is the product of <current tech hype>"
      - h0l0cube 2y ago
        You do realize that artificial neural networks are based on the organic ones? Unless you have some dualist interpretation of intelligence that involves some ethereal intangible soul that somehow eludes all detection, nor holds any explanatory power for intelligence? If you were obliterate your own neural network, by, I dunno, shooting yourself in the head, might that have some affect on your intelligence? Head trauma? Brain cancer? Alzheimer's? Senescence? Do I really need a citation that your brain is where intelligence comes from? *scratches head*

by gessha 2y ago

kubanczyk 2y ago

Isn't a large company more handy model than a "Chinese room"? Hordes of these in practice own our species. They come up with the decisions on carbon footprint, on whether to colonize Mars, on where to shift collective attention. Although these entities are only one of achievements of our culture, for now they dominate on global level. Big achievement. "Consciousness" not strictly needed. Whatever the definition of "consciousness", a big company is driven by a glorified stack of excels and definitely couldn't write Chinese (assuming none of employees does, after Searle). So, running a planet and overriding any individual human is not enough, let's focus on entertaining an individual human with an interesting conversation in Chinese? What do you even seek with that model? /s

h0l0cube 2y ago

> Searle's thought experiment begins with this hypothetical premise: suppose that artificial intelligence research has succeeded in constructing a computer that behaves as if it understands Chinese. It takes Chinese characters as input and, by following the instructions of a computer program, produces other Chinese characters, which it presents as output. Suppose, says Searle, that this computer performs its task so convincingly that it comfortably passes the Turing test: it convinces a human Chinese speaker that the program is itself a live Chinese speaker. To all of the questions that the person asks, it makes appropriate responses, such that any Chinese speaker would be convinced that they are talking to another Chinese-speaking human being. > The question Searle wants to answer is this: does the machine literally "understand" Chinese? Or is it merely simulating the ability to understand Chinese? Searle calls the first position "strong AI" and the latter "weak AI". (Emphasis added) If we were to make an analogy to contemporary machine learning, we're talking about the difference between an LLM (with context) and a Markov Chain. 'Understanding' requires novel reuse of recollections. Recollections require memory (i.e. context), and the novel reuse of those recollections require a world model with which to inference.

TaylorAlexander 2y ago
I am personally in favor of the use of the word 'understanding' with LLMs, but it should be noted that many people strongly disagree with that use of the term.
Terr_ 2y ago
You're saying that LLMs meet the stronger definition of "understanding?" I disagree: You're confusing necessary with sufficient. [0] Take the original analogy of a person with an if-then opaque phrase table, and make it a slightly fancier two-step process where they must also compute a score based on recent phrases or words to determine the proper output. So now the system has "memory" and gives better output... But was that sufficient to create understanding? Nope. [0] And perhaps not even necessary, if we consider an operator who does understand Chinese but their short-term memory has been damaged by an injury.
- h0l0cube 2y ago
  I think the confusion comes here in believing that humans have some exceptional mechanic for understanding. Ultimately human intelligence is the product of neural networks. The only thing that's separates us from ML is both scale (in terms of compute and memory) and agency, the latter giving us the ability to train ourselves. We take inputs, we output behaviors (including language). The idea that we are, in all actuality, deterministic computers is really at the heart of the existential panic around AI.
  - pests 2y ago
    > Ultimately human intelligence is the product of neural networks Citation needed? Sounds like "human intelligence is the product of <current tech hype>"
    - h0l0cube 2y ago
      You do realize that artificial neural networks are based on the organic ones? Unless you have some dualist interpretation of intelligence that involves some ethereal intangible soul that somehow eludes all detection, nor holds any explanatory power for intelligence? If you were obliterate your own neural network, by, I dunno, shooting yourself in the head, might that have some affect on your intelligence? Head trauma? Brain cancer? Alzheimer's? Senescence? Do I really need a citation that your brain is where intelligence comes from? *scratches head*