3 ms·

Yes, in this instance I understand failings of today (though copilot has a much better hit rate, and at the moment it’s a great augmentation to coding if you tr

Yes, in this instance I understand failings of today (though copilot has a much better hit rate, and at the moment it’s a great augmentation to coding if you treat it like an enthusiastic intern). My question is about the future. The argument goes that a machine can never understand Chinese, even if it is capable of interpreting Chinese and responding to or acting on the input perfectly every time. My reply is that, if it acts as if it understands Chinese in every situation, then there’s no measurable way of distinguishing it from understanding. It’s kind of like the whole string theory vs SUSY vs… argument in physics. If the only outcomes are things that agree with the Standard Model in all measurable aspects, and don’t provide any measurable distinction, then for all intents and purposes they don’t matter. That’s why their active areas of research are looking for the measurable distinctions. FWIW, supersymmetry models predict measurable things (that so far have only ruled out those models when tested) but have applications elsewhere. String theory research has had implications in mathematics, condensed matter, and a bunch of other places. They’re useful. But that’s beside the point, because the premise of the Chinese room problem is that there exists a machine that passes all scenarios, where no measurable difference can be found, and that this machine does not understand Chinese.

missingrib 4y ago
> My reply is that, if it acts as if it understands Chinese in every situation, then there’s no measurable way of distinguishing it from understanding. I'm not sure if you understood the argument. The argument isn't asserting that there is a measurable way of distinguishing it, it's actually claiming that regardless of how well it seems like it understands Chinese, it doesn't actually understand Chinese. It's about intentionality and consciousness.
- int_19h 4y ago
  The argument is that the man inside the room doesn't understand Chinese. The entire system still obviously and demonstrably does. In a chatbot, the man inside the room is the LLM, but the whole system is not just the LLM - it's the whole setup that picks generated tokens and feeds them back into the input as a loop. And it demonstrably understands what you tell it, because it can carry out instructions, even extremely convoluted ones or using substitute words that are not part of its training set.
- i-use-nixos-btw 4y ago
  You are entirely correct: I do not understand the argument - or, rather, the relevance of it in any applied situation. In effect, my argument is that in order for you to require it to understand something, you require it to understand that thing for a reason. If it acts like it understands that thing under all probing, then your requirement is satisfied - the question about whether it truly understands the thing is moot, because it fulfils the requirement.
barrysteve 4y ago
It's hard to keep this theoretical. Yes a machine is just a machine. Defining a machine to be conscious, allows the individual to soak their mind in code and silicon as a receptacle for their spirit. It creates a pull into a 'second mind'. Anybody who believes this is likely to invest heavily in the maintenance of new technology. A 'conscious machine', creates an uneasy feeling that we should work to embed our spirit, knowledge, intellect into flipped bits, like expectant mothers. That we should work for the machine, and to the ends of the machine. And that machine is somehow defined-to-be or a naturally, consciously alive (to a large or small degree). It is said to have a mind worthy of a person's professional output and it can hold the power of a marginally believable conversation. While all of these described properties are vaguely plausible, it does nothing to help me understand the meaning of a technology, and only benefits those looking to create a fevor around a new tech product. Describing chatgpt as a stochastic parrot or chinese room grants me a metaphor or analogy for the inner functions of the tech. It also lets me see, or otherwise guesstimate the products abilities clearly, without the belief-as-marketing hype. I can take the stochastic parrot metaphor, to an article about LLMs and understand in a couple of days what took years of research to create. Following the belief of computing as real human intelligence and that human intelligence is fundamentally mathematical, requires on some level submission of your mind to a machine that has it's own goals programmed in by someone else. This centuries-long process of trying to encode and store all human knowledge behind the secure walls of complex coded signs.. and it's advocates for that process, create a subtle and deep twinge of future melancholy or dread or something. The idea that all written/typed meaning will be accessible only by the spiritual power brokers, and not our sons. No. On some level, machines are just machines, like an abacus or a weaving loom. It can host concepts in the same way that a weaving loom is 'intelligent'. It holds it's shape, abstractions and functions by the laws of physics/metaphysics and according to my human dictates. You follow the raven into the computers-are-conscious dream at your own risk. Computers are leaning towards controlling people rather than emancipating them. Leaning very hard in that direction. Do we want that? Freedom of mind and meaning is valueable.

missingrib 4y ago

> My reply is that, if it acts as if it understands Chinese in every situation, then there’s no measurable way of distinguishing it from understanding. I'm not sure if you understood the argument. The argument isn't asserting that there is a measurable way of distinguishing it, it's actually claiming that regardless of how well it seems like it understands Chinese, it doesn't actually understand Chinese. It's about intentionality and consciousness.

int_19h 4y ago
The argument is that the man inside the room doesn't understand Chinese. The entire system still obviously and demonstrably does. In a chatbot, the man inside the room is the LLM, but the whole system is not just the LLM - it's the whole setup that picks generated tokens and feeds them back into the input as a loop. And it demonstrably understands what you tell it, because it can carry out instructions, even extremely convoluted ones or using substitute words that are not part of its training set.
i-use-nixos-btw 4y ago
You are entirely correct: I do not understand the argument - or, rather, the relevance of it in any applied situation. In effect, my argument is that in order for you to require it to understand something, you require it to understand that thing for a reason. If it acts like it understands that thing under all probing, then your requirement is satisfied - the question about whether it truly understands the thing is moot, because it fulfils the requirement.

barrysteve 4y ago

It's hard to keep this theoretical. Yes a machine is just a machine. Defining a machine to be conscious, allows the individual to soak their mind in code and silicon as a receptacle for their spirit. It creates a pull into a 'second mind'. Anybody who believes this is likely to invest heavily in the maintenance of new technology. A 'conscious machine', creates an uneasy feeling that we should work to embed our spirit, knowledge, intellect into flipped bits, like expectant mothers. That we should work for the machine, and to the ends of the machine. And that machine is somehow defined-to-be or a naturally, consciously alive (to a large or small degree). It is said to have a mind worthy of a person's professional output and it can hold the power of a marginally believable conversation. While all of these described properties are vaguely plausible, it does nothing to help me understand the meaning of a technology, and only benefits those looking to create a fevor around a new tech product. Describing chatgpt as a stochastic parrot or chinese room grants me a metaphor or analogy for the inner functions of the tech. It also lets me see, or otherwise guesstimate the products abilities clearly, without the belief-as-marketing hype. I can take the stochastic parrot metaphor, to an article about LLMs and understand in a couple of days what took years of research to create. Following the belief of computing as real human intelligence and that human intelligence is fundamentally mathematical, requires on some level submission of your mind to a machine that has it's own goals programmed in by someone else. This centuries-long process of trying to encode and store all human knowledge behind the secure walls of complex coded signs.. and it's advocates for that process, create a subtle and deep twinge of future melancholy or dread or something. The idea that all written/typed meaning will be accessible only by the spiritual power brokers, and not our sons. No. On some level, machines are just machines, like an abacus or a weaving loom. It can host concepts in the same way that a weaving loom is 'intelligent'. It holds it's shape, abstractions and functions by the laws of physics/metaphysics and according to my human dictates. You follow the raven into the computers-are-conscious dream at your own risk. Computers are leaning towards controlling people rather than emancipating them. Leaning very hard in that direction. Do we want that? Freedom of mind and meaning is valueable.