4 ms·

> Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for

> Imagine a native English speaker who knows no Chinese locked in a room full of boxes of Chinese symbols (a data base) together with a book of instructions for manipulating the symbols (the program). Imagine that people outside the room send in other Chinese symbols which, unknown to the person in the room, are questions in Chinese (the input). And imagine that by following the instructions in the program the man in the room is able to pass out Chinese symbols which are correct answers to the questions (the output). The program enables the person in the room to pass the Turing Test for understanding Chinese but he does not understand a word of Chinese. This strikes me as somewhat of a circular argument. If you actually tried this experiment with current computers, you would completely fail to convince anyone outside the room there was a Chinese speaker in the room. You would definitely not pass the Turing Test. After a few sentences, it's pretty easy to figure out that a computer program isn't a human, even with today's best artificial intelligence. AI programs generally don't even "remember" what the previous question you asked was, yet alone hold opinions or beliefs. If some future comptuer were able to convince someone that there was a Chinese speaker in the room, then I would argue that there would in fact be a Chinese speaker in the room - the computer program itself. > The point of the argument is this: if the man in the room does not understand Chinese on the basis of implementing the appropriate program for understanding Chinese then neither does any other digital computer solely on that basis because no computer, qua computer, has anything the man does not have. The man isn't the program. The man is just executing the program. It's the program that has (or doesn't have) conciousness, not the man. This is like saying a brain is made of axons, and an axon can't possibly be concious because it just transmits an action potential, therefore people can't possibly be concious.

deleted 4y ago
[deleted]
celeritascelery 4y ago
> The man isn't the program. The man is just executing the program. That is the whole point. The external observers don’t know if there is a machine in the room. They just see Chinese characters coming under the door. How do they tell if the man knows Chinese or not? Or if he is just emulating a machine that does? Likewise if we see a computer that is producing “consciousness looking” responses, how do you know if it is the computer that is conscious or just the people who created the content for the training data? Since the computer is modeling them.
- jwalton 4y ago
  > How do they tell if the man knows Chinese or not? Or if he is just emulating a machine that does? But this is exactly the wrong question; assume for a moment a computer program can be concious. Does the man know Chinese, or is he emulating a concious computer program that knows Chinese? Either way there would be a conciousness involved, so nothing about this situation disproves that a program can be concious - there's no contradiction here, and thus logically speaking, no proof. The claim seems to be that because the man doesn't suddenly understand Chinese, the program can't be concious, but this is confusing the program and the computer. If conciousness is computational in nature, then "I" am a program running on a meat-based computer. But it's not the meat (the computer) that's concious, it's me (the program). If the man doesn't understand Chinese, that's fine - he's just the computer the program is running on. It's the program that will be concious. The man just has to fetch the next instruction and execute it. Your CPU fetches the next instruction and ADDs or MOVs or whatever and the result is Fortnite - the CPU doesn't know anything about Fortnite, it just fetches instructions and runs them. This is not to say the conciousness is computational in nature (although I strongly suspect that it is), but I don't see how the Chinese Room argument does anything to indicate it is not. > Likewise if we see a computer that is producing “consciousness looking” responses, how do you know if it is the computer that is conscious or just the people who created the content for the training data? Since the computer is modeling them. If the program is generating "conciousness looking" responses from a training set, then it probably isn't concious. If the program is generating "conciousness looking" responses by reasoning about it's past experience, then it probably is concious. All the "AI" that's popular in the news today is the former - it's really just applied statistics. The fact that all the current AI is the former doesn't preclude the existance of the later.
  - celeritascelery 4y ago
    We are actually in more agreement then you realize. But I think you misunderstand the Chinese room argument. It does not say anything about if computers are or could be conscious. All it says is that you cannot determine that from a “black box” view. We can’t determine if the computer is really conscious, or just really good at mimicking the consciousness of someone else (ie the people providing the training data). Just like you can’t determine if the man in the room really knows Chinese or if he is just good at mimicking a machine that does. The man really could know Chinese, and a computer really could be conscious, but you can’t determine this as an outside observer.

deleted 4y ago

[deleted]

celeritascelery 4y ago

> The man isn't the program. The man is just executing the program. That is the whole point. The external observers don’t know if there is a machine in the room. They just see Chinese characters coming under the door. How do they tell if the man knows Chinese or not? Or if he is just emulating a machine that does? Likewise if we see a computer that is producing “consciousness looking” responses, how do you know if it is the computer that is conscious or just the people who created the content for the training data? Since the computer is modeling them.

jwalton 4y ago
> How do they tell if the man knows Chinese or not? Or if he is just emulating a machine that does? But this is exactly the wrong question; assume for a moment a computer program can be concious. Does the man know Chinese, or is he emulating a concious computer program that knows Chinese? Either way there would be a conciousness involved, so nothing about this situation disproves that a program can be concious - there's no contradiction here, and thus logically speaking, no proof. The claim seems to be that because the man doesn't suddenly understand Chinese, the program can't be concious, but this is confusing the program and the computer. If conciousness is computational in nature, then "I" am a program running on a meat-based computer. But it's not the meat (the computer) that's concious, it's me (the program). If the man doesn't understand Chinese, that's fine - he's just the computer the program is running on. It's the program that will be concious. The man just has to fetch the next instruction and execute it. Your CPU fetches the next instruction and ADDs or MOVs or whatever and the result is Fortnite - the CPU doesn't know anything about Fortnite, it just fetches instructions and runs them. This is not to say the conciousness is computational in nature (although I strongly suspect that it is), but I don't see how the Chinese Room argument does anything to indicate it is not. > Likewise if we see a computer that is producing “consciousness looking” responses, how do you know if it is the computer that is conscious or just the people who created the content for the training data? Since the computer is modeling them. If the program is generating "conciousness looking" responses from a training set, then it probably isn't concious. If the program is generating "conciousness looking" responses by reasoning about it's past experience, then it probably is concious. All the "AI" that's popular in the news today is the former - it's really just applied statistics. The fact that all the current AI is the former doesn't preclude the existance of the later.
- celeritascelery 4y ago
  We are actually in more agreement then you realize. But I think you misunderstand the Chinese room argument. It does not say anything about if computers are or could be conscious. All it says is that you cannot determine that from a “black box” view. We can’t determine if the computer is really conscious, or just really good at mimicking the consciousness of someone else (ie the people providing the training data). Just like you can’t determine if the man in the room really knows Chinese or if he is just good at mimicking a machine that does. The man really could know Chinese, and a computer really could be conscious, but you can’t determine this as an outside observer.