4 ms·

One of my biggest gripes is around the Chinese room argument and these are different forms of the same. It is assumed that a mechanistic process could not have

by vaidhy 3y ago

One of my biggest gripes is around the Chinese room argument and these are different forms of the same. It is assumed that a mechanistic process could not have "intelligence" and it has been morphed into many forms without anyone defining what is intelligence (or conscious for that matter). The arguments fall into one of these two buckets: 1. Human exceptionalism (religious argument) - Humans (and maybe some of the lower creatures) have intelligence, but others and things created by humans do not have it. It does not matter what or how you define intelligence, but by definition only some living beings can have it. 2. Frankenstein (Shit..it is scary argument) - We have seen these emergent properties and it is not something that we can explain easily to people. Everyone is already wary and let us placate the masses by saying it is not scary and in fact, it is just a small tame cute furball. Personally, I think all these arguments are useless and misplaced. If we create something that can do critical reasoning, all the more power to us.

DougWebb 3y ago
You complain about arguments that don't define "intelligence" and what it means for AI to have that, and then you say: If we create something that can do critical reasoning, all the more power to us. It seems to me you're not defining "critical reasoning", and that it's effectively a synonym for "intelligence", at least as applied to AI capabilities. I think AI responses are like the responses of a person who's very new to a subject and doesn't understand it, but they know all of the terminology and how the terms are usually used together. They can be helpful, and often correct, but only an expert in the field is going to know for sure if they're correct. And my fear about AI is not the AI itself, it's how the non-experts are going to use the AI to try to replace the experts in fields where being correct is really important. That will work, but only sometimes. If we don't make the people who choose the AI this way responsible for the outcomes, we're going to have a real mess on our hands.
- vaidhy 3y ago
  You are correct that I did not define critical reasoning. However, here is an sample case - I set a context and ask the answer from the computer. If the computer rightly understands the context and answers the question, then that is critical reasoning for me. Now that I type it out, it sounds like a version of a Turing test and I do not know if we can do better. So, in your hypothetical case, I would argue that using the terminology and providing the answers assume some level of intelligence. We get caught up in the training process and automatically assume what computer can infer because it is what we can infer. But the computer is not human and its world model is not the same as our world model. Your fear is reasonable, but religion has gotten away with it for so long :)
shagie 3y ago
The point at which the Chinese room fails for me when trying to describe GPT is when I make up a world with certain rules for it and have GPT do a task in that world using those rules. The idea that there's a lookup table somewhere seems to fail in these situations - or rather that if there's a lookup table it is astronomically large compared to the size of the model. In writing this I was thinking about the classic wolf, goat and cabbage problem which has been well studied. What about Wizards, Warriors and Priests with a set of powers? Imagine a universe where there are three types of people: wizards, warriors, and priests. Wizards can open a portal that allows two people to go through at a time, but they cannot go through the portal themselves. Priests can summon people from other locations to their location or teleport to the location of another person. Warriors cannot teleport or summon, but may be teleported or summoned by others. --- Given four wizards, a priest, and a warrior - what are the necessary steps to move them all to a new location? The when GPT answers that, it doesn't seem reasonable that it has been trained on that type of data or problem (in the abstract) or that according to the Chinese room problem there's a lookup table such that when you get that text you can look up the answer for it and generate an answer (that seems to have been reasoned out - it is also interesting to then tweak the problem domain and have it create a new response ("wizards can only open one portal"). The Chinese room is ok enough to say "nope, that doesn't need consciousness" for the chatbots of old that are very much a parse / lookup / response style approach. Even the digital assistants of Siri, Alexa, and Google are acceptable with that model. But with GPT-4 that it can't be explained with the Chinese room... as I said, that's interesting. It doesn't mean that GPT-4 is conscious (I would tend to argue against it but I don't have a good definition of consciousness to argue from) its just that one can't say that GPT-4 isn't conscious using the Chinese room to describe its abilities. The Frankenstein fear, I believe, is two part. First, there's the "it's coming for our jobs" part of automation and robots. There's also an existential fear that it is harder and harder to say what makes us human and separates us from machine. That gets very much into questions that come into conflict with many individuals identity and the value of knowledge. If those questions haven't been considered before or if you are coming from a rigid worldview where the possible new answers are coming into conflict with them - the LLM can be quite frightening.

by vaidhy 3y ago

DougWebb 3y ago

You complain about arguments that don't define "intelligence" and what it means for AI to have that, and then you say: If we create something that can do critical reasoning, all the more power to us. It seems to me you're not defining "critical reasoning", and that it's effectively a synonym for "intelligence", at least as applied to AI capabilities. I think AI responses are like the responses of a person who's very new to a subject and doesn't understand it, but they know all of the terminology and how the terms are usually used together. They can be helpful, and often correct, but only an expert in the field is going to know for sure if they're correct. And my fear about AI is not the AI itself, it's how the non-experts are going to use the AI to try to replace the experts in fields where being correct is really important. That will work, but only sometimes. If we don't make the people who choose the AI this way responsible for the outcomes, we're going to have a real mess on our hands.

vaidhy 3y ago
You are correct that I did not define critical reasoning. However, here is an sample case - I set a context and ask the answer from the computer. If the computer rightly understands the context and answers the question, then that is critical reasoning for me. Now that I type it out, it sounds like a version of a Turing test and I do not know if we can do better. So, in your hypothetical case, I would argue that using the terminology and providing the answers assume some level of intelligence. We get caught up in the training process and automatically assume what computer can infer because it is what we can infer. But the computer is not human and its world model is not the same as our world model. Your fear is reasonable, but religion has gotten away with it for so long :)

shagie 3y ago

The point at which the Chinese room fails for me when trying to describe GPT is when I make up a world with certain rules for it and have GPT do a task in that world using those rules. The idea that there's a lookup table somewhere seems to fail in these situations - or rather that if there's a lookup table it is astronomically large compared to the size of the model. In writing this I was thinking about the classic wolf, goat and cabbage problem which has been well studied. What about Wizards, Warriors and Priests with a set of powers? Imagine a universe where there are three types of people: wizards, warriors, and priests. Wizards can open a portal that allows two people to go through at a time, but they cannot go through the portal themselves. Priests can summon people from other locations to their location or teleport to the location of another person. Warriors cannot teleport or summon, but may be teleported or summoned by others. --- Given four wizards, a priest, and a warrior - what are the necessary steps to move them all to a new location? The when GPT answers that, it doesn't seem reasonable that it has been trained on that type of data or problem (in the abstract) or that according to the Chinese room problem there's a lookup table such that when you get that text you can look up the answer for it and generate an answer (that seems to have been reasoned out - it is also interesting to then tweak the problem domain and have it create a new response ("wizards can only open one portal"). The Chinese room is ok enough to say "nope, that doesn't need consciousness" for the chatbots of old that are very much a parse / lookup / response style approach. Even the digital assistants of Siri, Alexa, and Google are acceptable with that model. But with GPT-4 that it can't be explained with the Chinese room... as I said, that's interesting. It doesn't mean that GPT-4 is conscious (I would tend to argue against it but I don't have a good definition of consciousness to argue from) its just that one can't say that GPT-4 isn't conscious using the Chinese room to describe its abilities. The Frankenstein fear, I believe, is two part. First, there's the "it's coming for our jobs" part of automation and robots. There's also an existential fear that it is harder and harder to say what makes us human and separates us from machine. That gets very much into questions that come into conflict with many individuals identity and the value of knowledge. If those questions haven't been considered before or if you are coming from a rigid worldview where the possible new answers are coming into conflict with them - the LLM can be quite frightening.