3 ms·

What you are probably missing is that for the model to be competent at predicting the next the word IT HAS to learn how to reason as a human being would.

peteradio 4y ago
If it quacks like a duck it must reason like a duck? Here I am quacking alone in my office, I guess I know how a duck thinks.
goatlover 4y ago
I don't see why this is. Humans aren't trained to perform a statistical analysis of what the next word is likely to be. We learn language from other humans in social and environmental context. The LLM is just being trained on text.
- rafaelero 4y ago
  Nobody is saying these models are trained the same way humans are. I am just saying that during their training task (predicting the next word) they end up developing human reasoning. That of course wasn't obvious before these large models appeared. But now it's pretty clear that's the case. It wouldn't be able to perform so well in so many tasks if it didn't have something resembling reasoning capabilities.
  - mcguire 4y ago
    I asked ChatGPT about the Chinese Room and asked for references. The first it came up with was "Minds, Brains, and Programs" by John Searle (1980). The next two were, "The Chinese Room: Searle and Artificial Intelligence" edited by John Preston and Mark Bishop (2002) (Preston and Bishop did publish Views into the Chinese Room in 2002, but I can't find anything by the given title) and "The Chinese Room Argument" by David Chalmers (2006) (Chalmers has written on the topic, and published three articles in 2006, that's not one of them. Also: "Two cities: one, a city of “knights”, and the other of “knaves”. The residents of the two cities like to visit each other. One day, a man enters one of the cities, but doesn’t know which. He decides to ask the first person he meets where he is. The answer he receives is that he is in the “knaves” city." produced the result: "The man cannot be sure that the person he met was telling the truth, because the person could be a knight from the city of knights, who always lie, or a knave from the city of knaves, who always tell the truth. "To determine which city he is in, the man could ask the second person he meets the same question. If the first person was a knight, then the second person would also be a knight, and would lie about being in the city of knights. If the first person was a knave, then the second person would also be a knave, and would tell the truth about being in the city of knaves. "By asking the same question to multiple people and analyzing their responses, the man can eventually determine which city he is in. This problem, known as the "Knights and Knaves" puzzle, is a classic example of logical reasoning and problem-solving." It seems to recognize the "knights and knaves" form of puzzle, but doesn't get the knights and knaves right. And it's not getting the reasoning part at all even given varying prompts.
    - rafaelero 4y ago
      I don't understand what point you are making. Are you saying that it lacks any reasoning capability because it fails this test? I say try a bunch more tests and then get the average to really have a better sense; one test is meaningless.
      - mcguire 4y ago
        How many opportunities would you give someone who, when asked to justify their answer, routinely made up references, and who ignored important parts of a question and whose answer wouldn't work anyway? As it turns out, we kinda do know what is going on in a LLM, and there's no model of reality for it to base reasoning on in there. Does that mean that's impossible? No. Does it mean seeing reasoning there is making the same mistake as those who enjoyed chatting with Parry and Eliza? Probably.

peteradio 4y ago

If it quacks like a duck it must reason like a duck? Here I am quacking alone in my office, I guess I know how a duck thinks.

goatlover 4y ago

I don't see why this is. Humans aren't trained to perform a statistical analysis of what the next word is likely to be. We learn language from other humans in social and environmental context. The LLM is just being trained on text.

rafaelero 4y ago
Nobody is saying these models are trained the same way humans are. I am just saying that during their training task (predicting the next word) they end up developing human reasoning. That of course wasn't obvious before these large models appeared. But now it's pretty clear that's the case. It wouldn't be able to perform so well in so many tasks if it didn't have something resembling reasoning capabilities.
- mcguire 4y ago
  I asked ChatGPT about the Chinese Room and asked for references. The first it came up with was "Minds, Brains, and Programs" by John Searle (1980). The next two were, "The Chinese Room: Searle and Artificial Intelligence" edited by John Preston and Mark Bishop (2002) (Preston and Bishop did publish Views into the Chinese Room in 2002, but I can't find anything by the given title) and "The Chinese Room Argument" by David Chalmers (2006) (Chalmers has written on the topic, and published three articles in 2006, that's not one of them. Also: "Two cities: one, a city of “knights”, and the other of “knaves”. The residents of the two cities like to visit each other. One day, a man enters one of the cities, but doesn’t know which. He decides to ask the first person he meets where he is. The answer he receives is that he is in the “knaves” city." produced the result: "The man cannot be sure that the person he met was telling the truth, because the person could be a knight from the city of knights, who always lie, or a knave from the city of knaves, who always tell the truth. "To determine which city he is in, the man could ask the second person he meets the same question. If the first person was a knight, then the second person would also be a knight, and would lie about being in the city of knights. If the first person was a knave, then the second person would also be a knave, and would tell the truth about being in the city of knaves. "By asking the same question to multiple people and analyzing their responses, the man can eventually determine which city he is in. This problem, known as the "Knights and Knaves" puzzle, is a classic example of logical reasoning and problem-solving." It seems to recognize the "knights and knaves" form of puzzle, but doesn't get the knights and knaves right. And it's not getting the reasoning part at all even given varying prompts.
  - rafaelero 4y ago
    I don't understand what point you are making. Are you saying that it lacks any reasoning capability because it fails this test? I say try a bunch more tests and then get the average to really have a better sense; one test is meaningless.
    - mcguire 4y ago
      How many opportunities would you give someone who, when asked to justify their answer, routinely made up references, and who ignored important parts of a question and whose answer wouldn't work anyway? As it turns out, we kinda do know what is going on in a LLM, and there's no model of reality for it to base reasoning on in there. Does that mean that's impossible? No. Does it mean seeing reasoning there is making the same mistake as those who enjoyed chatting with Parry and Eliza? Probably.