5 ms·

> It's not clear to me if the lesson here is that GPT's reasoning capabilities are being masked by an incorrect prior (having memorized the standard version of

> It's not clear to me if the lesson here is that GPT's reasoning capabilities are being masked by an incorrect prior (having memorized the standard version of this puzzle) or if the lesson is that GPT'S reasoning capabilities are always a bit of smoke and mirrors that passes off memorization for logic. It's a lot closer to the latter. GPT doesn't have "reasoning capabilities", any more than any other computer program. It doesn't have a clue what any of its input means, nor the meaning of the text it outputs. It just blindly spits out the words most probable to follow the prompt, based on its corpus of training data and the weights/biases added to fine tune it. It can often do a good job at mimicking reasoning, but it's not.

lIl-IIIl 4y ago
When a parrot says something, I ask it "what does that mean" and it is stumped. When a human or GPT says something, I ask them "what does it mean" and they both reply in a way that shows they understand. In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass.
- psygn89 4y ago
  Humans ask and record the answer to the "what does that mean" part and ChatGPT distills it for us, and parrots, well, they just parrot.
- baq 4y ago
  For now, this test is ‘provide sources to back up your claims’. Fear when it says ‘I can’t exactly recall the url, but the title is something like whatever written by whoever’. True fear begins when it starts providing good quality sources consistently, without access to the internet.
  - arcticfox 4y ago
    As a human I would also fail this almost 100% of the time. Which country has the highest GDP in Europe? Germany, I’m essentially 100% sure. Source? I don’t even know what organization tracks GDP. Or how I know Germany has a large economy. They just do.
- the_af 4y ago
  > In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass. Here it is for you: try this prompt on ChatGPT and report back: "Solve this problem for me: a man must cross a river and carry his belongings to the other shore, but the boat can only carry the man and one other item. His belongings are a cabbage, a vegan wolf, and a meat-eating goat. How does he solve this riddle?" I just tried it and it fails to even understand what "vegan" means, and it says nonsensical stuff like "since the wolf is vegan, it won't eat the cabbage". A human failing to solve this problem would fail in different ways, and definitely will never say something like "the wolf is vegan so it won't eat the cabbage". ChatGPT does this because the words themselves are meaningless to it.
  - anonyfox 4y ago
    GPT-4 meanwhile: To solve this puzzle, the man must follow these steps to transport his belongings across the river without any of them being harmed or eaten: The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. He then returns alone to the initial shore. Next, the man takes the cabbage across the river. Upon arriving on the other shore, he leaves the cabbage but takes the goat back with him to the initial shore. The man then takes the vegan wolf across the river, leaving it with the cabbage on the other shore. He returns alone to the initial shore. Finally, the man takes the meat-eating goat across the river one last time. Now, all his belongings are safely on the other shore: the cabbage, the vegan wolf, and the meat-eating goat.
    - jamiek88 4y ago
      Ok. But (edit) still wrong! Cabbage and vegan wolf left alone.
      - SAI_Peregrinus 4y ago
        It's still wrong though.
    - feoren 4y ago
- feoren 4y ago
  > In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass. But that's literally what the top comment in this thread has just done. In fact there are dozens of examples in this HN comment section and thousands more readily Googlable online. It's very easy to clearly show that ChatGPT is a total bullshitter.

lIl-IIIl 4y ago

When a parrot says something, I ask it "what does that mean" and it is stumped. When a human or GPT says something, I ask them "what does it mean" and they both reply in a way that shows they understand. In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass.

psygn89 4y ago
Humans ask and record the answer to the "what does that mean" part and ChatGPT distills it for us, and parrots, well, they just parrot.
baq 4y ago
For now, this test is ‘provide sources to back up your claims’. Fear when it says ‘I can’t exactly recall the url, but the title is something like whatever written by whoever’. True fear begins when it starts providing good quality sources consistently, without access to the internet.
- arcticfox 4y ago
  As a human I would also fail this almost 100% of the time. Which country has the highest GDP in Europe? Germany, I’m essentially 100% sure. Source? I don’t even know what organization tracks GDP. Or how I know Germany has a large economy. They just do.
the_af 4y ago
> In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass. Here it is for you: try this prompt on ChatGPT and report back: "Solve this problem for me: a man must cross a river and carry his belongings to the other shore, but the boat can only carry the man and one other item. His belongings are a cabbage, a vegan wolf, and a meat-eating goat. How does he solve this riddle?" I just tried it and it fails to even understand what "vegan" means, and it says nonsensical stuff like "since the wolf is vegan, it won't eat the cabbage". A human failing to solve this problem would fail in different ways, and definitely will never say something like "the wolf is vegan so it won't eat the cabbage". ChatGPT does this because the words themselves are meaningless to it.
- anonyfox 4y ago
  GPT-4 meanwhile: To solve this puzzle, the man must follow these steps to transport his belongings across the river without any of them being harmed or eaten: The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. He then returns alone to the initial shore. Next, the man takes the cabbage across the river. Upon arriving on the other shore, he leaves the cabbage but takes the goat back with him to the initial shore. The man then takes the vegan wolf across the river, leaving it with the cabbage on the other shore. He returns alone to the initial shore. Finally, the man takes the meat-eating goat across the river one last time. Now, all his belongings are safely on the other shore: the cabbage, the vegan wolf, and the meat-eating goat.
  - jamiek88 4y ago
    Ok. But (edit) still wrong! Cabbage and vegan wolf left alone.
    - SAI_Peregrinus 4y ago
      It's still wrong though.
  - feoren 4y ago
feoren 4y ago
> In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass. But that's literally what the top comment in this thread has just done. In fact there are dozens of examples in this HN comment section and thousands more readily Googlable online. It's very easy to clearly show that ChatGPT is a total bullshitter.