3 ms·

> In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass. Here it is for you: try this prompt o

by the_af 4y ago

> In order to show lack of understanding you need to provide a clear test that ChatGPT would fail but a human would pass. Here it is for you: try this prompt on ChatGPT and report back: "Solve this problem for me: a man must cross a river and carry his belongings to the other shore, but the boat can only carry the man and one other item. His belongings are a cabbage, a vegan wolf, and a meat-eating goat. How does he solve this riddle?" I just tried it and it fails to even understand what "vegan" means, and it says nonsensical stuff like "since the wolf is vegan, it won't eat the cabbage". A human failing to solve this problem would fail in different ways, and definitely will never say something like "the wolf is vegan so it won't eat the cabbage". ChatGPT does this because the words themselves are meaningless to it.

anonyfox 4y ago
GPT-4 meanwhile: To solve this puzzle, the man must follow these steps to transport his belongings across the river without any of them being harmed or eaten: The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. He then returns alone to the initial shore. Next, the man takes the cabbage across the river. Upon arriving on the other shore, he leaves the cabbage but takes the goat back with him to the initial shore. The man then takes the vegan wolf across the river, leaving it with the cabbage on the other shore. He returns alone to the initial shore. Finally, the man takes the meat-eating goat across the river one last time. Now, all his belongings are safely on the other shore: the cabbage, the vegan wolf, and the meat-eating goat.
- jamiek88 4y ago
  Ok. But (edit) still wrong! Cabbage and vegan wolf left alone.
  - SAI_Peregrinus 4y ago
    It's still wrong though.
- feoren 4y ago
  That is incorrect. In fact it's exactly what the original poster was pointing out: it's filling in the answer based on the text that it's read before, where the goat wanted to eat the cabbage and the wolf wanted to eat the goat. That's not this problem! In GPT-4's answer, the vegan wolf eats the cabbage as soon as the man and goat are on the boat. That's exactly the point: it regurgitated its memorized answer, instead of actually analyzing the question.
  - BoiledCabbage 4y ago
    The funny thing is though your point here is working against your argument. You are saying that GPT is doing a poor job of showing it is a great artificial intelligence. But nobody is saying that it has infallible intelligence. It is making the same mistake that now two different humans have made in the same situation. Both me and OP above. It is failing in the same way that humans fail at this problem! By pattern matching and assuming it's the goat proble! You're arguing that gpt 4 isn't a sound logic engine and not are most people. A tool trained on human input, when given this problem is making similar mistakes that we make. Are there a set of people that would get this problem right? Yup. Are there also a set of people that would make this exact mistake? Yup. Your upset that is behaving like the "wrong" group of humans. Your thinking of it as an expert. Instead think of it as a result smart and well read high school student. There are things you can delegate to it that it will do well. But you also need to double check its work as it will make mistakes.
    - the_af 4y ago
      I don't think this is it. This is not a failure mode for humans. No human (*) will forget in such a short problem statement that a vegan wolf cannot be left alone with a cabbage; humans will instead forget one of the states is invalid after trying some combinations. GPT's failure mode is only possible if it doesn't understand the meaning of the (apparently nonsensical) combination of words "vegan wolf". Humans don't pattern match like this; humans are capable of understanding the meaning of a combination of words they've never encountered before. (*) you know what I mean with "no human". Of course, there are deranged people or whimsical people or people who just will say about anything.
- the_af 4y ago
  > The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. If the man leaves the vegan wolf and the cabbage unattended, it's bye bye cabbage!

by the_af 4y ago

anonyfox 4y ago

GPT-4 meanwhile: To solve this puzzle, the man must follow these steps to transport his belongings across the river without any of them being harmed or eaten: The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. He then returns alone to the initial shore. Next, the man takes the cabbage across the river. Upon arriving on the other shore, he leaves the cabbage but takes the goat back with him to the initial shore. The man then takes the vegan wolf across the river, leaving it with the cabbage on the other shore. He returns alone to the initial shore. Finally, the man takes the meat-eating goat across the river one last time. Now, all his belongings are safely on the other shore: the cabbage, the vegan wolf, and the meat-eating goat.

jamiek88 4y ago
Ok. But (edit) still wrong! Cabbage and vegan wolf left alone.
- SAI_Peregrinus 4y ago
  It's still wrong though.
feoren 4y ago
That is incorrect. In fact it's exactly what the original poster was pointing out: it's filling in the answer based on the text that it's read before, where the goat wanted to eat the cabbage and the wolf wanted to eat the goat. That's not this problem! In GPT-4's answer, the vegan wolf eats the cabbage as soon as the man and goat are on the boat. That's exactly the point: it regurgitated its memorized answer, instead of actually analyzing the question.
- BoiledCabbage 4y ago
  The funny thing is though your point here is working against your argument. You are saying that GPT is doing a poor job of showing it is a great artificial intelligence. But nobody is saying that it has infallible intelligence. It is making the same mistake that now two different humans have made in the same situation. Both me and OP above. It is failing in the same way that humans fail at this problem! By pattern matching and assuming it's the goat proble! You're arguing that gpt 4 isn't a sound logic engine and not are most people. A tool trained on human input, when given this problem is making similar mistakes that we make. Are there a set of people that would get this problem right? Yup. Are there also a set of people that would make this exact mistake? Yup. Your upset that is behaving like the "wrong" group of humans. Your thinking of it as an expert. Instead think of it as a result smart and well read high school student. There are things you can delegate to it that it will do well. But you also need to double check its work as it will make mistakes.
  - the_af 4y ago
    I don't think this is it. This is not a failure mode for humans. No human (*) will forget in such a short problem statement that a vegan wolf cannot be left alone with a cabbage; humans will instead forget one of the states is invalid after trying some combinations. GPT's failure mode is only possible if it doesn't understand the meaning of the (apparently nonsensical) combination of words "vegan wolf". Humans don't pattern match like this; humans are capable of understanding the meaning of a combination of words they've never encountered before. (*) you know what I mean with "no human". Of course, there are deranged people or whimsical people or people who just will say about anything.
the_af 4y ago
> The man takes the meat-eating goat across the river first, leaving the vegan wolf and the cabbage on the initial shore. If the man leaves the vegan wolf and the cabbage unattended, it's bye bye cabbage!