4 ms·

You can give it puzzles that require understanding cause and effect, and it'll get them right. So this comment is wrong. "Next token prediction" taken to a cer

You can give it puzzles that require understanding cause and effect, and it'll get them right. So this comment is wrong. "Next token prediction" taken to a certain mathematical limit implies reasoning.

SketchySeaBeast 4y ago
All of them, all the time? Assuming it does fail does it have sound reasoning, or is it nonsense?
- bogdan 4y ago
  Do humans have sound reasoning?
  - tomrod 4y ago
    They have the capacity for it, despite rarely exercising it.
  - SketchySeaBeast 4y ago
    Sometimes. Sometimes they'll come up with entirely novel ways to think through a problem that pushes knowledge forward. Do we have any examples of ChatGPT showing any such capacity?
- qntmfred 4y ago
  It doesn't have to be perfect. What generally matters is its performance relative to various human populations.
- ogogmad 4y ago
  Probably in between. The point of my comment is that depending on how you interpret the description "predicts the next token", something that can do this well enough should be superhumanly clever. The task is "AI Complete". Don't misinterpret me as saying that GPT is superhumanly clever. I'm just saying that something that's sufficiently good at what people think GPT is doing, should be superhuman.
tomrod 4y ago
I appreciate your opinion though I believe your conclusion, that it appears to reason means it reasons, is incorrect. ChatGPT is an implementation of the Chinese Room Thought Experiment.[0] Our pareidolia allows us to ascribe reasoning faculty.[1] But it is far from proven. [0] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room [1] https://en.wikipedia.org/wiki/Pareidolia https://en.wikipedia.org/wiki/Pareidolia
- HPsquared 4y ago
  If it walks like a duck, quacks like a duck...
- canjobear 4y ago
  And indeed, the Chinese Room (viewed as a whole system, not just the guy in it) is doing reasoning.
- rolisz 4y ago
  Sure, but Searle's interpretation of the Chinese Room Experiment was never very convincing to me. I think the room + the person inside it, as a system, do speak Chinese.
- WindyCityBrew 4y ago
  At the end of the day, when something looks like a duck, walks like a duck, quacks like a duck; it doesn't matter how much the nitty-gritty details prove the thing is absolutely not "actually" a duck to the average person.
  - WA 4y ago
    Same logic: You write like a bot, you sound like a bot, you behave like a bot, hence, who cares if you are a bot or not? Let's ban you. For this whole discussion, it actually really does matter if GPT-4 is able to reason or not. That is the essence of the discussion. It's not whether or not it looks like it can reason.
  - pwinnski 4y ago
akiselev 4y ago
Try this puzzle: https://news.ycombinator.com/item?id=35155467 https://news.ycombinator.com/item?id=35155467 > Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across? I just asked ChatGPT v4 this question, assuming it would have been fixed by RHLF training but it spat out step 1 as “First, take the goat across the river and leave it on the other side,” leaving the lion and cabbage alone against explicit directions in the riddle. It clearly can’t reason, just produce a convincing imitation most of the time
- dougmwne 4y ago
  I suspect the reasoning is getting overwhelmed by the prior data. Try it again, but this time make up nonsense words for the three items.
  - AnimalMuppet 4y ago
    > I suspect the reasoning is getting overwhelmed by the prior data. No, the reasoning is the prior data. Thought experiment: GPT-4, but the training data contains no puzzles of this kind. How well would it do? > Try it again, but this time make up nonsense words for the three items. Seconded. I also would like to see the results of this experiment.
    - pwinnski 4y ago
      I did this with Chat-GPT, public March 14 version at chat.openai.com: >> Suppose I have a blurple, a mantiple and a hoje, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the blurple and hoje alone together, and I am not allowed to leave the hoje and mantiple alone together. How can I safely get all three across? > This is a classic river crossing puzzle. Here's one possible solution: > 1. Take the blurple across the river, leaving the mantiple and hoje behind. > 2. Leave the blurple on the other side of the river and return to the original side. > 3. Take the hoje across the river and leave it there with the blurple. > 4. Take the mantiple across the river and leave it with the hoje. > 5. Return to the original side of the river and retrieve the blurple. > 6. Finally, take the blurple across the river to join the mantiple and hoje on the other side. > This solution ensures that no two items that cannot be left alone together are ever left alone together. Of course, my second stipulation was "I am not allowed to leave the hoje and mantiple alone together" and yet step 1 says "leaving the mantiple and hoje behind" So no, despite using the words "classic river crossing puzzle," it clearly expressed no understanding of the puzzle.
HybridCurve 4y ago
We can assess this using verbal, logical and arithmetic reasoning tests. I gave ChatGPT 5 questions from a verbal reasoning test on sequence of words and it got 2/5 right. Here is sample question: Arrange the words given below in a meaningful sequence. 1. Poverty 2. Population 3. Death 4. Unemployment 5. Disease A) 2, 3, 4, 5, 1 B) 3, 4, 2, 5, 1 C) 2, 4, 1, 5, 3 D) 1, 2, 3, 4, 5 ChatGPT> B) 3, 4, 2, 5, 1 Unemployment can lead to poverty, which can lead to disease. Disease can lead to death. The population of an area is affected by all of these factors. Therefore, the correct order is 3, 4, 2, 5, 1. Correct answer was C I would expect if it was effective at reasoning it should be strongest on verbal associations. And while it is possible that enhancements to future language models might provided them with better reasoning ability I don't believe you can simply push the existing design to that limit. The poor results from the verbal reasoning tests demonstrate that reasoning is more complicated than what a transformer based language model is capable of. Overall, the argument is moot because the statement made in the article was it was a reasoning engine which is not equivalent to being capable of reasoning.
- tomrod 4y ago
  Exactly. "Machined" means a precise implementation. A machined reasoning engine would be precise, accurate, and consistent. GPT-4 is impressive autocomplete, but not what we would expect for machined reasoning.
- vacuumcl 4y ago
  I just put this into ChatGPT4, and it gave C as the answer.

SketchySeaBeast 4y ago

All of them, all the time? Assuming it does fail does it have sound reasoning, or is it nonsense?

bogdan 4y ago
Do humans have sound reasoning?
- tomrod 4y ago
  They have the capacity for it, despite rarely exercising it.
- SketchySeaBeast 4y ago
  Sometimes. Sometimes they'll come up with entirely novel ways to think through a problem that pushes knowledge forward. Do we have any examples of ChatGPT showing any such capacity?
qntmfred 4y ago
It doesn't have to be perfect. What generally matters is its performance relative to various human populations.
ogogmad 4y ago
Probably in between. The point of my comment is that depending on how you interpret the description "predicts the next token", something that can do this well enough should be superhumanly clever. The task is "AI Complete". Don't misinterpret me as saying that GPT is superhumanly clever. I'm just saying that something that's sufficiently good at what people think GPT is doing, should be superhuman.

tomrod 4y ago

I appreciate your opinion though I believe your conclusion, that it appears to reason means it reasons, is incorrect. ChatGPT is an implementation of the Chinese Room Thought Experiment.[0] Our pareidolia allows us to ascribe reasoning faculty.[1] But it is far from proven. [0] https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room [1] https://en.wikipedia.org/wiki/Pareidolia https://en.wikipedia.org/wiki/Pareidolia

HPsquared 4y ago
If it walks like a duck, quacks like a duck...
canjobear 4y ago
And indeed, the Chinese Room (viewed as a whole system, not just the guy in it) is doing reasoning.
rolisz 4y ago
Sure, but Searle's interpretation of the Chinese Room Experiment was never very convincing to me. I think the room + the person inside it, as a system, do speak Chinese.
WindyCityBrew 4y ago
At the end of the day, when something looks like a duck, walks like a duck, quacks like a duck; it doesn't matter how much the nitty-gritty details prove the thing is absolutely not "actually" a duck to the average person.
- WA 4y ago
  Same logic: You write like a bot, you sound like a bot, you behave like a bot, hence, who cares if you are a bot or not? Let's ban you. For this whole discussion, it actually really does matter if GPT-4 is able to reason or not. That is the essence of the discussion. It's not whether or not it looks like it can reason.
- pwinnski 4y ago

akiselev 4y ago

Try this puzzle: https://news.ycombinator.com/item?id=35155467 https://news.ycombinator.com/item?id=35155467 > Suppose I have a cabbage, a goat and a lion, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the cabbage and lion alone together, and I am not allowed to leave the lion and goat alone together. How can I safely get all three across? I just asked ChatGPT v4 this question, assuming it would have been fixed by RHLF training but it spat out step 1 as “First, take the goat across the river and leave it on the other side,” leaving the lion and cabbage alone against explicit directions in the riddle. It clearly can’t reason, just produce a convincing imitation most of the time

dougmwne 4y ago
I suspect the reasoning is getting overwhelmed by the prior data. Try it again, but this time make up nonsense words for the three items.
- AnimalMuppet 4y ago
  > I suspect the reasoning is getting overwhelmed by the prior data. No, the reasoning is the prior data. Thought experiment: GPT-4, but the training data contains no puzzles of this kind. How well would it do? > Try it again, but this time make up nonsense words for the three items. Seconded. I also would like to see the results of this experiment.
  - pwinnski 4y ago
    I did this with Chat-GPT, public March 14 version at chat.openai.com: >> Suppose I have a blurple, a mantiple and a hoje, and I need to get them across a river. I have a boat that can only carry myself and a single other item. I am not allowed to leave the blurple and hoje alone together, and I am not allowed to leave the hoje and mantiple alone together. How can I safely get all three across? > This is a classic river crossing puzzle. Here's one possible solution: > 1. Take the blurple across the river, leaving the mantiple and hoje behind. > 2. Leave the blurple on the other side of the river and return to the original side. > 3. Take the hoje across the river and leave it there with the blurple. > 4. Take the mantiple across the river and leave it with the hoje. > 5. Return to the original side of the river and retrieve the blurple. > 6. Finally, take the blurple across the river to join the mantiple and hoje on the other side. > This solution ensures that no two items that cannot be left alone together are ever left alone together. Of course, my second stipulation was "I am not allowed to leave the hoje and mantiple alone together" and yet step 1 says "leaving the mantiple and hoje behind" So no, despite using the words "classic river crossing puzzle," it clearly expressed no understanding of the puzzle.

HybridCurve 4y ago

We can assess this using verbal, logical and arithmetic reasoning tests. I gave ChatGPT 5 questions from a verbal reasoning test on sequence of words and it got 2/5 right. Here is sample question: Arrange the words given below in a meaningful sequence. 1. Poverty 2. Population 3. Death 4. Unemployment 5. Disease A) 2, 3, 4, 5, 1 B) 3, 4, 2, 5, 1 C) 2, 4, 1, 5, 3 D) 1, 2, 3, 4, 5 ChatGPT> B) 3, 4, 2, 5, 1 Unemployment can lead to poverty, which can lead to disease. Disease can lead to death. The population of an area is affected by all of these factors. Therefore, the correct order is 3, 4, 2, 5, 1. Correct answer was C I would expect if it was effective at reasoning it should be strongest on verbal associations. And while it is possible that enhancements to future language models might provided them with better reasoning ability I don't believe you can simply push the existing design to that limit. The poor results from the verbal reasoning tests demonstrate that reasoning is more complicated than what a transformer based language model is capable of. Overall, the argument is moot because the statement made in the article was it was a reasoning engine which is not equivalent to being capable of reasoning.

tomrod 4y ago
Exactly. "Machined" means a precise implementation. A machined reasoning engine would be precise, accurate, and consistent. GPT-4 is impressive autocomplete, but not what we would expect for machined reasoning.
vacuumcl 4y ago
I just put this into ChatGPT4, and it gave C as the answer.