4 ms·

I don't know for Q* of course, but all the tests I made with GPT4, and all what I've read and seen about it, show that it is unable to reason. It was trained wi

I don't know for Q* of course, but all the tests I made with GPT4, and all what I've read and seen about it, show that it is unable to reason. It was trained with an unfathomable amount of data, so it can simulate reasoning very well, but it is unable to reason

oezi 3y ago
What is the difference between simulating reasoning very well and "actual" reasoning?
- parentheses 3y ago
  I think the poster meant that it's capable of having a high probability of correct reasoning - simulating reasoning is lossy, actual reasoning is not. Though, human reasoning is still lossy.
- silvaring 3y ago
  Actual reasoning is made up of various biological feedback loops that happen in the body and brain, essentially your physical senses give you the ability to reason in the first place, without the eyes, ears etc there is no ability to learn basic reasoning, which is why kids who are blind or mute from birth have huge issues learning about object permanence, spatial awaraness etc. You cant expect human reasoning without human perception. My question is how does the AI perceive. Basically how good is the simulation for its perception. If we know that, then we can probably assess its ability to reason because we can compare it to the closest benchmark we have (your average human being). How do AI's see, how did they learn concepts in strings of words and pixels? How does the concept it learnt in text carry through to images of colors, of shapes? Does it show a transfer of conceptual understanding across both two and three dimentional shapes? I know these are more questions than answers, but its just things that I've been wondering about.
  - ajuc 3y ago
    This ship can't swim because only living creatures swim. It's true but it only shows your definition sucks.
- Jare 3y ago
  Probably best to say "simulate the appearance of reasoning": looks and feels 100% acceptable at a surface level, but the actual details and conclusions are completely wrong / do not follow.
- seanhunter 3y ago
  Actual reasoning shows the understanding and use of a model of the key features of the underlying problem/domain. As a simple example that you can replicate using chatgpt, ask it to solve some simple maths problem. Very frequently you will get a solution that looks like reasoning but is not, and reveals that it does not have an actual model of the underlying maths but is in fact doing text prediction based on a history of maths. For example see here[1]. I ask it for some quadratics in x with some specification on the number of roots. It gives me what looks at first glance like a decent answer. Then I ask the same exact question but asking for quadratics in x and y[2]. Again the answer looks plausible except that for the solution "with one real root" it says the solution has one real root when x + y =1. Well there are infinite real values for x and y such that x + y =1, not one real root. It looks like it has solved the problem but instead it has simulated the solving of the problem. Likewise stacking problems, used to check for whether an AI has a model of the world. This is covered in "From task structures to world models: What do LLMs know?"[3] but for example here[4] I ask it whether it's easier to balance a barrel on a plank or a plank on a barrel. The model says it's easier to balance a plank on a barrel with an output text that simulates reasoning discussing center of mass and the difference between the flatness of the plank and the tendency of the barrel to roll because of its curvature. Actual reasoning would say to put the barrel on its end so it doesn't roll (whether you put the plank on top or not). [1] https://chat.openai.com/share/64556be8-ad20-41aa-99af-ed5a42201517 https://chat.openai.com/share/64556be8-ad20-41aa-99af-ed5a42... [2] https://chat.openai.com/share/2cd39197-dc09-4d07-a0d6-6cd800a08b80 https://chat.openai.com/share/2cd39197-dc09-4d07-a0d6-6cd800... [3] https://arxiv.org/abs/2310.04276 https://arxiv.org/abs/2310.04276 [4] https://chat.openai.com/share/4b631a92-0d55-4ae5-8892-9be02559e185 https://chat.openai.com/share/4b631a92-0d55-4ae5-8892-9be025...
  - chipsambos 3y ago
    I generally agree with what you're saying and the first half of your answer makes perfect sense but I think the second is unfair (i.e. "[is it] easier to balance a barrel on a plank or a plank on a barrel"). It's a trick question and "it" tried to answer in good faith. If you were to ask the same question of a real person and they replied with the exact same answer you could not conclude that person was not capable of "actual reasoning". It's a bit of witch-hunt question set to give you the conclusion you want.
- Levitz 3y ago
  Being able to extrapolate with newly found data. You can get a LLM to simulate it "discovering" the pythagorean theorem, but can it actually, with the knowledge that was available at the time, discover the pythagorean theorem by itself? Any parent will tell you, it's easy to simulate discovery and reasoning, it's a trick played for kids all the time. The actual, real stuff, that's way harder.
swombat 3y ago
Similarly, AlphaGo and Stockfish are only able to simulate reasoning their way through a game of Go or a game of Chess. That simulated reasoning is enough to annihilate any human player they're faced with. As Dijkstra famously said, "whether Machines Can Think... is about as relevant as the question of whether Submarines Can Swim". Submarines don't swim, cars don't walk or gallop, cameras don't paint or draw... So what? Once AI can simulate reasoning better than we can do the genuine thing, the question really becomes utterly irrelevant to the likely outcome.
- deleted 3y ago
  [deleted]

oezi 3y ago

What is the difference between simulating reasoning very well and "actual" reasoning?

parentheses 3y ago
I think the poster meant that it's capable of having a high probability of correct reasoning - simulating reasoning is lossy, actual reasoning is not. Though, human reasoning is still lossy.
silvaring 3y ago
Actual reasoning is made up of various biological feedback loops that happen in the body and brain, essentially your physical senses give you the ability to reason in the first place, without the eyes, ears etc there is no ability to learn basic reasoning, which is why kids who are blind or mute from birth have huge issues learning about object permanence, spatial awaraness etc. You cant expect human reasoning without human perception. My question is how does the AI perceive. Basically how good is the simulation for its perception. If we know that, then we can probably assess its ability to reason because we can compare it to the closest benchmark we have (your average human being). How do AI's see, how did they learn concepts in strings of words and pixels? How does the concept it learnt in text carry through to images of colors, of shapes? Does it show a transfer of conceptual understanding across both two and three dimentional shapes? I know these are more questions than answers, but its just things that I've been wondering about.
- ajuc 3y ago
  This ship can't swim because only living creatures swim. It's true but it only shows your definition sucks.
Jare 3y ago
Probably best to say "simulate the appearance of reasoning": looks and feels 100% acceptable at a surface level, but the actual details and conclusions are completely wrong / do not follow.
seanhunter 3y ago
Actual reasoning shows the understanding and use of a model of the key features of the underlying problem/domain. As a simple example that you can replicate using chatgpt, ask it to solve some simple maths problem. Very frequently you will get a solution that looks like reasoning but is not, and reveals that it does not have an actual model of the underlying maths but is in fact doing text prediction based on a history of maths. For example see here[1]. I ask it for some quadratics in x with some specification on the number of roots. It gives me what looks at first glance like a decent answer. Then I ask the same exact question but asking for quadratics in x and y[2]. Again the answer looks plausible except that for the solution "with one real root" it says the solution has one real root when x + y =1. Well there are infinite real values for x and y such that x + y =1, not one real root. It looks like it has solved the problem but instead it has simulated the solving of the problem. Likewise stacking problems, used to check for whether an AI has a model of the world. This is covered in "From task structures to world models: What do LLMs know?"[3] but for example here[4] I ask it whether it's easier to balance a barrel on a plank or a plank on a barrel. The model says it's easier to balance a plank on a barrel with an output text that simulates reasoning discussing center of mass and the difference between the flatness of the plank and the tendency of the barrel to roll because of its curvature. Actual reasoning would say to put the barrel on its end so it doesn't roll (whether you put the plank on top or not). [1] https://chat.openai.com/share/64556be8-ad20-41aa-99af-ed5a42201517 https://chat.openai.com/share/64556be8-ad20-41aa-99af-ed5a42... [2] https://chat.openai.com/share/2cd39197-dc09-4d07-a0d6-6cd800a08b80 https://chat.openai.com/share/2cd39197-dc09-4d07-a0d6-6cd800... [3] https://arxiv.org/abs/2310.04276 https://arxiv.org/abs/2310.04276 [4] https://chat.openai.com/share/4b631a92-0d55-4ae5-8892-9be02559e185 https://chat.openai.com/share/4b631a92-0d55-4ae5-8892-9be025...
- chipsambos 3y ago
  I generally agree with what you're saying and the first half of your answer makes perfect sense but I think the second is unfair (i.e. "[is it] easier to balance a barrel on a plank or a plank on a barrel"). It's a trick question and "it" tried to answer in good faith. If you were to ask the same question of a real person and they replied with the exact same answer you could not conclude that person was not capable of "actual reasoning". It's a bit of witch-hunt question set to give you the conclusion you want.
Levitz 3y ago
Being able to extrapolate with newly found data. You can get a LLM to simulate it "discovering" the pythagorean theorem, but can it actually, with the knowledge that was available at the time, discover the pythagorean theorem by itself? Any parent will tell you, it's easy to simulate discovery and reasoning, it's a trick played for kids all the time. The actual, real stuff, that's way harder.

swombat 3y ago

Similarly, AlphaGo and Stockfish are only able to simulate reasoning their way through a game of Go or a game of Chess. That simulated reasoning is enough to annihilate any human player they're faced with. As Dijkstra famously said, "whether Machines Can Think... is about as relevant as the question of whether Submarines Can Swim". Submarines don't swim, cars don't walk or gallop, cameras don't paint or draw... So what? Once AI can simulate reasoning better than we can do the genuine thing, the question really becomes utterly irrelevant to the likely outcome.

deleted 3y ago
[deleted]