6 ms·

>i'm curious if you still have this opinion Largely. I'm not surprised that another model could be trained on a larger chess corpus or otherwise tuned for impr

>i'm curious if you still have this opinion Largely. I'm not surprised that another model could be trained on a larger chess corpus or otherwise tuned for improvement. I actually alluded to that in my original comment above. In fact, I was really surprised that gpt4 was so bad, given the total volume of training data it had obviously ingested. So, my initial point was around how that revealed the disparity between LLMs appearing to understand versus actually understanding (i.e. engaging in reasoning).

famouswaffles 3y ago
If i came upon a random stranger who did not know how to play chess (but otherwise had heard of the game) and i put the board in front of him, made my move and told him to proceed, he will almost certainly move a piece to some other position on the board. There is no difference between this exercise and the one you have given GPT-4. In fact, the results are manifested the exact same way. By at most a few moves in, i will notice that both the stranger and GPT-4 are moving pieces to some general area of the board but woefully failing to adhere to the rules of the game. "The stranger can not reason" would be an absurd conclusion. Swapping out The stranger for GPT-4 does not make it any less so. So really by this standard then Humans regularly only "appear to understand" a great number of things. I can get behind this, but I doubt this is your conclusion. It seems to me that you were using "appear to understand" and "understand" as something that separates Humans from LLMs.
- bongodongobob 3y ago
  This is a fantastic argument. Great food for thought.
- unclebucknasty 3y ago
  Except that the premise of ChatGPT is that it "knows" things, and that you can ask it about these things in specific ways and it will answer. Further, that knowledge represents a vast body of training data that certainly includes the rules of chess, how pieces move, etc. This is verifiable. Go ask ChatGPT those questions now. How does a bishop or rook move? Can a piece move through or over another piece, etc. What is en passant? When can it be employed? You will find that it answers these correctly, and even provides examples. It talks about ranks and files, and authoritatively sprinkles in other knowledge related to the questions. But, then you put a board in front of it and you ask it to make moves, and it doesn't adhere to these rules. You're not asking it to win the game. Just make a legal move. But it cannot, even while confidently responding with invalid moves. So, it's not about whether it's been tuned to be a grandmaster. It's about whether it can even execute on the knowledge it clearly possesses. Now, turning back to your analogy with a human player. You ask Gary all of these questions and he gives you the answers perfectly, so you recognize that his factual knowledge is intact. But, despite his best efforts, he repeatedly makes illegal moves. You would have to conclude that there is some problem with reasoning or understanding what those facts actually mean in practice. That is the difference between appearing to understand and actually understanding. And certainly, there are some humans who fit this "appearance" description as well, perhaps through some processing deficit or otherwise. But as a broad category, this is something humans do quite well. The concept of understanding is really not complicated. When we explain something to someone and we ask the question "do you understand?", we're not asking whether they can memorize or repeat back what we just told them. We're asking something else. That something else is the well-understood (irony intended) meaning of "understand".
  - famouswaffles 3y ago
    >Except that the premise of ChatGPT is that it "knows" things, and that you can ask it about these things in specific ways and it will answer. The premise of ChatGPT has never been that it knows everything. >So, it's not about whether it's been tuned to be a grandmaster. It's about whether it can even execute on the knowledge it clearly possesses. Factual knowledge on paper has never been a guarantee you can actually perform the task. In fact, it usually isn't. >And certainly, there are some humans who fit this "appearance" description as well, perhaps through some processing deficit or otherwise. But as a broad category, this is something humans do quite well. I'm telling you this is a normal occurrence and not something that only happens with a "processing deficit". You can rant about all the details or rules of a tennis game all you like and fail to play it, you can know the physics theory but still struggle to solve problems. Hell we don't need to leave chess, give our stranger a proper book with rules and the strategies employed by grandmasters. For a while after, He's still going to make illegal moves and he will be nowhere near the level of a grandmaster. >Again you are still not bringing up anything that serves as the distinguisher you want it to serve. The concept of understanding is really not complicated. When we explain something to someone and we ask the question "do you understand?", we're not asking whether they can memorize or repeat back what we just told them. We're asking something else. Ok and ? GPT passes many tests that demonstrate understanding.
    - unclebucknasty 3y ago
      >You can rant about all the details or rules of a tennis game all you like and fail to play it This analogy reveals what you're missing. The correct analogy is not that you know the rules, but "fail to play" tennis (which requires physical aptitude, etc). It's that you can cite the rules, but actually try to hit the ball over the fence and claim a point when you succeed. Likewise, the stranger with the chess rule/strategies book. You're conflating quality with understanding. But, I've not claimed that all humans excel at or even understand all things. And, you've still not defined the gap between GPT being able to cite the rules of chess, yet not being able to apply them. >GPT passes many tests that demonstrate understanding. It actually does not. It's just better at making you believe it does when it simulates understanding in those tests.

famouswaffles 3y ago

If i came upon a random stranger who did not know how to play chess (but otherwise had heard of the game) and i put the board in front of him, made my move and told him to proceed, he will almost certainly move a piece to some other position on the board. There is no difference between this exercise and the one you have given GPT-4. In fact, the results are manifested the exact same way. By at most a few moves in, i will notice that both the stranger and GPT-4 are moving pieces to some general area of the board but woefully failing to adhere to the rules of the game. "The stranger can not reason" would be an absurd conclusion. Swapping out The stranger for GPT-4 does not make it any less so. So really by this standard then Humans regularly only "appear to understand" a great number of things. I can get behind this, but I doubt this is your conclusion. It seems to me that you were using "appear to understand" and "understand" as something that separates Humans from LLMs.

bongodongobob 3y ago
This is a fantastic argument. Great food for thought.
unclebucknasty 3y ago
Except that the premise of ChatGPT is that it "knows" things, and that you can ask it about these things in specific ways and it will answer. Further, that knowledge represents a vast body of training data that certainly includes the rules of chess, how pieces move, etc. This is verifiable. Go ask ChatGPT those questions now. How does a bishop or rook move? Can a piece move through or over another piece, etc. What is en passant? When can it be employed? You will find that it answers these correctly, and even provides examples. It talks about ranks and files, and authoritatively sprinkles in other knowledge related to the questions. But, then you put a board in front of it and you ask it to make moves, and it doesn't adhere to these rules. You're not asking it to win the game. Just make a legal move. But it cannot, even while confidently responding with invalid moves. So, it's not about whether it's been tuned to be a grandmaster. It's about whether it can even execute on the knowledge it clearly possesses. Now, turning back to your analogy with a human player. You ask Gary all of these questions and he gives you the answers perfectly, so you recognize that his factual knowledge is intact. But, despite his best efforts, he repeatedly makes illegal moves. You would have to conclude that there is some problem with reasoning or understanding what those facts actually mean in practice. That is the difference between appearing to understand and actually understanding. And certainly, there are some humans who fit this "appearance" description as well, perhaps through some processing deficit or otherwise. But as a broad category, this is something humans do quite well. The concept of understanding is really not complicated. When we explain something to someone and we ask the question "do you understand?", we're not asking whether they can memorize or repeat back what we just told them. We're asking something else. That something else is the well-understood (irony intended) meaning of "understand".
- famouswaffles 3y ago
  >Except that the premise of ChatGPT is that it "knows" things, and that you can ask it about these things in specific ways and it will answer. The premise of ChatGPT has never been that it knows everything. >So, it's not about whether it's been tuned to be a grandmaster. It's about whether it can even execute on the knowledge it clearly possesses. Factual knowledge on paper has never been a guarantee you can actually perform the task. In fact, it usually isn't. >And certainly, there are some humans who fit this "appearance" description as well, perhaps through some processing deficit or otherwise. But as a broad category, this is something humans do quite well. I'm telling you this is a normal occurrence and not something that only happens with a "processing deficit". You can rant about all the details or rules of a tennis game all you like and fail to play it, you can know the physics theory but still struggle to solve problems. Hell we don't need to leave chess, give our stranger a proper book with rules and the strategies employed by grandmasters. For a while after, He's still going to make illegal moves and he will be nowhere near the level of a grandmaster. >Again you are still not bringing up anything that serves as the distinguisher you want it to serve. The concept of understanding is really not complicated. When we explain something to someone and we ask the question "do you understand?", we're not asking whether they can memorize or repeat back what we just told them. We're asking something else. Ok and ? GPT passes many tests that demonstrate understanding.
  - unclebucknasty 3y ago
    >You can rant about all the details or rules of a tennis game all you like and fail to play it This analogy reveals what you're missing. The correct analogy is not that you know the rules, but "fail to play" tennis (which requires physical aptitude, etc). It's that you can cite the rules, but actually try to hit the ball over the fence and claim a point when you succeed. Likewise, the stranger with the chess rule/strategies book. You're conflating quality with understanding. But, I've not claimed that all humans excel at or even understand all things. And, you've still not defined the gap between GPT being able to cite the rules of chess, yet not being able to apply them. >GPT passes many tests that demonstrate understanding. It actually does not. It's just better at making you believe it does when it simulates understanding in those tests.