7 ms·

Grandmasters do not make an illegal move every 300 moves. That is noticeably higher than what we would expect, and it should give us pause when evaluating how 3

Grandmasters do not make an illegal move every 300 moves. That is noticeably higher than what we would expect, and it should give us pause when evaluating how 3.5-turbo works and how it's evaluating chess moves. If an 1800 elo human did that, we would find the behavior to be odd and we would question how they learned to play and what their decision-making process was when making moves. "Thing happening commonly in scenario X can also rarely happen in scenario Y, therefore both scenarios are exactly the same" is one of the more annoyingly reductive arguments to have come out of recent AI discussions. You have to actually take frequency into account when you're talking about this stuff, and events occurring at abnormal or atypical frequencies can communicate information about the world.

famouswaffles 3y ago
>Grandmasters do not make an illegal move every 300 moves. Sure. Point is that even the best players will make an illegal move (even if rarely) so a computer that makes an illegal move is not grounds for "does not understand chess" just because it doesn't always make a legal move. He says, "The fact that it’s making illegal moves at all should tell you something." He is literally making the assertion that a single instance of an illegal move means you don't understanding how to play the game. This is trivially proven false. >If an 1800 elo human did that, we would find the behavior to be odd and we would question how they learned to play and what their decision-making process was when making moves. Yes because humans usually learn the rules first then play. If a human learnt chess from watching games, i would not find this behavior odd at all. In fact, i would expect it. Just because you've never seen a move played before does not mean the move is illegal. Moreover, odd as it may be, I still wouldn't call it "struggling to make a legal move". >"Thing happening commonly in scenario X can also rarely happen in scenario Y, therefore both scenarios are exactly the same" is one of the more annoyingly reductive arguments to have come out of recent AI discussions. That wasn't the argument i was making.
- danShumway 3y ago
  > Sure. Point is that even the best players will make an illegal move (even if rarely) so a computer that makes an illegal move is not grounds for "does not understand chess" just because it doesn't always make a legal move. You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. An analogy: sometimes cell phone batteries explode. If 1/300 cellphone batteries for a specific brand start exploding, you would not say, "that's not proof of anything, all cell phone batteries have a risk of exploding." You would say, "huh, something is probably going very wrong at the factory." Observing an abnormally high rate of something that should be much more uncommon is actually evidence that something could be going wrong with the underlying process. Observing that a computer has an abnormally high rates of illegal moves is evidence that it struggles to actually understand the rules of chess and that it might be imitating play instead of understanding it. > Yes because humans usually learn the rules first then play. That feels like you're agreeing with the original poster? The computer is imitating play that it's seen before, it's not starting from a baseline understanding of how chess works or why those moves are being made. I would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person.
  - famouswaffles 3y ago
    >You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. Anything can be called a mistake if you ignore crucial context. I replied to someone else addressing very specific points he/she brought up. >would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person. A GPT grandmaster would make even less mistakes. That aside because I understand it's not your main point, again this is more nuance than who I was replying to. You have said "does not have as complete an understanding" where the person I replied to said "does not understand the game at all". I feel like you want to argue against a point that was not present in the original discussion I was having. >That feels like you're agreeing with the original poster? The original poster is lacking the nuance to have me agree with him.
    - danShumway 3y ago
      We're kind of splitting straws here. Whether GPT understands some of the rules of chess or whether it's completely relying on modulation between states, or whether it's using some other strategy -- it seems to be pretty obviously approaching chess using a different strategy than a human being would. And I think that's the main point that janalsncm was getting at -- GPT thinks about the world and thinks about predictions and thinks about subjects like chess differently than a human would. It does not appear to "understand" chess in the same way that a human does when approaching the game. Of course, you can have different definitions of understanding, GPT might be looking at chess more as a series of patterns or thinking about the shape of a game in a way that a human might not. But the value in understanding the distinction is that GPT is going to be good at some things that a human is bad at, and bad at some things that a human is good at, and understanding that it approaches tasks differently than humans is key to understanding why GPT struggles or succeeds at certain tasks. I'm not going to run around defending other people's comments but for whatever it is worth, I don't think janalsncm's comment was unreasonable or lacked nuance. It didn't go out of its way to clarify that GPT isn't useless, it didn't go out of its way to say that it's still impressive that a language model can get the performance that it does get, but the basic idea of "hey, this is acting different than a human and maybe it shouldn't be directly compared to a human, and the deviations in its behavior should tell us something about the model" is sound. It's not just sound, it's also useful advice for people to internalize. On a purely practical level, you will get much better results out of GPT if you modify your prompts and approaches to cater to its strengths rather than its weaknesses. To do that, you have to understand what the weaknesses and strengths are and you have to understand that imitation of human behavior does not necessarily imply that GPT is approaching or understanding those tasks in remotely the same way that a human being would. It is useful information for prompting GPT and you will get better results out of GPT if you understand that it's very often not approaching tasks from first principles. And chess is frankly a really good example of that, it's a great example of a task where the end result initially suggests that humans and LLMs are approaching a task identically, and then upon further examination reveals that in all likelyhood, the model does not have the same first-principles or understanding of the rulesets that a human player does. It's striking because for a human to get to the same level of play they would need to have a fundamental understanding of the rulesets, and GPT is able to get to that level of play seemingly without building that level of understanding. That is an interesting thing to learn about GPT.
- janalsncm 3y ago
  Are you suggesting that ChatGPT only had access to examples of games, and not any of the rules of the game? ChatGPT has been exposed to the entire internet. I’m sure the rules of the game are on there somewhere. I’m also a bit curious as to how far you’re willing to take your “illegal moves don’t indicate lack of understanding” logic. What if it was making illegal moves 1 out of 10 times? What about 1 out of 2 times? Even if it made illegal moves 80% of the time, 20% is still better than randomly picking up pieces and putting them down. So maybe this could count as “understanding” too.
  - famouswaffles 3y ago
    GPT doesn't read text. It tries to predict it. Gradient descent prediction is just a dumb optimizer like evolution. Generalization from it, like evolution is broad and sweeping rather than exact and pinpoint. Understanding is a spectrum. Yes if it is making moves above random chance, it has a little level of understanding.

famouswaffles 3y ago

>Grandmasters do not make an illegal move every 300 moves. Sure. Point is that even the best players will make an illegal move (even if rarely) so a computer that makes an illegal move is not grounds for "does not understand chess" just because it doesn't always make a legal move. He says, "The fact that it’s making illegal moves at all should tell you something." He is literally making the assertion that a single instance of an illegal move means you don't understanding how to play the game. This is trivially proven false. >If an 1800 elo human did that, we would find the behavior to be odd and we would question how they learned to play and what their decision-making process was when making moves. Yes because humans usually learn the rules first then play. If a human learnt chess from watching games, i would not find this behavior odd at all. In fact, i would expect it. Just because you've never seen a move played before does not mean the move is illegal. Moreover, odd as it may be, I still wouldn't call it "struggling to make a legal move". >"Thing happening commonly in scenario X can also rarely happen in scenario Y, therefore both scenarios are exactly the same" is one of the more annoyingly reductive arguments to have come out of recent AI discussions. That wasn't the argument i was making.

danShumway 3y ago
> Sure. Point is that even the best players will make an illegal move (even if rarely) so a computer that makes an illegal move is not grounds for "does not understand chess" just because it doesn't always make a legal move. You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. An analogy: sometimes cell phone batteries explode. If 1/300 cellphone batteries for a specific brand start exploding, you would not say, "that's not proof of anything, all cell phone batteries have a risk of exploding." You would say, "huh, something is probably going very wrong at the factory." Observing an abnormally high rate of something that should be much more uncommon is actually evidence that something could be going wrong with the underlying process. Observing that a computer has an abnormally high rates of illegal moves is evidence that it struggles to actually understand the rules of chess and that it might be imitating play instead of understanding it. > Yes because humans usually learn the rules first then play. That feels like you're agreeing with the original poster? The computer is imitating play that it's seen before, it's not starting from a baseline understanding of how chess works or why those moves are being made. I would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person.
- famouswaffles 3y ago
  >You're just making the exact same mistake a second time in a row. It is evidence; maybe not full on undeniable 100% proof, but it should shift your priors about how you think about AI, and if it doesn't I would say you're ignoring obvious statistical evidence. Anything can be called a mistake if you ignore crucial context. I replied to someone else addressing very specific points he/she brought up. >would say a human grandmaster who regularly makes illegal moves at a rate of 1/300 does not have as deep or complete of an understanding of the game as other grandmasters who do not make those errors at nearly the same level or frequency. I think that's what most people would say about that person. A GPT grandmaster would make even less mistakes. That aside because I understand it's not your main point, again this is more nuance than who I was replying to. You have said "does not have as complete an understanding" where the person I replied to said "does not understand the game at all". I feel like you want to argue against a point that was not present in the original discussion I was having. >That feels like you're agreeing with the original poster? The original poster is lacking the nuance to have me agree with him.
  - danShumway 3y ago
    We're kind of splitting straws here. Whether GPT understands some of the rules of chess or whether it's completely relying on modulation between states, or whether it's using some other strategy -- it seems to be pretty obviously approaching chess using a different strategy than a human being would. And I think that's the main point that janalsncm was getting at -- GPT thinks about the world and thinks about predictions and thinks about subjects like chess differently than a human would. It does not appear to "understand" chess in the same way that a human does when approaching the game. Of course, you can have different definitions of understanding, GPT might be looking at chess more as a series of patterns or thinking about the shape of a game in a way that a human might not. But the value in understanding the distinction is that GPT is going to be good at some things that a human is bad at, and bad at some things that a human is good at, and understanding that it approaches tasks differently than humans is key to understanding why GPT struggles or succeeds at certain tasks. I'm not going to run around defending other people's comments but for whatever it is worth, I don't think janalsncm's comment was unreasonable or lacked nuance. It didn't go out of its way to clarify that GPT isn't useless, it didn't go out of its way to say that it's still impressive that a language model can get the performance that it does get, but the basic idea of "hey, this is acting different than a human and maybe it shouldn't be directly compared to a human, and the deviations in its behavior should tell us something about the model" is sound. It's not just sound, it's also useful advice for people to internalize. On a purely practical level, you will get much better results out of GPT if you modify your prompts and approaches to cater to its strengths rather than its weaknesses. To do that, you have to understand what the weaknesses and strengths are and you have to understand that imitation of human behavior does not necessarily imply that GPT is approaching or understanding those tasks in remotely the same way that a human being would. It is useful information for prompting GPT and you will get better results out of GPT if you understand that it's very often not approaching tasks from first principles. And chess is frankly a really good example of that, it's a great example of a task where the end result initially suggests that humans and LLMs are approaching a task identically, and then upon further examination reveals that in all likelyhood, the model does not have the same first-principles or understanding of the rulesets that a human player does. It's striking because for a human to get to the same level of play they would need to have a fundamental understanding of the rulesets, and GPT is able to get to that level of play seemingly without building that level of understanding. That is an interesting thing to learn about GPT.
janalsncm 3y ago
Are you suggesting that ChatGPT only had access to examples of games, and not any of the rules of the game? ChatGPT has been exposed to the entire internet. I’m sure the rules of the game are on there somewhere. I’m also a bit curious as to how far you’re willing to take your “illegal moves don’t indicate lack of understanding” logic. What if it was making illegal moves 1 out of 10 times? What about 1 out of 2 times? Even if it made illegal moves 80% of the time, 20% is still better than randomly picking up pieces and putting them down. So maybe this could count as “understanding” too.
- famouswaffles 3y ago
  GPT doesn't read text. It tries to predict it. Gradient descent prediction is just a dumb optimizer like evolution. Generalization from it, like evolution is broad and sweeping rather than exact and pinpoint. Understanding is a spectrum. Yes if it is making moves above random chance, it has a little level of understanding.