6 ms·

Most likely it has seen a similar sequence of moves in its training set. There are numerous chess sites with databases displayed in the form of web pages with m

Most likely it has seen a similar sequence of moves in its training set. There are numerous chess sites with databases displayed in the form of web pages with millions of games in them. If it had any understanding of chess, it would never play an illegal move. It's not surprising that given a sequence of algebraic notation it can regurgitate the next move in a similar sequence of algebraic notation.

circuit10 4y ago
I wouldn’t say it has no understanding whatsoever, it probably just doesn’t have the capability to keep track of everything that happened all the time
- dwmbt 4y ago
  that's also not entirely true either. it's prone to make illegal moves. one could argue abiding by the rules of the game is a requisite for understanding!
- imtringued 4y ago
  When I ask it to display moves in an ASCII board it messes up. I doubt that having a representation of the board is the key to improving its chess skills.
SanderNL 4y ago
You don’t get to 1400 like that. The amount of moves it has to literally remember is stupendous.
- WoodenChair 4y ago
  Nobody who is 1400 plays outright illegal moves.
  - SanderNL 4y ago
    Humans don’t, but that is an unnecessary bar IMO.
    - WoodenChair 4y ago
      > Humans don’t, but that is an unnecessary bar IMO. People are always telling me that I'm moving the goalposts when I challenge the hyperbole about LLMs. But now you're moving the goalposts about chess. Not playing illegal moves is a pre-requisite for any strong understanding of how to play chess. That is definitely the goal post.
      - scj 4y ago
        If a two year old had an ELO of 1400 but sometimes took a nap halfway through a game, it'd be a prodigy. It's not like an AI making silly mistakes when driving a car.
  - mrbungie 4y ago
    Yet it achieves 1400. Add hard rules to stop it spewing out said moves and you have a 1400 ELO Player (most UIs won't even let you make illegal moves). It is difficult to say that is not impressive due to it being an emergent ability.
    - WoodenChair 4y ago
      > It is difficult to say that is not impressive due to it being an emergent ability. I don't know why you think it's an emergent ability. It's seeing a sequence of moves, and playing the most likely next move (i.e. the most likely next token) given the previous complete move sequences it was trained on. That's the baseline of what an LLM does—not something emergent. Games in online chess databases tend to be of relatively good players. Nobody wants to look up games played by two 800 ELO players. As an aside, there have been chess programs for years that show you for a given position all of the previous games in its database with the same position and the win outcome % of each move. That's all that's going on here.
      - oezi 4y ago
        > here have been chess programs for years that show you for a given position all of the previous games in its database with the same position and the win outcome % of each move. That's all that's going on here. It could be, but would you think that of the 100-300 bn parameters in the model a lot are dedicated to chess move sequences? It seems likely that it has seen such data, but I would be surprised if it is using a considerable chunk to store chess database information.
        Jensson 4y ago
        The web has millions of grandmaster chess games and probably billions of chess games overall. So I wouldn't be surprised if it has like 0.01% to chess games since there are so many, if so that would mean it has 10-30 million parameters to play chess with, for comparison stockfish has 10 million parameters to its chess engine.
      - mrbungie 4y ago
  - vanviegen 4y ago
    Does that still hold when the player doesn't have a board in front of them, but just a list of previous moves?
  - mtlmtlmtlmtl 4y ago
    I have seen 1400 FIDE elo(much much stronger than chess.com 1400) players make illegal moves in real life long time control tournament games after thinking for over 10 minutes. People regularly create an erroneous board state in their head while thinking. This is often how blunders happen; merging together a possible future board state with the current one. Illegal moves are more rare because you have to then also miss that the move is illegal while physically making it. But they do happen. 1400s on chess.com never play illegal moves. 300s on chess.com never play illegal moves. Because it's impossible to do. In the real world, even grandmasters can make illegal moves, though they almost always have to be under time pressure. This idea that the illegal moves completely invalidate this result is just ill-conceived. On the other hand I do agree this is mostly returning common sequences of moves. And if you actually analyse the games, especially the ones with illegal moves, you'll find plenty of qualitative evidence of that. But I'm fed up of doing people's thinking for them for today, so this is peace out for me today. See my others comments on this post to see a more detailed analysis of what this is doing.
- Jensson 4y ago
  It doesn't remember exact moves, it remembers patterns of moves and continues those. That is why it so easily makes illegal moves, because the patterns doesn't have anything to do with board state.
  - SanderNL 4y ago
    Again, not enough for 1400. 800 maybe.
    - Jensson 4y ago
      We have evidence right here that patterns of moves leads to 1400. To me this is evidence that 1400 level play is very rigid and mostly relies on defined openings and strategies, making it easy for an LLM to find similar games and win using standard continuations. Just like grammar the patterns are too hard for humans to see and encode, but LLMs can encode pretty complex patterns. Domains that are easy to encode as grammars will be really easy for LLMs to solve, and the further from a grammar the harder for it.
      - SanderNL 4y ago
        At what point can we just say that understanding “patterns of moves” is understanding chess? It seems you suggest there is more to it, but maybe I am mistaken.
        Jensson 4y ago
        At least it should make valid moves, that is the minimum level required. It didn't reach that level here. If it never made illegal moves we could talk and see what it does, but until then we can be sure it didn't understand the rules.
        serverholic 4y ago
        I don’t understand why the threshold is “never”. Isn’t it entirely possible that the AI is learning a model of chess but this model is imperfect? What if AIs don’t fail the same way as humans?
mrbungie 4y ago
I would argue that a decent human player also has "similar sequences of moves in its training set", due to reading plus playing or watching games. The illegal move argument is good though, and indicates no direct understanding of what it is spewing out.
PoignardAzur 4y ago
> Most likely it has seen a similar sequence of moves in its training set. Wouldn't we expect a much higher rate of illegal moves if that was the case?
- WoodenChair 4y ago
  If there was an actual understanding of chess at a 1400 level we wouldn't expect any illegal moves.
  - scarmig 4y ago
    We would expect some vanishingly small fraction. Even at the grandmaster level, they happen (rarely): https://chess24.com/en/read/news/the-7-most-illegal-chess-moves-of-all-time https://chess24.com/en/read/news/the-7-most-illegal-chess-mo...
  - famouswaffles 4y ago
    Not if training is unsupervised. If you've never been explicitly told the rules of game, you can never be 100% sure of all possible illegal moves. anyway the 3.5 series can't ply chess but gpt-4 certainly can.
    - Jensson 4y ago
      > anyway the 3.5 series can't ply chess but gpt-4 certainly can. This article stated the opposite, gpt-4 couldn't play chess while gpt-3.5 could. So this is a case where the model got dumber.
      - famouswaffles 4y ago
        Not sure what's up but from here https://www.reddit.com/r/chess/comments/11rfbch/gpt4_seems_to_improve_chess_skills_significantly/ https://www.reddit.com/r/chess/comments/11rfbch/gpt4_seems_t... nd here https://twitter.com/zswitten/status/1631107663500304384 https://twitter.com/zswitten/status/1631107663500304384 Gpt-4 plys chess just fine.
johnfn 4y ago
> Most likely it has seen a similar sequence of moves in its training set. Is this a joke making fun of the common way people dismiss other ChatGPT successes? This makes no sense with respect to chess, because every game is unique, and playing a move from a different game in a new game is nonsensical.
- onos 4y ago
  Apparently not, at least at this level.
- fxtentacle 4y ago
  For Bomberland, we were quite surprised how strongly we could compress and quantize the current game state and still get useful movement predictions. I wouldn't be surprised if the relevant state in a typical beginner's chess game also excluded many units in the sense that yes, you could move them, but a beginner is going to just ignore them in any case.
- fsckboy 4y ago
  >playing a move from a different game in a new game is nonsensical GP did say "sequence of moves", and if it matches what it has seen from the first move on, including the opponent, it will be in a valid "sequence of moves". then, even midgame or endgame, if a sequence is played on one side of the board, even though the other side of the board may be different, the sequence has a great chance of being good (not always of course, but a 1400 rating is solid (you know the rules and some moves) but not amazing
- melenaboija 4y ago
  If it isn not memorizing, how do you think is doing it?
  - LeanderK 4y ago
    by trying to learning the general rules that to explain the dataset and minimise its loss. That's what machine learning is about, it's not called machine memorising.
ptmx 4y ago
I played chess against ChatGPT4 a few days ago without any special prompt engineering, and it played at what I would estimate to be a ~1500-1700 level without making any illegal moves in a 49 move game. Up to 10 or 15 moves, sure, we're well within common openings that could be regurgitated. By the time we're at move 20+, and especially 30+ and 40+, these are completely unique positions that haven't ever been reached before. I'd expect many more illegal moves just based on predicting sequences, though it's also possible I got "lucky" in my one game against ChatGPT and that it typically makes more errors than that. Of course, all positions have _some_ structural similarity or patterns compared to past positions, otherwise how would an LLM ever learn them? The nature of ChatGPT's understanding has to be different from the nature of a human's understanding, but that's more of a philosophical or semantic distinction. To me, it's still fascinating that by "just" learning from millions of PGNs, ChatGPT builds up a model of chess rules and strategy that's good enough to play at a club level.
- mtlmtlmtlmtl 4y ago
  I'd be interested in seeing this game, if you saved it?
  - nwienert 4y ago
    Me too, I couldn't get it to reliably go past move 15 without numerous errors. In my mind it's closer to 150 ELO than 1300, so I'd be happy to be proven wrong.
  - ptmx 4y ago
    I uploaded the PGN to lichess: https://lichess.org/rzSriO6I#97 https://lichess.org/rzSriO6I#97 After reviewing the chat history I actually have to issue a correction here, because there were two moves where ChatGPT played illegally: 1. ChatGPT tried to play 32. ... Nc5, despite there being a pawn on c5 2. ChatGPT tried to play 42. ... Kxe6, despite my king being on d5 It corrected itself after I questioned whether the previous move was legal. I was pretty floored that it managed to play a coherent game at all, so evidently I forgot about the few missteps it made. Much like ChatGPT itself, it turns out I'm not an entirely reliable narrator!
wongarsu 4y ago
Pattern matching on databases of previous moves is a legitimate technique that was poplar in chess AI back when computing power was more limited. Even if that was all that GPT did here it would still be interesting. But the presence of illegal moves doesn't really show that in my eyes. I fully understand the rules of chess, but I still occasionally make illegal moves. In 2017 Magnus Carlsen made one in a tournament [1]. The number of illegal moves suggests that either GPT is pretty new to chess, has low intelligence, or is playing under difficult circumstances (like not having a chess board at hand to keep track of the current state). I'm not sure we can deduce more than that 1: https://www.chessbase.in/news/Carlsen_Inarkiev_controversy https://www.chessbase.in/news/Carlsen_Inarkiev_controversy
- div72 4y ago
  I agree with your conclusion but the example you gave of Magnus Carlsen doing an illegal move is a pretty bad one imo. That was clearly a trap from his opponent as he did an illegal move first by checking Magnus while he was under check and then immediately calling the arbiter when Magnus moved his king in response. Magnus also probably did not have had much time left as it was a Blitz game.
  - leereeves 4y ago
    Also, according to the link, the final ruling was that Carlson's move was not illegal.
- EarthLaunch 4y ago
  An arbiter determined that Inarkiev made an illegal move and Carlsen did not. Interesting article.
- illiarian 4y ago
  But then it was one case out of thousands of games that Carlsen played. In the article ChatGPT is ding illegal moves in 3 games out of 19. The sample is smallm but the rate is much, much, higher. You'd expect maybe one, or none at all. Even for a supposed 1400 ELO player. Because even 800 ELO players rarely do that many illegal moves I think.
xigency 4y ago
> If it had any understanding of chess, it would never play an illegal move. Plenty of people who have a basic understanding of chess would make an illegal move if they had no board to see and had to play only with notation.
whimsicalism 4y ago
For some reason, seeing wrong comment after wrong comment like the above is just exhausting. Why are people struggling so hard to understand that it's not just regurgitating its training set? Is it motivated reasoning? Apologies if your comment was meant as parody of this view, it's hard for me to tell at this point.

circuit10 4y ago

I wouldn’t say it has no understanding whatsoever, it probably just doesn’t have the capability to keep track of everything that happened all the time

dwmbt 4y ago
that's also not entirely true either. it's prone to make illegal moves. one could argue abiding by the rules of the game is a requisite for understanding!
imtringued 4y ago
When I ask it to display moves in an ASCII board it messes up. I doubt that having a representation of the board is the key to improving its chess skills.

SanderNL 4y ago

You don’t get to 1400 like that. The amount of moves it has to literally remember is stupendous.

WoodenChair 4y ago
Nobody who is 1400 plays outright illegal moves.
- SanderNL 4y ago
  Humans don’t, but that is an unnecessary bar IMO.
  - WoodenChair 4y ago
    > Humans don’t, but that is an unnecessary bar IMO. People are always telling me that I'm moving the goalposts when I challenge the hyperbole about LLMs. But now you're moving the goalposts about chess. Not playing illegal moves is a pre-requisite for any strong understanding of how to play chess. That is definitely the goal post.
    - scj 4y ago
      If a two year old had an ELO of 1400 but sometimes took a nap halfway through a game, it'd be a prodigy. It's not like an AI making silly mistakes when driving a car.
- mrbungie 4y ago
  Yet it achieves 1400. Add hard rules to stop it spewing out said moves and you have a 1400 ELO Player (most UIs won't even let you make illegal moves). It is difficult to say that is not impressive due to it being an emergent ability.
  - WoodenChair 4y ago
    > It is difficult to say that is not impressive due to it being an emergent ability. I don't know why you think it's an emergent ability. It's seeing a sequence of moves, and playing the most likely next move (i.e. the most likely next token) given the previous complete move sequences it was trained on. That's the baseline of what an LLM does—not something emergent. Games in online chess databases tend to be of relatively good players. Nobody wants to look up games played by two 800 ELO players. As an aside, there have been chess programs for years that show you for a given position all of the previous games in its database with the same position and the win outcome % of each move. That's all that's going on here.
    - oezi 4y ago
      > here have been chess programs for years that show you for a given position all of the previous games in its database with the same position and the win outcome % of each move. That's all that's going on here. It could be, but would you think that of the 100-300 bn parameters in the model a lot are dedicated to chess move sequences? It seems likely that it has seen such data, but I would be surprised if it is using a considerable chunk to store chess database information.
      - Jensson 4y ago
        The web has millions of grandmaster chess games and probably billions of chess games overall. So I wouldn't be surprised if it has like 0.01% to chess games since there are so many, if so that would mean it has 10-30 million parameters to play chess with, for comparison stockfish has 10 million parameters to its chess engine.
    - mrbungie 4y ago
- vanviegen 4y ago
  Does that still hold when the player doesn't have a board in front of them, but just a list of previous moves?
- mtlmtlmtlmtl 4y ago
  I have seen 1400 FIDE elo(much much stronger than chess.com 1400) players make illegal moves in real life long time control tournament games after thinking for over 10 minutes. People regularly create an erroneous board state in their head while thinking. This is often how blunders happen; merging together a possible future board state with the current one. Illegal moves are more rare because you have to then also miss that the move is illegal while physically making it. But they do happen. 1400s on chess.com never play illegal moves. 300s on chess.com never play illegal moves. Because it's impossible to do. In the real world, even grandmasters can make illegal moves, though they almost always have to be under time pressure. This idea that the illegal moves completely invalidate this result is just ill-conceived. On the other hand I do agree this is mostly returning common sequences of moves. And if you actually analyse the games, especially the ones with illegal moves, you'll find plenty of qualitative evidence of that. But I'm fed up of doing people's thinking for them for today, so this is peace out for me today. See my others comments on this post to see a more detailed analysis of what this is doing.
Jensson 4y ago
It doesn't remember exact moves, it remembers patterns of moves and continues those. That is why it so easily makes illegal moves, because the patterns doesn't have anything to do with board state.
- SanderNL 4y ago
  Again, not enough for 1400. 800 maybe.
  - Jensson 4y ago
    We have evidence right here that patterns of moves leads to 1400. To me this is evidence that 1400 level play is very rigid and mostly relies on defined openings and strategies, making it easy for an LLM to find similar games and win using standard continuations. Just like grammar the patterns are too hard for humans to see and encode, but LLMs can encode pretty complex patterns. Domains that are easy to encode as grammars will be really easy for LLMs to solve, and the further from a grammar the harder for it.
    - SanderNL 4y ago
      At what point can we just say that understanding “patterns of moves” is understanding chess? It seems you suggest there is more to it, but maybe I am mistaken.
      - Jensson 4y ago
        At least it should make valid moves, that is the minimum level required. It didn't reach that level here. If it never made illegal moves we could talk and see what it does, but until then we can be sure it didn't understand the rules.
        serverholic 4y ago
        I don’t understand why the threshold is “never”. Isn’t it entirely possible that the AI is learning a model of chess but this model is imperfect? What if AIs don’t fail the same way as humans?

mrbungie 4y ago

I would argue that a decent human player also has "similar sequences of moves in its training set", due to reading plus playing or watching games. The illegal move argument is good though, and indicates no direct understanding of what it is spewing out.

PoignardAzur 4y ago

> Most likely it has seen a similar sequence of moves in its training set. Wouldn't we expect a much higher rate of illegal moves if that was the case?

WoodenChair 4y ago
If there was an actual understanding of chess at a 1400 level we wouldn't expect any illegal moves.
- scarmig 4y ago
  We would expect some vanishingly small fraction. Even at the grandmaster level, they happen (rarely): https://chess24.com/en/read/news/the-7-most-illegal-chess-moves-of-all-time https://chess24.com/en/read/news/the-7-most-illegal-chess-mo...
- famouswaffles 4y ago
  Not if training is unsupervised. If you've never been explicitly told the rules of game, you can never be 100% sure of all possible illegal moves. anyway the 3.5 series can't ply chess but gpt-4 certainly can.
  - Jensson 4y ago
    > anyway the 3.5 series can't ply chess but gpt-4 certainly can. This article stated the opposite, gpt-4 couldn't play chess while gpt-3.5 could. So this is a case where the model got dumber.
    - famouswaffles 4y ago
      Not sure what's up but from here https://www.reddit.com/r/chess/comments/11rfbch/gpt4_seems_to_improve_chess_skills_significantly/ https://www.reddit.com/r/chess/comments/11rfbch/gpt4_seems_t... nd here https://twitter.com/zswitten/status/1631107663500304384 https://twitter.com/zswitten/status/1631107663500304384 Gpt-4 plys chess just fine.

johnfn 4y ago

> Most likely it has seen a similar sequence of moves in its training set. Is this a joke making fun of the common way people dismiss other ChatGPT successes? This makes no sense with respect to chess, because every game is unique, and playing a move from a different game in a new game is nonsensical.

onos 4y ago
Apparently not, at least at this level.
fxtentacle 4y ago
For Bomberland, we were quite surprised how strongly we could compress and quantize the current game state and still get useful movement predictions. I wouldn't be surprised if the relevant state in a typical beginner's chess game also excluded many units in the sense that yes, you could move them, but a beginner is going to just ignore them in any case.
fsckboy 4y ago
>playing a move from a different game in a new game is nonsensical GP did say "sequence of moves", and if it matches what it has seen from the first move on, including the opponent, it will be in a valid "sequence of moves". then, even midgame or endgame, if a sequence is played on one side of the board, even though the other side of the board may be different, the sequence has a great chance of being good (not always of course, but a 1400 rating is solid (you know the rules and some moves) but not amazing
melenaboija 4y ago
If it isn not memorizing, how do you think is doing it?
- LeanderK 4y ago
  by trying to learning the general rules that to explain the dataset and minimise its loss. That's what machine learning is about, it's not called machine memorising.

ptmx 4y ago

I played chess against ChatGPT4 a few days ago without any special prompt engineering, and it played at what I would estimate to be a ~1500-1700 level without making any illegal moves in a 49 move game. Up to 10 or 15 moves, sure, we're well within common openings that could be regurgitated. By the time we're at move 20+, and especially 30+ and 40+, these are completely unique positions that haven't ever been reached before. I'd expect many more illegal moves just based on predicting sequences, though it's also possible I got "lucky" in my one game against ChatGPT and that it typically makes more errors than that. Of course, all positions have _some_ structural similarity or patterns compared to past positions, otherwise how would an LLM ever learn them? The nature of ChatGPT's understanding has to be different from the nature of a human's understanding, but that's more of a philosophical or semantic distinction. To me, it's still fascinating that by "just" learning from millions of PGNs, ChatGPT builds up a model of chess rules and strategy that's good enough to play at a club level.

mtlmtlmtlmtl 4y ago
I'd be interested in seeing this game, if you saved it?
- nwienert 4y ago
  Me too, I couldn't get it to reliably go past move 15 without numerous errors. In my mind it's closer to 150 ELO than 1300, so I'd be happy to be proven wrong.
- ptmx 4y ago
  I uploaded the PGN to lichess: https://lichess.org/rzSriO6I#97 https://lichess.org/rzSriO6I#97 After reviewing the chat history I actually have to issue a correction here, because there were two moves where ChatGPT played illegally: 1. ChatGPT tried to play 32. ... Nc5, despite there being a pawn on c5 2. ChatGPT tried to play 42. ... Kxe6, despite my king being on d5 It corrected itself after I questioned whether the previous move was legal. I was pretty floored that it managed to play a coherent game at all, so evidently I forgot about the few missteps it made. Much like ChatGPT itself, it turns out I'm not an entirely reliable narrator!

wongarsu 4y ago

Pattern matching on databases of previous moves is a legitimate technique that was poplar in chess AI back when computing power was more limited. Even if that was all that GPT did here it would still be interesting. But the presence of illegal moves doesn't really show that in my eyes. I fully understand the rules of chess, but I still occasionally make illegal moves. In 2017 Magnus Carlsen made one in a tournament [1]. The number of illegal moves suggests that either GPT is pretty new to chess, has low intelligence, or is playing under difficult circumstances (like not having a chess board at hand to keep track of the current state). I'm not sure we can deduce more than that 1: https://www.chessbase.in/news/Carlsen_Inarkiev_controversy https://www.chessbase.in/news/Carlsen_Inarkiev_controversy

div72 4y ago
I agree with your conclusion but the example you gave of Magnus Carlsen doing an illegal move is a pretty bad one imo. That was clearly a trap from his opponent as he did an illegal move first by checking Magnus while he was under check and then immediately calling the arbiter when Magnus moved his king in response. Magnus also probably did not have had much time left as it was a Blitz game.
- leereeves 4y ago
  Also, according to the link, the final ruling was that Carlson's move was not illegal.
EarthLaunch 4y ago
An arbiter determined that Inarkiev made an illegal move and Carlsen did not. Interesting article.
illiarian 4y ago
But then it was one case out of thousands of games that Carlsen played. In the article ChatGPT is ding illegal moves in 3 games out of 19. The sample is smallm but the rate is much, much, higher. You'd expect maybe one, or none at all. Even for a supposed 1400 ELO player. Because even 800 ELO players rarely do that many illegal moves I think.

xigency 4y ago

> If it had any understanding of chess, it would never play an illegal move. Plenty of people who have a basic understanding of chess would make an illegal move if they had no board to see and had to play only with notation.

whimsicalism 4y ago

For some reason, seeing wrong comment after wrong comment like the above is just exhausting. Why are people struggling so hard to understand that it's not just regurgitating its training set? Is it motivated reasoning? Apologies if your comment was meant as parody of this view, it's hard for me to tell at this point.