14 ms·

> For one, gpt-3.5-turbo-instruct rarely suggests illegal moves, even in the late game. This requires “understanding” chess. Here's one way to test whether it

by tromp 2y ago

> For one, gpt-3.5-turbo-instruct rarely suggests illegal moves, even in the late game. This requires “understanding” chess. Here's one way to test whether it really understands chess. Make it play the next move in 1000 random legal positions (in which no side is checkmated yet). Such positions can be generated using the ChessPositionRanking project at [1]. Does it still rarely suggest illegal moves in these totally weird positions, that will be completely unlike any it would have seen in training (and in which the legal move choice is often highly restricted) ? While good for testing legality of next moves, these positions are not so useful for distinguishing their quality, since usually one side already has an overwhelming advantage. [1] https://github.com/tromp/ChessPositionRanking https://github.com/tromp/ChessPositionRanking

deleted 2y ago
[deleted]
BurningFrog 2y ago
Not that I understand the internals of current AI tech, but... I'd expect that an AI that has seen billions of chess positions, and the moves played in them, can figure out the rules for legal moves without being told?
- rscho 2y ago
  Statistical 'AI' doesn't 'understand' anything, strictly speaking. It predicts a move with high probability, which could be legal or illegal.
  - griomnib 2y ago
    Likewise with LLM you don’t know if it is truly in the “chess” branch of the statistical distribution or it is picking up something else entirely, like some arcane overlap of tokens. So much of the training data (eg common crawl, pile, Reddit) is dogshit, so it generates reheated dogshit.
    - Helonomoto 2y ago
      You generalize this without mentioning that there are LLMs which do not just use random 'dogshit'. Also what does a normal human do? It looks around how to move one random piece and it uses a very small dictionary / set of basic rules to move it. I do not remember me learning to count every piece and its options by looking up that rulebook. I learned to 'see' how i can move one type of chess piece. If a LLM uses only these piece moves on a mathematical level, it would do the same thing as i do. And yes there is also absolutly the option for an LLM to learn some kind of meta game.
  - Helonomoto 2y ago
    How do you define 'understand'? There is plenty of AI which learns the rules of games like Alpha Zero. LLMs might not have the architecture to 'learn', but it also might. If it optimizes all possible moves one chess peace can do (which is not that much to learn) it can easily only 'move' from one game set to another by this type of dictionary.
    - rscho 2y ago
      Understanding a rules-based system (chess) means to be able to learn non-probabilistic rules (an abstraction over the concrete world). Humans are a mix of symbolic and probabilistic learning, allowing them to get a huge boost in performance by admitting rules. It doesn't mean a human will never make an illegal move, but it means a much smaller probability of illegal move based on less training data. Asymptotically, performance from humans and purely probabilistic systems converge. But that also means that in appropriate situations, humans are hugely more data-efficient.
      - david-gpu 2y ago
        > in appropriate situations, humans are hugely more data-efficient After spending some years raising my children I gave up the notion that humans are data efficient. It takes a mind numbing amount of training to get them to learn the most basic skills.
        rscho 2y ago
        You could compare childhood with the training phase of a model. Still think humans are not data-efficient ?
        david-gpu 2y ago
        Yes, that is exactly the point I am making. It takes many repetitions (epochs) to teach them anything.
        2y ago
  - fragmede 2y ago
    The illegal moves are interesting as it goes to "understanding". In children learning to play chess, how often do they try and make illegal moves? When first learning the game I remember that I'd lose track of all the things going on at once and try to make illegal moves, but eventually the rules became second nature and I stopped trying to make illegal moves. With an ELO of 1800, I'd expect ChatGPT not to make any illegal moves.
  - sixfiveotwo 2y ago
    I think the article briefly touch on that topic at some point: > For one, gpt-3.5-turbo-instruct rarely suggests illegal moves, even in the late game. This requires “understanding” chess. If this doesn’t convince you, I encourage you to write a program that can take strings like 1. e4 d5 2. exd5 Qxd5 3. Nc3 and then say if the last move was legal. However, I can't say if LLMs fall in the "statistical AI" category.
- pvitz 2y ago
  A system that would just output the most probable tokens based on the text it was fed and trained on the games played by players with ratings greater than 1800 would certainly fail to output the right moves to totally unlikely board positions.
- Helonomoto 2y ago
  Yes in theory it could. Depends on how it learns. Does it learn by memorization or by learning the rules. It depends on the architecture and the amount of 'pressure' you put on it to be more efficient or not.
griomnib 2y ago
I think at this point it’s very clear LLM aren’t achieving any form of “reasoning” as commonly understood. Among other factors it can be argued that true reasoning involves symbolic logic and abstractions, and LLM are next token predictors.
- DiogenesKynikos 2y ago
  Effective next-token prediction requires reasoning. You can also say humans are "just XYZ biological system," but that doesn't mean they don't reason. The same goes for LLMs.
  - griomnib 2y ago
    Take a word problem for example. A child will be told the first step is to translate the problem from human language to mathematical notation (symbolic representation), then solve the math (logic). A human doesn’t use next token prediction to solve word problems.
    - Majromax 2y ago
      But the LLM isn't "using next-token prediction" to solve the problem, that's only how it's evaluated. The "real processing" happens through the various transformer layers (and token-wise nonlinear networks), where it seems as if progressively richer meanings are added to each token. That rich feature set then decodes to the next predicted token, but that decoding step is throwing away a lot of information contained in the latent space. If language models (per Anthropic's work) can have a direction in latent space correspond to the concept of the Golden Gate Bridge, then I think it's reasonable (albeit far from certain) to say that LLMs are performing some kind of symbolic-ish reasoning.
      - griomnib 2y ago
        Anthropic had a vested interest in people thinking Claude is reasoning. However, in coding tasks I’ve been able to find it directly regurgitating Stack overflow answers (like literally a google search turns up the code). Giving coding is supposed to be Claude’s strength, and it’s clearly just parroting web data, I’m not seeing any sort of “reasoning”. LLM may be useful but they don’t think. They’ve already plateaued, and given the absurd energy requirements I think they will prove to be far less impactful than people think.
NitpickLawyer 2y ago
Interesting tidbit I once learned from a chess livestream. Even human super-GMs have a really hard time "scoring" or "solving" extremely weird positions. That is, positions that shouldn't come from logical opening - mid game - end game regular play. It's absolutely amazing to see a super-GM (in that case it was Hikaru) see a position, and basically "play-by-play" it from the beginning, to show people how they got in that position. It wasn't his game btw. But later in that same video when asked he explained what I wrote in the first paragraph. It works with proper games, but it rarely works with weird random chess puzzles, as he put it. Or, in other words, chess puzzles that come from real games are much better than "randomly generated", and make more sense even to the best of humans.
- saghm 2y ago
  Super interesting (although it also makes some sense that experts would focus on "likely" subsets given how the number of permutations of chess games is too high for it to be feasible to learn them all)! That said, I still imagine that even most intermediate chess players would perfectly make only _legal_ moves in weird positions, even if they're low quality.
- MarcelOlsz 2y ago
  Would love a link to that video!
- lukan 2y ago
  "Even human super-GMs have a really hard time "scoring" or "solving" extremely weird positions. " I can sort of confirm that. I never learned all the formal theoretical standard chess strategies except for the basic ones. So when playing against really good players, way above my level, I could win sometimes (or allmost) simply by making unconventional (dumb by normal strategy) moves in the beginning - resulting in a non standard game where I could apply pressure in a way the opponent was not prepared for (also they underestimated me after the initial dumb moves). For me, the unconventional game was just like a standard game, I had no routine - but for the experienced one, it was way more challenging. But then of course in the standard situations, to which allmost every chess game evolves to - they destroyed me, simply for experience and routine.
fragmede 2y ago
How well does it play modified versions of chess? eg, a modified opening board like the back row is all knights, or modified movement eg rooks can move like a queen. A human should be able to reason their way through playing a modified game, but I'd expect an LLM, if it's just parroting its training data, to suggest illegal moves, or stick to previously legal moves.
snowwrestler 2y ago
It’s kind of crazy to assert that the systems understand chess, and then disclose further down the article that sometimes he failed to get a legal move after 10 tries and had to sub in a random move. A person who understands chess well (Elo 1800, let’s say) will essentially never fail to provide a legal move on the first try.
- famouswaffles 2y ago
  He is testing several models, some of which cannot reliably output legal moves. That's different from saying all models including the one he thinks understands can't generate a legal move in 10 tries. 3.5-turbo-instruct's illegal move rate is about 5 or less in 8205
  - IanCal 2y ago
    I also wonder what kind of invalid moves they are. There's "you can't move your knight to j9 that's off the board", "there's already a piece there" and "actually that would leave you in check". I think it's also significantly harder to play chess if you were to hear a sequence of moves over the phone and had to reply with a followup move, with no space or time to think or talk through moves.
- navane 2y ago
  Pretty sure elo 1200 will only give legal moves. It's really not hard to make legal moves in chess.
  - thaumasiotes 2y ago
    Casual players make illegal moves all the time. The problem isn't knowing how the pieces move. It's that it's illegal to leave your own king in check. It's not so common to accidentally move your king into check, though I'm sure it happens, but it's very common to accidentally move a piece that was blocking an attack on your king. I would tend to agree that there's a big difference between attempting to make a move that's illegal because of the state of a different region of the board, and attempting to make one that's illegal because of the identity of the piece being moved, but if your only category of interest is "illegal moves", you can't see that difference. Software that knows the rules of the game shouldn't be making either mistake.
cma 2y ago
Its training set would include a lot of randomly generated positions like that that then get played out by chess engines wouldn't it? Just from people messing around andbposting results. Not identical ones, but similarly oddball.
thaumasiotes 2y ago
> Here's one way to test whether it really understands chess. Make it play the next move in 1000 random legal positions Suppose it tries to capture en passant. How do you know whether that's legal?
- BalinKing 2y ago
  I feel like you could add “do not capture en passant unless it is the only possible move” to the test without changing what it’s trying to prove—if anything, some small permutation like this might even make it a stronger test of “reasoning capability.” (Personally I’m unconvinced of the utility of this test in the first place, but I think it can be reasonably steelmanned.)
namaria 2y ago
Assigning "understanding" to an undefined entity is an undefined statement. It isn't even wrong.
_heimdall 2y ago
Would that be enough to prove it? If the LLM was trained only on a set of legal moves, isn't it possible that it functionally learned how each piece is allowed to move without learning how to actually reason about it? Said differently in case I phrased that poorly - couldn't the LLM still learn the it only ever saw bishops move diagonally and therefore only considering those moves without actually reasoning through the concept of legal and illegal moves?
zbyforgotp 2y ago
The problem is that the llm don’t learn to play moves from a position, the internet archives contain only game records. They might be building something to represent position internationally but it will not be automatically activated with an encoded chess position.
- tromp 2y ago
  The ChessPositionRanking project, with help from the Texel chess engine author, tries to prove random positions (that are not obviously illegal) legal by constructing a game ending in the position. If that fails it tries to prove the position illegal. This now works for over 99.99% of randomly generated positions, so one can feed the legal game record found for random legal positions.

by tromp 2y ago

deleted 2y ago

[deleted]

BurningFrog 2y ago

Not that I understand the internals of current AI tech, but... I'd expect that an AI that has seen billions of chess positions, and the moves played in them, can figure out the rules for legal moves without being told?

rscho 2y ago
Statistical 'AI' doesn't 'understand' anything, strictly speaking. It predicts a move with high probability, which could be legal or illegal.
- griomnib 2y ago
  Likewise with LLM you don’t know if it is truly in the “chess” branch of the statistical distribution or it is picking up something else entirely, like some arcane overlap of tokens. So much of the training data (eg common crawl, pile, Reddit) is dogshit, so it generates reheated dogshit.
  - Helonomoto 2y ago
    You generalize this without mentioning that there are LLMs which do not just use random 'dogshit'. Also what does a normal human do? It looks around how to move one random piece and it uses a very small dictionary / set of basic rules to move it. I do not remember me learning to count every piece and its options by looking up that rulebook. I learned to 'see' how i can move one type of chess piece. If a LLM uses only these piece moves on a mathematical level, it would do the same thing as i do. And yes there is also absolutly the option for an LLM to learn some kind of meta game.
- Helonomoto 2y ago
  How do you define 'understand'? There is plenty of AI which learns the rules of games like Alpha Zero. LLMs might not have the architecture to 'learn', but it also might. If it optimizes all possible moves one chess peace can do (which is not that much to learn) it can easily only 'move' from one game set to another by this type of dictionary.
  - rscho 2y ago
    Understanding a rules-based system (chess) means to be able to learn non-probabilistic rules (an abstraction over the concrete world). Humans are a mix of symbolic and probabilistic learning, allowing them to get a huge boost in performance by admitting rules. It doesn't mean a human will never make an illegal move, but it means a much smaller probability of illegal move based on less training data. Asymptotically, performance from humans and purely probabilistic systems converge. But that also means that in appropriate situations, humans are hugely more data-efficient.
    - david-gpu 2y ago
      > in appropriate situations, humans are hugely more data-efficient After spending some years raising my children I gave up the notion that humans are data efficient. It takes a mind numbing amount of training to get them to learn the most basic skills.
      - rscho 2y ago
        You could compare childhood with the training phase of a model. Still think humans are not data-efficient ?
        david-gpu 2y ago
        Yes, that is exactly the point I am making. It takes many repetitions (epochs) to teach them anything.
        2y ago
- fragmede 2y ago
  The illegal moves are interesting as it goes to "understanding". In children learning to play chess, how often do they try and make illegal moves? When first learning the game I remember that I'd lose track of all the things going on at once and try to make illegal moves, but eventually the rules became second nature and I stopped trying to make illegal moves. With an ELO of 1800, I'd expect ChatGPT not to make any illegal moves.
- sixfiveotwo 2y ago
  I think the article briefly touch on that topic at some point: > For one, gpt-3.5-turbo-instruct rarely suggests illegal moves, even in the late game. This requires “understanding” chess. If this doesn’t convince you, I encourage you to write a program that can take strings like 1. e4 d5 2. exd5 Qxd5 3. Nc3 and then say if the last move was legal. However, I can't say if LLMs fall in the "statistical AI" category.
pvitz 2y ago
A system that would just output the most probable tokens based on the text it was fed and trained on the games played by players with ratings greater than 1800 would certainly fail to output the right moves to totally unlikely board positions.
Helonomoto 2y ago
Yes in theory it could. Depends on how it learns. Does it learn by memorization or by learning the rules. It depends on the architecture and the amount of 'pressure' you put on it to be more efficient or not.

griomnib 2y ago

I think at this point it’s very clear LLM aren’t achieving any form of “reasoning” as commonly understood. Among other factors it can be argued that true reasoning involves symbolic logic and abstractions, and LLM are next token predictors.

DiogenesKynikos 2y ago
Effective next-token prediction requires reasoning. You can also say humans are "just XYZ biological system," but that doesn't mean they don't reason. The same goes for LLMs.
- griomnib 2y ago
  Take a word problem for example. A child will be told the first step is to translate the problem from human language to mathematical notation (symbolic representation), then solve the math (logic). A human doesn’t use next token prediction to solve word problems.
  - Majromax 2y ago
    But the LLM isn't "using next-token prediction" to solve the problem, that's only how it's evaluated. The "real processing" happens through the various transformer layers (and token-wise nonlinear networks), where it seems as if progressively richer meanings are added to each token. That rich feature set then decodes to the next predicted token, but that decoding step is throwing away a lot of information contained in the latent space. If language models (per Anthropic's work) can have a direction in latent space correspond to the concept of the Golden Gate Bridge, then I think it's reasonable (albeit far from certain) to say that LLMs are performing some kind of symbolic-ish reasoning.
    - griomnib 2y ago
      Anthropic had a vested interest in people thinking Claude is reasoning. However, in coding tasks I’ve been able to find it directly regurgitating Stack overflow answers (like literally a google search turns up the code). Giving coding is supposed to be Claude’s strength, and it’s clearly just parroting web data, I’m not seeing any sort of “reasoning”. LLM may be useful but they don’t think. They’ve already plateaued, and given the absurd energy requirements I think they will prove to be far less impactful than people think.

NitpickLawyer 2y ago

Interesting tidbit I once learned from a chess livestream. Even human super-GMs have a really hard time "scoring" or "solving" extremely weird positions. That is, positions that shouldn't come from logical opening - mid game - end game regular play. It's absolutely amazing to see a super-GM (in that case it was Hikaru) see a position, and basically "play-by-play" it from the beginning, to show people how they got in that position. It wasn't his game btw. But later in that same video when asked he explained what I wrote in the first paragraph. It works with proper games, but it rarely works with weird random chess puzzles, as he put it. Or, in other words, chess puzzles that come from real games are much better than "randomly generated", and make more sense even to the best of humans.

saghm 2y ago
Super interesting (although it also makes some sense that experts would focus on "likely" subsets given how the number of permutations of chess games is too high for it to be feasible to learn them all)! That said, I still imagine that even most intermediate chess players would perfectly make only _legal_ moves in weird positions, even if they're low quality.
MarcelOlsz 2y ago
Would love a link to that video!
lukan 2y ago
"Even human super-GMs have a really hard time "scoring" or "solving" extremely weird positions. " I can sort of confirm that. I never learned all the formal theoretical standard chess strategies except for the basic ones. So when playing against really good players, way above my level, I could win sometimes (or allmost) simply by making unconventional (dumb by normal strategy) moves in the beginning - resulting in a non standard game where I could apply pressure in a way the opponent was not prepared for (also they underestimated me after the initial dumb moves). For me, the unconventional game was just like a standard game, I had no routine - but for the experienced one, it was way more challenging. But then of course in the standard situations, to which allmost every chess game evolves to - they destroyed me, simply for experience and routine.

fragmede 2y ago

How well does it play modified versions of chess? eg, a modified opening board like the back row is all knights, or modified movement eg rooks can move like a queen. A human should be able to reason their way through playing a modified game, but I'd expect an LLM, if it's just parroting its training data, to suggest illegal moves, or stick to previously legal moves.

snowwrestler 2y ago

It’s kind of crazy to assert that the systems understand chess, and then disclose further down the article that sometimes he failed to get a legal move after 10 tries and had to sub in a random move. A person who understands chess well (Elo 1800, let’s say) will essentially never fail to provide a legal move on the first try.

famouswaffles 2y ago
He is testing several models, some of which cannot reliably output legal moves. That's different from saying all models including the one he thinks understands can't generate a legal move in 10 tries. 3.5-turbo-instruct's illegal move rate is about 5 or less in 8205
- IanCal 2y ago
  I also wonder what kind of invalid moves they are. There's "you can't move your knight to j9 that's off the board", "there's already a piece there" and "actually that would leave you in check". I think it's also significantly harder to play chess if you were to hear a sequence of moves over the phone and had to reply with a followup move, with no space or time to think or talk through moves.
navane 2y ago
Pretty sure elo 1200 will only give legal moves. It's really not hard to make legal moves in chess.
- thaumasiotes 2y ago
  Casual players make illegal moves all the time. The problem isn't knowing how the pieces move. It's that it's illegal to leave your own king in check. It's not so common to accidentally move your king into check, though I'm sure it happens, but it's very common to accidentally move a piece that was blocking an attack on your king. I would tend to agree that there's a big difference between attempting to make a move that's illegal because of the state of a different region of the board, and attempting to make one that's illegal because of the identity of the piece being moved, but if your only category of interest is "illegal moves", you can't see that difference. Software that knows the rules of the game shouldn't be making either mistake.

cma 2y ago

Its training set would include a lot of randomly generated positions like that that then get played out by chess engines wouldn't it? Just from people messing around andbposting results. Not identical ones, but similarly oddball.

thaumasiotes 2y ago

> Here's one way to test whether it really understands chess. Make it play the next move in 1000 random legal positions Suppose it tries to capture en passant. How do you know whether that's legal?

BalinKing 2y ago
I feel like you could add “do not capture en passant unless it is the only possible move” to the test without changing what it’s trying to prove—if anything, some small permutation like this might even make it a stronger test of “reasoning capability.” (Personally I’m unconvinced of the utility of this test in the first place, but I think it can be reasonably steelmanned.)

namaria 2y ago

Assigning "understanding" to an undefined entity is an undefined statement. It isn't even wrong.

_heimdall 2y ago

Would that be enough to prove it? If the LLM was trained only on a set of legal moves, isn't it possible that it functionally learned how each piece is allowed to move without learning how to actually reason about it? Said differently in case I phrased that poorly - couldn't the LLM still learn the it only ever saw bishops move diagonally and therefore only considering those moves without actually reasoning through the concept of legal and illegal moves?

zbyforgotp 2y ago

The problem is that the llm don’t learn to play moves from a position, the internet archives contain only game records. They might be building something to represent position internationally but it will not be automatically activated with an encoded chess position.

tromp 2y ago
The ChessPositionRanking project, with help from the Texel chess engine author, tries to prove random positions (that are not obviously illegal) legal by constructing a game ending in the position. If that fails it tries to prove the position illegal. This now works for over 99.99% of randomly generated positions, so one can feed the legal game record found for random legal positions.