5 ms·

It’s kind of crazy to assert that the systems understand chess, and then disclose further down the article that sometimes he failed to get a legal move after 10

It’s kind of crazy to assert that the systems understand chess, and then disclose further down the article that sometimes he failed to get a legal move after 10 tries and had to sub in a random move. A person who understands chess well (Elo 1800, let’s say) will essentially never fail to provide a legal move on the first try.

famouswaffles 2y ago
He is testing several models, some of which cannot reliably output legal moves. That's different from saying all models including the one he thinks understands can't generate a legal move in 10 tries. 3.5-turbo-instruct's illegal move rate is about 5 or less in 8205
- IanCal 2y ago
  I also wonder what kind of invalid moves they are. There's "you can't move your knight to j9 that's off the board", "there's already a piece there" and "actually that would leave you in check". I think it's also significantly harder to play chess if you were to hear a sequence of moves over the phone and had to reply with a followup move, with no space or time to think or talk through moves.
navane 2y ago
Pretty sure elo 1200 will only give legal moves. It's really not hard to make legal moves in chess.
- thaumasiotes 2y ago
  Casual players make illegal moves all the time. The problem isn't knowing how the pieces move. It's that it's illegal to leave your own king in check. It's not so common to accidentally move your king into check, though I'm sure it happens, but it's very common to accidentally move a piece that was blocking an attack on your king. I would tend to agree that there's a big difference between attempting to make a move that's illegal because of the state of a different region of the board, and attempting to make one that's illegal because of the identity of the piece being moved, but if your only category of interest is "illegal moves", you can't see that difference. Software that knows the rules of the game shouldn't be making either mistake.
  - philipwhiuk 2y ago
    Casual players don’t make illegal moves so often that you have to assign them a random move after 10 goes.
Certhas 2y ago
What do you mean by "understand chess"? I think you don't appreciate how good the level of chess displayed here is. It would take an average adult years of dedicated practice to get to 1800. The article doesn't say how often the LLM fails to generate legal moves in ten tries, but it can't be often or the level of play would be much much much worse. As seems often the case, the LLM seems to have a brilliant intuition, but no precise rigid "world model". Of course words like intuition are anthropomorphic. At best a model for what LLMs are doing. But saying "they don't understand" when they can do _this well_ is absurd.
- ChoHag 2y ago
  [dead]
- deleted 2y ago
  [deleted]
- vundercind 2y ago
  > I think you don't appreciate how good the level of chess displayed here is. It would take an average adult years of dedicated practice to get to 1800. Since we already have programs that can do this, that definitely aren’t really thinking and don’t “understand” anything at all, I don’t see the relevance of this part.
  - throw310822 2y ago
    It seems you're shifting the discourse here. In the context of LLMs, "to understand" is short for "to have a world model beyond the pure relations between words". In that sense, chess engines do "understand" chess, as they operate on a world model. You can even say that they don't understand anything but chess, which makes them extremely un-intelligent and definitely not capable of understanding as we mean it. However, since an LLM is a generalist engine, if it understands chess there is no reason for it not to understand millions of other concepts and how they relate to each other. And this is the kind of understanding that humans do.
- photonthug 2y ago
stuaxo 2y ago
I hate the use of words like "understand" in these conversations. The system understands nothing, it's anthropomorphising it to say it does.
- Sharlin 2y ago
  Trying to appropriate perfectly well generalizable terms as "something that only humans do" brings zero value to a conversation. It's a "god in the gaps" argument, essentially, and we don't exactly have a great track record of correctly identifying things that are uniquely human.
  - fao_ 2y ago
    There's very literally currently a whole wealth of papers proving that LLMs do not understand, cannot reason, and cannot perform basic kinds of reasoning that even a dog can perform. But, ok.
    - TeMPOraL 2y ago
      There's very literally currently a whole wealth of papers proving the opposite, too, so ¯\_(ツ)_/¯.
    - wizzwizz4 2y ago
      There's a whole wealth of papers proving that LLMs do not understand the concepts they write about. That doesn't mean they don't understand grammar – which (as I've claimed since the GPT-2 days) we should, theoretically, expect them to "understand". And what is chess, but a particularly sophisticated grammar?
- trashtester 2y ago
  I have the same conclusion, but for the opposite reason. It seems like many people tend to use the word "understand" to that not only does someone believe that a given move is good, they also belive that this knowledge comes from a rational evaluation. Some attribute this to a non-material soul/mind, some to quantum mechanics or something else that seems magic, while others never realized the problem with such a belief in the first place. I would claim that when someone can instantly recognize good moves in a given situation, it doesn't come from rationality at all, but from some mix of memory and an intuition that has been build by playing the game many times, with only tiny elements of actual rational thought sprinkled in. This even holds true when these people start to calculate. It is primarily their intuition that prevens them from spending time on all sorts of unlikely moves. And this intuition, I think, represents most of their real "understanding" of the game. This is quite different from understanding something like a mathematical proof, which is almost exclusively inducive logic. And since "understand" so often is associated with rational inductive logic, I think the proper term would be to have "good intuition" when playing the game. And this "good intuition" seems to me precisely the kind of thing that is trained within most neural nets, even LLM's. (Q*, AlphaZero, etc also add the ability to "calculate", meaning traverse the search space efficiently). If we wanted to measure how good this intuition is compared to human chess intuition, we could limit an engine like AlphaZero to only evaluate the same number of moves per second that good humans would be able to, which might be around 10 or so. Maybe with this limitation, the engine wouldn't currently be able to beat the best humans, but even if it reaches a rating of 2000-2500 this way, I would say it has a pretty good intuitive understanding.

famouswaffles 2y ago

He is testing several models, some of which cannot reliably output legal moves. That's different from saying all models including the one he thinks understands can't generate a legal move in 10 tries. 3.5-turbo-instruct's illegal move rate is about 5 or less in 8205

IanCal 2y ago
I also wonder what kind of invalid moves they are. There's "you can't move your knight to j9 that's off the board", "there's already a piece there" and "actually that would leave you in check". I think it's also significantly harder to play chess if you were to hear a sequence of moves over the phone and had to reply with a followup move, with no space or time to think or talk through moves.

navane 2y ago

Pretty sure elo 1200 will only give legal moves. It's really not hard to make legal moves in chess.

thaumasiotes 2y ago
Casual players make illegal moves all the time. The problem isn't knowing how the pieces move. It's that it's illegal to leave your own king in check. It's not so common to accidentally move your king into check, though I'm sure it happens, but it's very common to accidentally move a piece that was blocking an attack on your king. I would tend to agree that there's a big difference between attempting to make a move that's illegal because of the state of a different region of the board, and attempting to make one that's illegal because of the identity of the piece being moved, but if your only category of interest is "illegal moves", you can't see that difference. Software that knows the rules of the game shouldn't be making either mistake.
- philipwhiuk 2y ago
  Casual players don’t make illegal moves so often that you have to assign them a random move after 10 goes.

Certhas 2y ago

What do you mean by "understand chess"? I think you don't appreciate how good the level of chess displayed here is. It would take an average adult years of dedicated practice to get to 1800. The article doesn't say how often the LLM fails to generate legal moves in ten tries, but it can't be often or the level of play would be much much much worse. As seems often the case, the LLM seems to have a brilliant intuition, but no precise rigid "world model". Of course words like intuition are anthropomorphic. At best a model for what LLMs are doing. But saying "they don't understand" when they can do _this well_ is absurd.

ChoHag 2y ago
[dead]
deleted 2y ago
[deleted]
vundercind 2y ago
> I think you don't appreciate how good the level of chess displayed here is. It would take an average adult years of dedicated practice to get to 1800. Since we already have programs that can do this, that definitely aren’t really thinking and don’t “understand” anything at all, I don’t see the relevance of this part.
- throw310822 2y ago
  It seems you're shifting the discourse here. In the context of LLMs, "to understand" is short for "to have a world model beyond the pure relations between words". In that sense, chess engines do "understand" chess, as they operate on a world model. You can even say that they don't understand anything but chess, which makes them extremely un-intelligent and definitely not capable of understanding as we mean it. However, since an LLM is a generalist engine, if it understands chess there is no reason for it not to understand millions of other concepts and how they relate to each other. And this is the kind of understanding that humans do.
photonthug 2y ago

stuaxo 2y ago

I hate the use of words like "understand" in these conversations. The system understands nothing, it's anthropomorphising it to say it does.

Sharlin 2y ago
Trying to appropriate perfectly well generalizable terms as "something that only humans do" brings zero value to a conversation. It's a "god in the gaps" argument, essentially, and we don't exactly have a great track record of correctly identifying things that are uniquely human.
- fao_ 2y ago
  There's very literally currently a whole wealth of papers proving that LLMs do not understand, cannot reason, and cannot perform basic kinds of reasoning that even a dog can perform. But, ok.
  - TeMPOraL 2y ago
    There's very literally currently a whole wealth of papers proving the opposite, too, so ¯\_(ツ)_/¯.
  - wizzwizz4 2y ago
    There's a whole wealth of papers proving that LLMs do not understand the concepts they write about. That doesn't mean they don't understand grammar – which (as I've claimed since the GPT-2 days) we should, theoretically, expect them to "understand". And what is chess, but a particularly sophisticated grammar?
trashtester 2y ago
I have the same conclusion, but for the opposite reason. It seems like many people tend to use the word "understand" to that not only does someone believe that a given move is good, they also belive that this knowledge comes from a rational evaluation. Some attribute this to a non-material soul/mind, some to quantum mechanics or something else that seems magic, while others never realized the problem with such a belief in the first place. I would claim that when someone can instantly recognize good moves in a given situation, it doesn't come from rationality at all, but from some mix of memory and an intuition that has been build by playing the game many times, with only tiny elements of actual rational thought sprinkled in. This even holds true when these people start to calculate. It is primarily their intuition that prevens them from spending time on all sorts of unlikely moves. And this intuition, I think, represents most of their real "understanding" of the game. This is quite different from understanding something like a mathematical proof, which is almost exclusively inducive logic. And since "understand" so often is associated with rational inductive logic, I think the proper term would be to have "good intuition" when playing the game. And this "good intuition" seems to me precisely the kind of thing that is trained within most neural nets, even LLM's. (Q*, AlphaZero, etc also add the ability to "calculate", meaning traverse the search space efficiently). If we wanted to measure how good this intuition is compared to human chess intuition, we could limit an engine like AlphaZero to only evaluate the same number of moves per second that good humans would be able to, which might be around 10 or so. Maybe with this limitation, the engine wouldn't currently be able to beat the best humans, but even if it reaches a rating of 2000-2500 this way, I would say it has a pretty good intuitive understanding.