4 ms·

If you take a neural network that already knows the basic rules of chess and train it on chess games, you produce a chess engine. From the Wikipedia page on on

If you take a neural network that already knows the basic rules of chess and train it on chess games, you produce a chess engine. From the Wikipedia page on one of the strongest ever[1]: "Like Leela Zero and AlphaGo Zero, Leela Chess Zero starts with no intrinsic chess-specific knowledge other than the basic rules of the game. Leela Chess Zero then learns how to play chess by reinforcement learning from repeated self-play" [1]: https://en.wikipedia.org/wiki/Leela_Chess_Zero https://en.wikipedia.org/wiki/Leela_Chess_Zero

btown 3y ago
As described in the OP's blog post https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c... - one of the incredible things here is that the standard GPT architecture, trained from scratch from PGN strings alone, can intuit the rules of the game from those examples, without any notion of the rules of chess or even that it is playing a game. Leela, by contrast, requires a specialized structure of iterative tree searching to generate move recommendations: https://lczero.org/dev/wiki/technical-explanation-of-leela-chess-zero/ https://lczero.org/dev/wiki/technical-explanation-of-leela-c... Which is not to diminish the work of the Leela team at all! But I find it fascinating that an unmodified GPT architecture can build up internal neural representations that correspond closely to board states, despite not having been designed for that task. As they say, attention may indeed be all you need.
- goatlover 3y ago
  What's the strength of play for the GPT architecture? It's impressive that it figures out the rules, but does it play strong chess? >> As they say, attention may indeed be all you need. I don't think drawing general conclusions about intelligence from a board game is warranted. We didn't evolve to play chess or Go.
  - jksk61 3y ago
    > What's the strength of play for the GPT architecture? It's impressive that it figures out the rules, but does it play strong chess? sometimes it is not a matter of "is it better? is it larger? is it more efficient?", but just a question. mountains are mountains, men are men.
  - foota 3y ago
    I can't see it being superhuman, that's for sure. Chess AI are superhuman because they do vast searches, and I can't see that being replicated by an LLM architecture.
    - macrolime 3y ago
      Any particular reason why that shouldn't work well with fine-tuning of an LLM using reinforcement learning?
    - WithinReason 3y ago
      The apples-to-apples comparison would be comparing an LLM with Leela with search turned off (only using a single board state) According to figure 6b [0] removing MCTS reduces Elo by about 40%, scaling 1800 Elo by 5/3 gives us 3000 Elo which would be superhuman but not as good as e.g. LeelaZero. [0]: https://gwern.net/doc/reinforcement-learning/model/alphago/2017-silver.pdf#page=5 https://gwern.net/doc/reinforcement-learning/model/alphago/2...
      - edgyquant 3y ago
        I don’t think 3000 is superhuman though, it’s peak human as iirc magnus had an Elo of 3000 at one point
      - sscg13 3y ago
        Leela policy is around 2600 elo, or around the level of a strong grandmaster. Note that Go is different from chess since there are no draws, so skill difference is greatly magnified. Elo is always a relative scale (expected score is based on elo difference) so multiplication should not really make sense anyways.
    - adrianN 3y ago
      Chess AI used to dominate by computational power but to my knowledge that is no longer true and the engines beat all but the very strongest players even when run on phone CPUs.
      - 3y ago
  - eigenket 3y ago
    > What's the strength of play for the GPT architecture? Pretty shit for a computer. He says his 50m model reached 1800 Elo (by the way, its Elo and not ELO as the article incorrectly has it, it is named after a Hungarian guy called Elo). It seems to be a bit better than Stockfish level 1 and a bit worse than Stockfish level 2 from the bar graph. Based on what we know I think its not surprising these models can learn to play chess, but they get absolutely smoked by a "real" chess bot like Stockfish or Leela.
    - golol 3y ago
      Afaik his small bot reaches 1300 and gpt-3.5-instruct reaches 1800. We have no idea how much and on what kind of PGNs the Openai model was trained. I heard a rumor that they specifically trained on games up to 1800 before but no idea.
    - jhrmnn 3y ago
      They also say “I left one training for a few more days and it reached 1500 ELO.” I find it quite likely the observed performance is largely limited by the spent compute.
- bananapub 3y ago
  > can intuit the rules of the game from those examples, I am pretty sure a bunch of matrix multiplications can't intuit anything. naively, it doesn't seem very surprising that enormous amounts of self play cause the internal structure to reflect the inputs and outputs?
  - jddj 3y ago
    We really need a list of verbs we're allowed to use when talking about computers and verbs that belong in the magic human/animal-only section
    - sandgiant 3y ago
      You are probably joking, but I think it's actually very important to look at the language we use around LLMs, in order not to get stuck in assumptions and sociological bias associated with a vocabulary usually reserved for "magical" beings, as it were. This goes both ways by the way. I could be convinced that LLMs can achieve something the likes of intuition, but I strongly believe that it is a very different kind of intuition than we normally associate with humans/animals. Usins the same label is thus potentially confusing, and (human pride aside) might even prevent us from appreciating the full scope of what LLMs are capable of.
      - jddj 3y ago
        I think the issue is that we're suddenly trying to pin down something that was previously fine being loosely understood, but without any new information. If someone came to the table with "intuition is the process of a system inferring a likely outcome from given inputs by the process X - not to be confused with matmultuition which is process Y", that might be a reasonable proposal.
  - jstummbillig 3y ago
    > naively, it doesn't seem very surprising that enormous amounts of self play cause the internal structure to reflect the inputs and outputs? Right. Wait, are you talking about AI or humans?
- wredue 3y ago
  I think that “intuit the rules” is just projecting. More likely, the 16 million games just has most of the piece move combinations. It does not know a knight moves in an L. It knows from each square where a knight can move based on 16 million games.
  - baq 3y ago
    You assert something that is a hypothesis for further research in the area. Alternative is that it in fact knows that knights move in an L-shaped fashion. The article is about testing hypotheses like that, except this particular one seems quite hard.
    - empath-nirvana 3y ago
      I think one thing the conversation around LLMs has shown is how poorly defined words like "know" are.
    - HarHarVeryFunny 3y ago
      It'd seem surprising to me if it had really learnt the generalization that knights move in an L-shaped fashion, especially since it's model of the board position seems to be more probabilistic than exact. We don't even know if it's representation of the board is spatial or not (e.g. that columns a & b are adjacent, or that rows 1 & 3 are two rows apart). We also don't know what internal representations of the state of play it's using other than what the author has discovered via probes... Maybe it has other representations effectively representing where pieces are (or what they may do next) other than just the board position. I'm guessing that it's just using all of it's learned representations to recognize patterns where, for example, Nf3 and Nh3 are both statistically likely, and has no spatial understanding of the relationship of these moves. I guess one way to explore this would be to generate a controlled training set where each knight only ever makes a different subset of it's legal (up to) 8 moves depending on which square it is on. Will the model learn a generalization that all L-shaped moves are possible from any square, or will it memorize the different subset of moves that "are possible" from each individual square?
      - 3y ago

btown 3y ago

As described in the OP's blog post https://adamkarvonen.github.io/machine_learning/2024/01/03/chess-world-models.html https://adamkarvonen.github.io/machine_learning/2024/01/03/c... - one of the incredible things here is that the standard GPT architecture, trained from scratch from PGN strings alone, can intuit the rules of the game from those examples, without any notion of the rules of chess or even that it is playing a game. Leela, by contrast, requires a specialized structure of iterative tree searching to generate move recommendations: https://lczero.org/dev/wiki/technical-explanation-of-leela-chess-zero/ https://lczero.org/dev/wiki/technical-explanation-of-leela-c... Which is not to diminish the work of the Leela team at all! But I find it fascinating that an unmodified GPT architecture can build up internal neural representations that correspond closely to board states, despite not having been designed for that task. As they say, attention may indeed be all you need.

goatlover 3y ago
What's the strength of play for the GPT architecture? It's impressive that it figures out the rules, but does it play strong chess? >> As they say, attention may indeed be all you need. I don't think drawing general conclusions about intelligence from a board game is warranted. We didn't evolve to play chess or Go.
- jksk61 3y ago
  > What's the strength of play for the GPT architecture? It's impressive that it figures out the rules, but does it play strong chess? sometimes it is not a matter of "is it better? is it larger? is it more efficient?", but just a question. mountains are mountains, men are men.
- foota 3y ago
  I can't see it being superhuman, that's for sure. Chess AI are superhuman because they do vast searches, and I can't see that being replicated by an LLM architecture.
  - macrolime 3y ago
    Any particular reason why that shouldn't work well with fine-tuning of an LLM using reinforcement learning?
  - WithinReason 3y ago
    The apples-to-apples comparison would be comparing an LLM with Leela with search turned off (only using a single board state) According to figure 6b [0] removing MCTS reduces Elo by about 40%, scaling 1800 Elo by 5/3 gives us 3000 Elo which would be superhuman but not as good as e.g. LeelaZero. [0]: https://gwern.net/doc/reinforcement-learning/model/alphago/2017-silver.pdf#page=5 https://gwern.net/doc/reinforcement-learning/model/alphago/2...
    - edgyquant 3y ago
      I don’t think 3000 is superhuman though, it’s peak human as iirc magnus had an Elo of 3000 at one point
    - sscg13 3y ago
      Leela policy is around 2600 elo, or around the level of a strong grandmaster. Note that Go is different from chess since there are no draws, so skill difference is greatly magnified. Elo is always a relative scale (expected score is based on elo difference) so multiplication should not really make sense anyways.
  - adrianN 3y ago
    Chess AI used to dominate by computational power but to my knowledge that is no longer true and the engines beat all but the very strongest players even when run on phone CPUs.
    - 3y ago
- eigenket 3y ago
  > What's the strength of play for the GPT architecture? Pretty shit for a computer. He says his 50m model reached 1800 Elo (by the way, its Elo and not ELO as the article incorrectly has it, it is named after a Hungarian guy called Elo). It seems to be a bit better than Stockfish level 1 and a bit worse than Stockfish level 2 from the bar graph. Based on what we know I think its not surprising these models can learn to play chess, but they get absolutely smoked by a "real" chess bot like Stockfish or Leela.
  - golol 3y ago
    Afaik his small bot reaches 1300 and gpt-3.5-instruct reaches 1800. We have no idea how much and on what kind of PGNs the Openai model was trained. I heard a rumor that they specifically trained on games up to 1800 before but no idea.
  - jhrmnn 3y ago
    They also say “I left one training for a few more days and it reached 1500 ELO.” I find it quite likely the observed performance is largely limited by the spent compute.
bananapub 3y ago
> can intuit the rules of the game from those examples, I am pretty sure a bunch of matrix multiplications can't intuit anything. naively, it doesn't seem very surprising that enormous amounts of self play cause the internal structure to reflect the inputs and outputs?
- jddj 3y ago
  We really need a list of verbs we're allowed to use when talking about computers and verbs that belong in the magic human/animal-only section
  - sandgiant 3y ago
    You are probably joking, but I think it's actually very important to look at the language we use around LLMs, in order not to get stuck in assumptions and sociological bias associated with a vocabulary usually reserved for "magical" beings, as it were. This goes both ways by the way. I could be convinced that LLMs can achieve something the likes of intuition, but I strongly believe that it is a very different kind of intuition than we normally associate with humans/animals. Usins the same label is thus potentially confusing, and (human pride aside) might even prevent us from appreciating the full scope of what LLMs are capable of.
    - jddj 3y ago
      I think the issue is that we're suddenly trying to pin down something that was previously fine being loosely understood, but without any new information. If someone came to the table with "intuition is the process of a system inferring a likely outcome from given inputs by the process X - not to be confused with matmultuition which is process Y", that might be a reasonable proposal.
- jstummbillig 3y ago
  > naively, it doesn't seem very surprising that enormous amounts of self play cause the internal structure to reflect the inputs and outputs? Right. Wait, are you talking about AI or humans?
wredue 3y ago
I think that “intuit the rules” is just projecting. More likely, the 16 million games just has most of the piece move combinations. It does not know a knight moves in an L. It knows from each square where a knight can move based on 16 million games.
- baq 3y ago
  You assert something that is a hypothesis for further research in the area. Alternative is that it in fact knows that knights move in an L-shaped fashion. The article is about testing hypotheses like that, except this particular one seems quite hard.
  - empath-nirvana 3y ago
    I think one thing the conversation around LLMs has shown is how poorly defined words like "know" are.
  - HarHarVeryFunny 3y ago
    It'd seem surprising to me if it had really learnt the generalization that knights move in an L-shaped fashion, especially since it's model of the board position seems to be more probabilistic than exact. We don't even know if it's representation of the board is spatial or not (e.g. that columns a & b are adjacent, or that rows 1 & 3 are two rows apart). We also don't know what internal representations of the state of play it's using other than what the author has discovered via probes... Maybe it has other representations effectively representing where pieces are (or what they may do next) other than just the board position. I'm guessing that it's just using all of it's learned representations to recognize patterns where, for example, Nf3 and Nh3 are both statistically likely, and has no spatial understanding of the relationship of these moves. I guess one way to explore this would be to generate a controlled training set where each knight only ever makes a different subset of it's legal (up to) 8 moves depending on which square it is on. Will the model learn a generalization that all L-shaped moves are possible from any square, or will it memorize the different subset of moves that "are possible" from each individual square?
    - 3y ago