10 ms·

Acquisition of chess knowledge in AlphaZero

225 points by Bostonian 5y ago

Tenoke 5y ago
>From recorded data, we can see that everyone seemed to play e4 in the 1500s. Over centuries, moves like d4, Nf3 or c4 emerged as credible and fashionable alternatives. The graph they have seems to suggest the opposite - nobody playing e4, just d4 early on (and AZ also gravitating most towards d4).
sarosh 5y ago
The paper itself is here: https://arxiv.org/abs/2111.09259 https://arxiv.org/abs/2111.09259 with the key conclusion that "Examining the evolution of human concepts using probing showed that many human concepts can be accurately regressed from the AZ network after training, even though AlphaZero has never seen a human game of chess, and there is no objective function promoting human-like play or activations" and "[t]he fact that human concepts can be located even in a superhuman system trained by self-play broadens the range of systems in which we should expect to find human-understandable concepts"
- Santosh83 5y ago
  We are quite literally alpha zero ourselves. We developed chess by playing ourselves. I don't get what is particularly surprising that there are not infinite ways to play good chess?
  - ogogmad 5y ago
    It's not a priori obvious that a machine can learn to play chess in a similar way to the way a human can. I suppose if AGI gets achieved then some of this will seem predictable in hindsight. But we don't yet know if it's achievable.
    - pmontra 5y ago
      Maybe for more complex games, example: economics. But chess is pretty simple compared to seven billion people going their own ways. On the other side, AlphaZero and all the previous Deep Mind's AIs played some very different go than human players did. Every single move was not unheard of but they were dismissed by human masters as not good. The most noticeable contribution was the total change of how to play corner invasions. By looking at game records we can draw a line between pre and post AlphaGo. Maybe go is a little bit more wide than chess and AIs have more space to find novel plays.
    - alchemst 5y ago
      We (humans, AlphaZero) are both distilling the rules down. Some rules are easily distilled (a queen is worth more than pawn) some are more difficult (mobility, knight vs bishop). I would expect the easily distilled rules to be discovered by both of us. What I think would be cool is trying to distill or revise better guidelines from what AlphaZero does.
  - V-2 5y ago
    Human minds didn't evolve for playing chess. So there was a possibility that there are "hardware" limitations that prevent us from playng it the way it should be (give or take the inevitable error margin).
- kevinventullo 5y ago
  The human-understandable concepts are there, but I think finding them is still very hard, right? I.e. it doesn’t sound like the system including AZ was able to identify and articulate the human concepts such as king safety a priori.
  - iratewizard 5y ago
    A game like chess is like two competing mathematical models. The human understanding of it is really just noticing patterns that exist in the model. I would hope an ML algorithm finds similar patterns.
    - deleted 5y ago
      [deleted]
- iechoz6H 5y ago
  Aren't the shared concepts simply predicated on 'the rules of chess'? No surprise that an algorithm and a 'rational actor' share similar themes when both are constrained by the same rules.
  - baq 5y ago
    the concept of material isn't found in the rules and yet was discovered independently during self-play.
    - magicalhippo 5y ago
      > the concept of material isn't found in the rules It's not spelled out directly, but it is a pretty low-hanging fruit. After all, the rules dictate that the number of pieces on the board never increases. The rules also dictate the different kinds of moves allowed by each piece. Together it seems to me those rules invariably lead to pieces not having the same value, aka the concept of material.
- posterboy 5y ago
  Can this be explained by the fact that the rules of chess evolved accomodating certain preferences that human players developed over the course of time? Take a much simpler game like paper, rock scissors, where certain strategies exist as well (I hear). This should be much easier to analyze. Can somebody apply alphaZero to rock, paper, scissors, please?
AutumnCurtain 5y ago
>AlphaZero’s neural network evaluation function doesn’t have the same level of structure as Stockfish’s evaluation function: the Stockfish function breaks down a position into a range of concepts (for example king safety, mobility, and material) and combines these concepts to reach an overall evaluation of the position. AlphaZero, on the other hand, outputs a value function ranging from -1 (defeat is certain) to +1 (victory is guaranteed) with no explicitly-stated intermediate steps. Although the neural network evaluation function is computing something, it’s not clear what. I find it fascinating that we can create something that can "learn" in some sense in a way we don't fully understand. Or am I misunderstanding the meaning of this passage?
- beaconstudios 5y ago
  We do understand how they learn, but the structure of their knowledge is currently hard to discern in the same way that you can't get thoughts or memories out of somebody's head with a MRI scan.
  - sowbug 5y ago
    It's interesting to think that if we create a human-level artificial intelligence using ML, we'll probably have no idea how it works.
    - ludamad 5y ago
      It's okay, it won't either
    - beaconstudios 5y ago
      To be fair, we have no idea how we think either!
- noxvilleza 5y ago
  The way in which I think about it is that "learning" (in humans) often occurs is in the form of heuristics - certain rules or shortcuts which are not guaranteed to be optimal, but do a good job at simplifying (the search-state of) the problem. Instead of trying many simple comparisons between legal moves, we instead use a heuristic to score them. Non-NN chess engines are really all about doing a good job at balancing these heuristics (like it says above). There's fundamentally an upper bound in how good the overall approach can be, simply because the underlying heuristics are defined and selected by humans: there could be many more heuristics that haven't been considered. This version of "learning" is really just removing human forced constraints which were introduced to help learning, by starting from a position without bias.
porphyra 5y ago
Why link to Chessbase instead of to the arxiv paper directly? https://arxiv.org/abs/2111.09259 https://arxiv.org/abs/2111.09259 Chessbase is notoriously sketchy and there is a lawsuit against them for blatantly stealing GPL code. https://stockfishchess.org/blog/2021/our-lawsuit-against-chessbase/ https://stockfishchess.org/blog/2021/our-lawsuit-against-che...
- lacker 5y ago
  The Chessbase article is a lot more readable than the arxiv paper. Yeah, they did some bad GPL stuff, but that just isn't really relevant to this particular discussion, and the chessbase link is better for the general reader.
  - throwaway6977 5y ago
    Personally, I'd rather not support dishonest, but pretty outlets over direct sources.
    - quaintdev 5y ago
      Somebody screwed up somewhere, so entire organization must suffer for all eternity?
      - kadoban 5y ago
        How big and disparate of an organization is some random chess website?
        zamadatix 5y ago
        ~30 full time employees. ChessBase isn't just some random person's hobby blog site it's a pretty big deal in the world of chess.
      - throwaway6977 5y ago
        No? But if I have the opportunity to exercise my preference- I will feel good about doing so, and suggest others do the same.
tedsanders 5y ago
Sounds like when you teach a neural network chess, it ends up learning many of the same concepts that we do. To me, this is a great illustration that superhuman intelligence is not magic. When a superhuman AI plays chess, the moves usually make sense to an expert. And even if they don't immediately make sense, they usually make sense once the expert plays out some lines to see what happens. Superhuman AIs have crushed humans at chess not by discovering new, wildly counterintuitive openings that dumb humans missed (it actually plays many of the same openings). Rather, it just does what humans do - win pieces, keep your king safe, attack, gain space, restrict your opponent's moves - but better and more consistently. This paper builds on that concept and finds that not only are a superhuman AI's moves usually understandable by experts, but even some of the superhuman AI's internal representations are understandable by experts (!). In the 5th century BCE, Greek philosophers thought the Earth was a sphere. Although they were eventually improved upon by Newton's ellipsoid, they still had arrived at the right concept. And here I think we see the same thing: although a superhuman AI 'understands' chess better than human experts, its understanding still builds upon the same concepts. Thinking broadly, like, suppose we invent some superhuman AGI for $100B in 2100. And then we ask it how to optimize our paperclip production. I don't think its recommendations will be magic. Paperclip production is a fairly well understood problem: you bring in some metal, you reshape it, you send it out. It's probably gonna have much of the same advice we've already figured out: invest in good equipment, take advantage of economies of scale, choose cheap but effective metals, put factories near sources of energy and labor and raw materials, ship out paperclips in a hierarchical network with caches along the way to supply unanticipated demand, etc. Like, an AGI's suggestions may well be better than a management consultant's, but it's not like it's going to wave a magic wand and flip the laws of physics. Rather, I expect it to win within the same frameworks we've developed, but by doing things better and more consistently than humans.
- barry-cotter 5y ago
  > Rather, it just does what [less successful people] do - ... - but better and more consistently. Reminds me of comparisons between Renaissance and other quant funds. If you get enough better it looks like what you're doing is magic until you analyse what was done and see they have better systems and people. Sometimes, magic is just someone spending more time on something than anyone else might reasonably expect - Teller
a_shovel 5y ago
The sci-fi trope that aliens or superintelligent AIs will have thought processes completely different and incomprehensible to humans is probably not true. If the aliens live in our universe, evolved on a planet with limited resources, need food, breathe, etc., then they'll likely have a lot in common with Earth animals psychologically. If the AI is tasked with doing something that humans also do, then its tactics will be at least recognizable to an expert in that field. Moreso in well-understood systems such as chess. Is it really plausible that an advanced chess AI's tactics would have no concept of threats or king safety? The same tactics apply to it as do to us.
- rtpg 5y ago
  I don't want to be too dismissive of the work going into this, but Chess isn't that huge of a search space. Games can quickly end in many scenarios from certain points. Does having an evaluation system that takes into account king safety only as a consequence of having taken the entire game state into account, mean that it understands king safety? Can you extract these tactics from the AI? Can it explain the moves? I would be interested in seeing work here that can provide some comprehensibility to the correlations, as that's how we can get to actual novel intuition. "The computer is good with this neural network due to pattern matching" is less actionable than "the computer is good and found this kind of pattern, and we think it relates to this". EDIT: I think that if anything Chess is an interesting example of humans emulating computers. People sitting around thinking about variations, trying to snuff out what is good/bad. Granted, the fact that you're playing against other humans is pretty important at a higher level, but it's centuries of people sitting down and running through the search space, using our approximations for what are good and bad.
- csee 5y ago
  I don't share this view. They would have had a lot in common at a point in time in their history, but who's to say the consequences of millions of years of additional evolution post-energy abundance, let alone the consequences of embryo selection and genetic engineering? We've had a few tens of thousand years of evolution with relative (but still not really) caloric abundance and look how much bigger our brains got. Add nuclear fusion and an extra 50 million years to that nascent process. I think chess strategy is a flawed way to be thinking about this because we designed the rules and the solution space to be comprehensible to us. It's not surprising that we can understand AI strategy. That simply follows from the solution space itself being small and comprehensible.
njarboe 5y ago
I think it pretty cool how AlphaZero does not converge to a single best opening move and its variation of opening moves is higher than humans. Seeing people analyze AlphaZero games on YouTube, it plays a more open and aggressive game of chess and goes for the win instead of the very defensive play of most of the top players today. Some people hope that if people study how AlphaZero plays chess (the best chess player in the world) to improve their game it might make chess more interesting as most games at the high level now end in draws.
- Y_Y 5y ago
  It's a somewhat counterintuitive result from game theory that many of the best strategies we know to various non-trivial games are "mixed strategies". This refers to a strategy (presumably) like the one by AlphaZero here, where moves are chosen from a probabilistic distribution, rather than just picking "the best one". The sub-optimal move choice is counterbalanced by the unpredictability.
  - jstx1 5y ago
    Mixed strategies in game theory don’t involve making sub-optimal decisions for the sake of unpredictability. At equilibrium every sub-strategy of the mix is a best response (you have multiple best responses).
    - Y_Y 5y ago
      I think we're differing on unimportant semantics. Consider that optimal mixed strategies don't generally assign equal probability to each choice. If they were all equally good (say, as a pure strategy response) then they shouldn't have different probabilities.
      - jstx1 5y ago
        I don't think so. The payoffs depend on the mix of the opponent. You can't say "the probability of sub-strategy x is higher so it must be a better strategy". If it really was a better strategy, you would pick it all the time, not in some mix.
phreeza 5y ago
What I want to know is if it is possible to do the opposite: extract novel, useful and human-understandable heuristics from a chess engine?
- karol 5y ago
  Yes, it has been done with go and Deep Mind. The program found novel patterns of playing that were not known. I don't see why this wouldn't happen with chess.
  - phreeza 5y ago
    I was thinking more along the lines of automated discovery of such heuristics.
redsummer 5y ago
Can the highest rated Chess Engine beat a human if each of it's moves are chosen alternatively by a 1000 strength engine and maximum (3500?) strength engine?

225 points by Bostonian 5y ago

>From recorded data, we can see that everyone seemed to play e4 in the 1500s. Over centuries, moves like d4, Nf3 or c4 emerged as credible and fashionable alternatives. The graph they have seems to suggest the opposite - nobody playing e4, just d4 early on (and AZ also gravitating most towards d4).

sarosh 5y ago

The paper itself is here: https://arxiv.org/abs/2111.09259 https://arxiv.org/abs/2111.09259 with the key conclusion that "Examining the evolution of human concepts using probing showed that many human concepts can be accurately regressed from the AZ network after training, even though AlphaZero has never seen a human game of chess, and there is no objective function promoting human-like play or activations" and "[t]he fact that human concepts can be located even in a superhuman system trained by self-play broadens the range of systems in which we should expect to find human-understandable concepts"

Santosh83 5y ago
We are quite literally alpha zero ourselves. We developed chess by playing ourselves. I don't get what is particularly surprising that there are not infinite ways to play good chess?
- ogogmad 5y ago
  It's not a priori obvious that a machine can learn to play chess in a similar way to the way a human can. I suppose if AGI gets achieved then some of this will seem predictable in hindsight. But we don't yet know if it's achievable.
  - pmontra 5y ago
    Maybe for more complex games, example: economics. But chess is pretty simple compared to seven billion people going their own ways. On the other side, AlphaZero and all the previous Deep Mind's AIs played some very different go than human players did. Every single move was not unheard of but they were dismissed by human masters as not good. The most noticeable contribution was the total change of how to play corner invasions. By looking at game records we can draw a line between pre and post AlphaGo. Maybe go is a little bit more wide than chess and AIs have more space to find novel plays.
  - alchemst 5y ago
    We (humans, AlphaZero) are both distilling the rules down. Some rules are easily distilled (a queen is worth more than pawn) some are more difficult (mobility, knight vs bishop). I would expect the easily distilled rules to be discovered by both of us. What I think would be cool is trying to distill or revise better guidelines from what AlphaZero does.
- V-2 5y ago
  Human minds didn't evolve for playing chess. So there was a possibility that there are "hardware" limitations that prevent us from playng it the way it should be (give or take the inevitable error margin).
kevinventullo 5y ago
The human-understandable concepts are there, but I think finding them is still very hard, right? I.e. it doesn’t sound like the system including AZ was able to identify and articulate the human concepts such as king safety a priori.
- iratewizard 5y ago
  A game like chess is like two competing mathematical models. The human understanding of it is really just noticing patterns that exist in the model. I would hope an ML algorithm finds similar patterns.
  - deleted 5y ago
    [deleted]
iechoz6H 5y ago
Aren't the shared concepts simply predicated on 'the rules of chess'? No surprise that an algorithm and a 'rational actor' share similar themes when both are constrained by the same rules.
- baq 5y ago
  the concept of material isn't found in the rules and yet was discovered independently during self-play.
  - magicalhippo 5y ago
    > the concept of material isn't found in the rules It's not spelled out directly, but it is a pretty low-hanging fruit. After all, the rules dictate that the number of pieces on the board never increases. The rules also dictate the different kinds of moves allowed by each piece. Together it seems to me those rules invariably lead to pieces not having the same value, aka the concept of material.
posterboy 5y ago
Can this be explained by the fact that the rules of chess evolved accomodating certain preferences that human players developed over the course of time? Take a much simpler game like paper, rock scissors, where certain strategies exist as well (I hear). This should be much easier to analyze. Can somebody apply alphaZero to rock, paper, scissors, please?

AutumnCurtain 5y ago

>AlphaZero’s neural network evaluation function doesn’t have the same level of structure as Stockfish’s evaluation function: the Stockfish function breaks down a position into a range of concepts (for example king safety, mobility, and material) and combines these concepts to reach an overall evaluation of the position. AlphaZero, on the other hand, outputs a value function ranging from -1 (defeat is certain) to +1 (victory is guaranteed) with no explicitly-stated intermediate steps. Although the neural network evaluation function is computing something, it’s not clear what. I find it fascinating that we can create something that can "learn" in some sense in a way we don't fully understand. Or am I misunderstanding the meaning of this passage?

beaconstudios 5y ago
We do understand how they learn, but the structure of their knowledge is currently hard to discern in the same way that you can't get thoughts or memories out of somebody's head with a MRI scan.
- sowbug 5y ago
  It's interesting to think that if we create a human-level artificial intelligence using ML, we'll probably have no idea how it works.
  - ludamad 5y ago
    It's okay, it won't either
  - beaconstudios 5y ago
    To be fair, we have no idea how we think either!
noxvilleza 5y ago
The way in which I think about it is that "learning" (in humans) often occurs is in the form of heuristics - certain rules or shortcuts which are not guaranteed to be optimal, but do a good job at simplifying (the search-state of) the problem. Instead of trying many simple comparisons between legal moves, we instead use a heuristic to score them. Non-NN chess engines are really all about doing a good job at balancing these heuristics (like it says above). There's fundamentally an upper bound in how good the overall approach can be, simply because the underlying heuristics are defined and selected by humans: there could be many more heuristics that haven't been considered. This version of "learning" is really just removing human forced constraints which were introduced to help learning, by starting from a position without bias.

porphyra 5y ago

Why link to Chessbase instead of to the arxiv paper directly? https://arxiv.org/abs/2111.09259 https://arxiv.org/abs/2111.09259 Chessbase is notoriously sketchy and there is a lawsuit against them for blatantly stealing GPL code. https://stockfishchess.org/blog/2021/our-lawsuit-against-chessbase/ https://stockfishchess.org/blog/2021/our-lawsuit-against-che...

lacker 5y ago
The Chessbase article is a lot more readable than the arxiv paper. Yeah, they did some bad GPL stuff, but that just isn't really relevant to this particular discussion, and the chessbase link is better for the general reader.
- throwaway6977 5y ago
  Personally, I'd rather not support dishonest, but pretty outlets over direct sources.
  - quaintdev 5y ago
    Somebody screwed up somewhere, so entire organization must suffer for all eternity?
    - kadoban 5y ago
      How big and disparate of an organization is some random chess website?
      - zamadatix 5y ago
        ~30 full time employees. ChessBase isn't just some random person's hobby blog site it's a pretty big deal in the world of chess.
    - throwaway6977 5y ago
      No? But if I have the opportunity to exercise my preference- I will feel good about doing so, and suggest others do the same.

tedsanders 5y ago

Sounds like when you teach a neural network chess, it ends up learning many of the same concepts that we do. To me, this is a great illustration that superhuman intelligence is not magic. When a superhuman AI plays chess, the moves usually make sense to an expert. And even if they don't immediately make sense, they usually make sense once the expert plays out some lines to see what happens. Superhuman AIs have crushed humans at chess not by discovering new, wildly counterintuitive openings that dumb humans missed (it actually plays many of the same openings). Rather, it just does what humans do - win pieces, keep your king safe, attack, gain space, restrict your opponent's moves - but better and more consistently. This paper builds on that concept and finds that not only are a superhuman AI's moves usually understandable by experts, but even some of the superhuman AI's internal representations are understandable by experts (!). In the 5th century BCE, Greek philosophers thought the Earth was a sphere. Although they were eventually improved upon by Newton's ellipsoid, they still had arrived at the right concept. And here I think we see the same thing: although a superhuman AI 'understands' chess better than human experts, its understanding still builds upon the same concepts. Thinking broadly, like, suppose we invent some superhuman AGI for $100B in 2100. And then we ask it how to optimize our paperclip production. I don't think its recommendations will be magic. Paperclip production is a fairly well understood problem: you bring in some metal, you reshape it, you send it out. It's probably gonna have much of the same advice we've already figured out: invest in good equipment, take advantage of economies of scale, choose cheap but effective metals, put factories near sources of energy and labor and raw materials, ship out paperclips in a hierarchical network with caches along the way to supply unanticipated demand, etc. Like, an AGI's suggestions may well be better than a management consultant's, but it's not like it's going to wave a magic wand and flip the laws of physics. Rather, I expect it to win within the same frameworks we've developed, but by doing things better and more consistently than humans.

barry-cotter 5y ago
> Rather, it just does what [less successful people] do - ... - but better and more consistently. Reminds me of comparisons between Renaissance and other quant funds. If you get enough better it looks like what you're doing is magic until you analyse what was done and see they have better systems and people. Sometimes, magic is just someone spending more time on something than anyone else might reasonably expect - Teller

a_shovel 5y ago

The sci-fi trope that aliens or superintelligent AIs will have thought processes completely different and incomprehensible to humans is probably not true. If the aliens live in our universe, evolved on a planet with limited resources, need food, breathe, etc., then they'll likely have a lot in common with Earth animals psychologically. If the AI is tasked with doing something that humans also do, then its tactics will be at least recognizable to an expert in that field. Moreso in well-understood systems such as chess. Is it really plausible that an advanced chess AI's tactics would have no concept of threats or king safety? The same tactics apply to it as do to us.

rtpg 5y ago
I don't want to be too dismissive of the work going into this, but Chess isn't that huge of a search space. Games can quickly end in many scenarios from certain points. Does having an evaluation system that takes into account king safety only as a consequence of having taken the entire game state into account, mean that it understands king safety? Can you extract these tactics from the AI? Can it explain the moves? I would be interested in seeing work here that can provide some comprehensibility to the correlations, as that's how we can get to actual novel intuition. "The computer is good with this neural network due to pattern matching" is less actionable than "the computer is good and found this kind of pattern, and we think it relates to this". EDIT: I think that if anything Chess is an interesting example of humans emulating computers. People sitting around thinking about variations, trying to snuff out what is good/bad. Granted, the fact that you're playing against other humans is pretty important at a higher level, but it's centuries of people sitting down and running through the search space, using our approximations for what are good and bad.
csee 5y ago
I don't share this view. They would have had a lot in common at a point in time in their history, but who's to say the consequences of millions of years of additional evolution post-energy abundance, let alone the consequences of embryo selection and genetic engineering? We've had a few tens of thousand years of evolution with relative (but still not really) caloric abundance and look how much bigger our brains got. Add nuclear fusion and an extra 50 million years to that nascent process. I think chess strategy is a flawed way to be thinking about this because we designed the rules and the solution space to be comprehensible to us. It's not surprising that we can understand AI strategy. That simply follows from the solution space itself being small and comprehensible.

njarboe 5y ago

I think it pretty cool how AlphaZero does not converge to a single best opening move and its variation of opening moves is higher than humans. Seeing people analyze AlphaZero games on YouTube, it plays a more open and aggressive game of chess and goes for the win instead of the very defensive play of most of the top players today. Some people hope that if people study how AlphaZero plays chess (the best chess player in the world) to improve their game it might make chess more interesting as most games at the high level now end in draws.

Y_Y 5y ago
It's a somewhat counterintuitive result from game theory that many of the best strategies we know to various non-trivial games are "mixed strategies". This refers to a strategy (presumably) like the one by AlphaZero here, where moves are chosen from a probabilistic distribution, rather than just picking "the best one". The sub-optimal move choice is counterbalanced by the unpredictability.
- jstx1 5y ago
  Mixed strategies in game theory don’t involve making sub-optimal decisions for the sake of unpredictability. At equilibrium every sub-strategy of the mix is a best response (you have multiple best responses).
  - Y_Y 5y ago
    I think we're differing on unimportant semantics. Consider that optimal mixed strategies don't generally assign equal probability to each choice. If they were all equally good (say, as a pure strategy response) then they shouldn't have different probabilities.
    - jstx1 5y ago
      I don't think so. The payoffs depend on the mix of the opponent. You can't say "the probability of sub-strategy x is higher so it must be a better strategy". If it really was a better strategy, you would pick it all the time, not in some mix.

phreeza 5y ago

What I want to know is if it is possible to do the opposite: extract novel, useful and human-understandable heuristics from a chess engine?

karol 5y ago
Yes, it has been done with go and Deep Mind. The program found novel patterns of playing that were not known. I don't see why this wouldn't happen with chess.
- phreeza 5y ago
  I was thinking more along the lines of automated discovery of such heuristics.

redsummer 5y ago

Can the highest rated Chess Engine beat a human if each of it's moves are chosen alternatively by a 1000 strength engine and maximum (3500?) strength engine?