3 ms·

Question: does Stockfish beat more generic systems like AlphaZero?

by piker 16d ago

tumdum_ 16d ago
I think so, see the tournament results: https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship#Tournament_results https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship#...
mattr03 16d ago
yes and it has for a very long time. Stockfish took the neural net approach after AlphaZero showed it was a good idea as all modern chess engines have
- lynx97 16d ago
  They basically replaced the static position evaluation function with a NN approach and kept the rest of the tree search mostly intact. I ran fishtest on an old cluster for a while. Was nice to watch the stockfish team methodically improving things. Basically, every commit has to show it can beat the main branch before it gets merged.
- Sopel 16d ago
  [dead]
gwd 16d ago
My understanding is that AlphaZero only really existed for a year or two; there's no objective way to compare it at the moment. Leela Zero tried to open-source that work, but Stockfish incorporated a number of improvements from AlphaZero, including a neural network and a different search method, and consistently beats Leela Zero. I have a book, "Game Changer", in which a chess expert calls out several instances where AlphaZero made moves surprising at the time; situations where all chess engines rated things one way and AlphaZero rated them differently. When I enter them into Stockfish now, it usually rates things more similarly to the way AlphaZero did, and often chooses the move chosen by AlphaZero. The only real test of course would be to dig up AlphaZero and run it again; but I think based on the evidence we have, Stockfish of 2026 would probably trounce AlphaZero of 2018 with equivalent compute available.
- sinuhe69 16d ago
  I guess someone can twit Hassabis and ask him :D He certainly has access.
  - NooneAtAll3 16d ago
    who is Hassabis?
    - xenocratus 16d ago
      Demis Hassabis [1], co-founder of DeepMind [1] https://en.wikipedia.org/wiki/Demis_Hassabis https://en.wikipedia.org/wiki/Demis_Hassabis
- kzrdude 16d ago
  Isn't one problem that it's hard to determine what equivalent compute is, for CPU search vs a neural net based engine like AZ or Leela?
  - gwd 16d ago
    One way would be to calculate a cost per game, factoring in both electricity and an amortized cost of the hardware, maybe having a penalty too for extra time run (e.g., if focusing only on hardware depreciation and electricity, 1 minute of TPU would translate to 2 weeks of CPU, that 2 weeks of waiting still costs you something). Obviously this isn't stable, as relative prices of GPUs and memory shift over time, and it's somewhat sensitive to setup; but done right it's probably more "what a user actually wants to know", in terms of what it would take to get equivalent performance.
  - zarzavat 16d ago
    It's more fundamental than that. AlphaZero is a shallower search with a heavier evaluation function. Stockfish is a deeper search with a lighter evaluation function. In chess, depth usually wins because of how narrow the search tree is compared e.g. to Go.
sapiogram 16d ago
Technically nobody knows, because AlphaZero was never made public. In practice, we know Stockfish 19 would destroy it, because it destroys open-source reimplementations of AlphaZero.
noir_lord 16d ago
> like AlphaZero? Unknown, it was only around for a little while however there is Lila which is based on the same ideas as AlphaZero (and also a fantastic project generally) they generally trade blows with Stockfish coming out very slightly ahead. That said Stockfish also uses an NN (NNUE) these days so the line is getting more blurry.
kqp 16d ago
Like a red headed stepchild.
NitpickLawyer 16d ago
On average, yes. Stockfish is the strongest engine and beats AZ-like implementations like Lc0 and the like. But on a game to game basis Lc0 can still win some games, depending on the starting position. It's rare that Lc0 can win both black and white starting from the same position, tho. There are a few yt content creators that cover great engine games, if you're curious.
eklitzke 16d ago
It's widely believed so because Stockfish finds all of the right (or better moves) in the games from the original paper.
Sopel 16d ago
AlphaZero does not exist, as far as modern computer chess is concerned
- VulgarExigency 16d ago
  Leela proved that the ideas worked, regardless of the conditions that the AlphaZero vs Stockfish matches were run in. What's funny is that if computer chess had paid more attention to computer shogi, Stockfish could have already been using NNUE by that time, and not lost to AlphaZero. But perhaps that would have stifled Leela's development, and would have led to worse training data for Stockfish in that parallel universe's present day, making it weaker overall.

Question: does Stockfish beat more generic systems like AlphaZero?

by piker 16d ago

Question: does Stockfish beat more generic systems like AlphaZero?

I think so, see the tournament results: https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship#Tournament_results https://en.wikipedia.org/wiki/Top_Chess_Engine_Championship#...

mattr03 16d ago

yes and it has for a very long time. Stockfish took the neural net approach after AlphaZero showed it was a good idea as all modern chess engines have

lynx97 16d ago
They basically replaced the static position evaluation function with a NN approach and kept the rest of the tree search mostly intact. I ran fishtest on an old cluster for a while. Was nice to watch the stockfish team methodically improving things. Basically, every commit has to show it can beat the main branch before it gets merged.
Sopel 16d ago
[dead]

gwd 16d ago

My understanding is that AlphaZero only really existed for a year or two; there's no objective way to compare it at the moment. Leela Zero tried to open-source that work, but Stockfish incorporated a number of improvements from AlphaZero, including a neural network and a different search method, and consistently beats Leela Zero. I have a book, "Game Changer", in which a chess expert calls out several instances where AlphaZero made moves surprising at the time; situations where all chess engines rated things one way and AlphaZero rated them differently. When I enter them into Stockfish now, it usually rates things more similarly to the way AlphaZero did, and often chooses the move chosen by AlphaZero. The only real test of course would be to dig up AlphaZero and run it again; but I think based on the evidence we have, Stockfish of 2026 would probably trounce AlphaZero of 2018 with equivalent compute available.

sinuhe69 16d ago
I guess someone can twit Hassabis and ask him :D He certainly has access.
- NooneAtAll3 16d ago
  who is Hassabis?
  - xenocratus 16d ago
    Demis Hassabis [1], co-founder of DeepMind [1] https://en.wikipedia.org/wiki/Demis_Hassabis https://en.wikipedia.org/wiki/Demis_Hassabis
kzrdude 16d ago
Isn't one problem that it's hard to determine what equivalent compute is, for CPU search vs a neural net based engine like AZ or Leela?
- gwd 16d ago
  One way would be to calculate a cost per game, factoring in both electricity and an amortized cost of the hardware, maybe having a penalty too for extra time run (e.g., if focusing only on hardware depreciation and electricity, 1 minute of TPU would translate to 2 weeks of CPU, that 2 weeks of waiting still costs you something). Obviously this isn't stable, as relative prices of GPUs and memory shift over time, and it's somewhat sensitive to setup; but done right it's probably more "what a user actually wants to know", in terms of what it would take to get equivalent performance.
- zarzavat 16d ago
  It's more fundamental than that. AlphaZero is a shallower search with a heavier evaluation function. Stockfish is a deeper search with a lighter evaluation function. In chess, depth usually wins because of how narrow the search tree is compared e.g. to Go.

sapiogram 16d ago

Technically nobody knows, because AlphaZero was never made public. In practice, we know Stockfish 19 would destroy it, because it destroys open-source reimplementations of AlphaZero.

noir_lord 16d ago

> like AlphaZero? Unknown, it was only around for a little while however there is Lila which is based on the same ideas as AlphaZero (and also a fantastic project generally) they generally trade blows with Stockfish coming out very slightly ahead. That said Stockfish also uses an NN (NNUE) these days so the line is getting more blurry.

kqp 16d ago

Like a red headed stepchild.

NitpickLawyer 16d ago

On average, yes. Stockfish is the strongest engine and beats AZ-like implementations like Lc0 and the like. But on a game to game basis Lc0 can still win some games, depending on the starting position. It's rare that Lc0 can win both black and white starting from the same position, tho. There are a few yt content creators that cover great engine games, if you're curious.

eklitzke 16d ago

It's widely believed so because Stockfish finds all of the right (or better moves) in the games from the original paper.

Sopel 16d ago

AlphaZero does not exist, as far as modern computer chess is concerned

VulgarExigency 16d ago
Leela proved that the ideas worked, regardless of the conditions that the AlphaZero vs Stockfish matches were run in. What's funny is that if computer chess had paid more attention to computer shogi, Stockfish could have already been using NNUE by that time, and not lost to AlphaZero. But perhaps that would have stifled Leela's development, and would have led to worse training data for Stockfish in that parallel universe's present day, making it weaker overall.